Fix tag normalization and migration not removing duplicate tags

[Gargron]

Fix #11428

[mayaeh]

The following error occurred in my environment:
$ RAILS_ENV=production bundle exec rake db:migrate
Migrating to AddCaseInsensitiveIndexToTags (20190726175042)
== 20190726175042 AddCaseInsensitiveIndexToTags: migrating ====================
-- execute("CREATE UNIQUE INDEX CONCURRENTLY index_tags_on_name_lower ON tags (lower(name))")
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::UniqueViolation: ERROR:  could not create unique index "index_tags_on_name_lower"
DETAIL:  Key (lower(name::text))=(pawoo人増えたし自己紹介しようぜ) is duplicated.
: CREATE UNIQUE INDEX CONCURRENTLY index_tags_on_name_lower ON tags (lower(name))

However, this may be due to my trial and error...

EDIT:
I'm sorry, this was due to my lack of confirmation. I confirm one more time.

[mayaeh]

I applied this PR and ran db:migrate.
The following error occurred in my environment:

$ RAILS_ENV=production bundle exec rake db:migrate
Migrating to AddCaseInsensitiveIndexToTags (20190726175042)
== 20190726175042 AddCaseInsensitiveIndexToTags: migrating ====================
-- execute("UPDATE accounts_tags SET tag_id = 887 WHERE tag_id IN (2368)")
   -> 0.0035s
-- execute("UPDATE statuses_tags SET tag_id = 887 WHERE tag_id IN (2368)")
   -> 0.0078s
-- execute("UPDATE account_tag_stats SET tag_id = 887 WHERE tag_id IN (2368)")
   -> 0.0049s
-- execute("UPDATE featured_tags SET tag_id = 887 WHERE tag_id IN (2368)")
   -> 0.0008s
-- execute("UPDATE accounts_tags SET tag_id = 1814 WHERE tag_id IN (1813)")
   -> 0.0004s
-- execute("UPDATE statuses_tags SET tag_id = 1814 WHERE tag_id IN (1813)")
   -> 0.0032s
-- execute("UPDATE account_tag_stats SET tag_id = 1814 WHERE tag_id IN (1813)")
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "index_account_tag_stats_on_tag_id"
DETAIL:  Key (tag_id)=(1814) already exists.
: UPDATE account_tag_stats SET tag_id = 1814 WHERE tag_id IN (1813)

1813 and 1814 are as follows:

$ bundle exec rails console

Tag.where("name like 'pawoo人増えたし自己紹介しようぜ%'").select('id','name')
=> #<ActiveRecord::Relation [#<Tag id: 192, name: "pawoo人増えたし自己紹介しようぜ">, #<Tag id: 1687, name: "pawoo人増えたし自己紹介しようぜ">, #<Tag id: 1813, name: "pawoo人増えたし自己紹介しよう">, #<Tag id: 1814, name: "pawoo人増えたし自己紹介しよう">]>

Postscript: I'm sorry, I remembered that I was deleting id: 2368 when I was trial and error.
I ran Tag.find(2368).destroy with rails console.

[Gargron]

I have updated the migration to omit updating rows where a row with the canonical tag already exists. Can you please try again @mayaeh?

[mayaeh]

Thank you! I will try now.

[mayaeh]

Errors still occur in my environment.

$ RAILS_ENV=production bundle exec rake db:migrate
Migrating to AddCaseInsensitiveIndexToTags (20190726175042)
== 20190726175042 AddCaseInsensitiveIndexToTags: migrating ====================
-- execute("UPDATE accounts_tags AS t0 SET tag_id = 1813 WHERE tag_id IN (1814) 
AND NOT EXISTS (SELECT t1.tag_id FROM accounts_tags AS t1 WHERE t1.tag_id = 1813
 AND t1.account_id = t0.account_id)")
   -> 0.0013s
-- execute("UPDATE statuses_tags AS t0 SET tag_id = 1813 WHERE tag_id IN (1814) 
AND NOT EXISTS (SELECT t1.tag_id FROM statuses_tags AS t1 WHERE t1.tag_id = 1813 
AND t1.status_id = t0.status_id)")
   -> 0.0011s
-- execute("UPDATE account_tag_stats AS t0 SET tag_id = 1813 WHERE tag_id IN 
(1814)  AND NOT EXISTS (SELECT t1.tag_id FROM account_tag_stats AS t1 WHERE t1.tag_
id = 1813 AND t1.account_id = t0.account_id)")
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::UndefinedColumn: ERROR:  column t1.account_id does not exist
LINE 1: ...ccount_tag_stats AS t1 WHERE t1.tag_id = 1813 AND t1.account...
                                                             ^
: UPDATE account_tag_stats AS t0 SET tag_id = 1813 WHERE tag_id IN (1814)  
AND NOT EXISTS (SELECT t1.tag_id FROM account_tag_stats AS t1 WHERE t1.tag_id = 1813 
AND t1.account_id = t0.account_id)

[Gargron]

Sorry, I got confused about what account_tag_stats represents. There is need to update it. Fixed

[mayaeh]

Thank you! I will try now.

[mayaeh]

The following error still occurred.
Is this because I ran db: migrate after #11416 ?

rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::DuplicateTable: ERROR:  relation "index_tags_on_name_lower" already exists
: CREATE UNIQUE INDEX CONCURRENTLY index_tags_on_name_lower ON tags (lower(name))

[Gargron]

Yes, probably. Try to drop that index manually first.

[mayaeh]

I ran DROP INDEX index_tags_on_name_lower;.
But I got the following error:

$ RAILS_ENV=production bundle exec rake db:migrate
Migrating to AddCaseInsensitiveIndexToTags (20190726175042)
== 20190726175042 AddCaseInsensitiveIndexToTags: migrating ====================
-- execute("CREATE UNIQUE INDEX CONCURRENTLY index_tags_on_name_lower ON tags (lower(name))")
   -> 0.0542s
-- remove_index(:tags, {:name=>"index_tags_on_name"})
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::UndefinedObject: ERROR:  index "index_tags_on_name" does not exist
: DROP INDEX  "index_tags_on_name"

So I commented out this line (remove_index :tags, name: 'index_tags_on_name') and executed db:migrate.
Then db:migrate succeeded successfully.

Perhaps I think this error only happens to people running db:migrate after 11416