1

I wanted to run a query that searches for a specific VARCHAR item in mysql DB. Since my project is Multi-lingual i have no choice but to use Unicode UTF8 as my charaset for that field.

The issue that i have is as follows:

This Char ی and this Char ى looks the same

However the unicode for above 2 is different (U+0649 & U+06CC)

I have alot of data and they contain mix of both above mentioned chars and many more.

There are many other characters in a similar way and what i need is a way to treat a series of these characters as the same so mysql matches them when i run the query.

I know it can be done by using OR in my query and including both the chars but is there any other way like maybe making my own charset/collation where i can define the two as same or if they look the same then they are treated the same?

5
  • 1
    Are you aware that these characters are different in usage and context?
    – frlan
    Jul 29, 2015 at 14:51
  • Yes i am, but in my application that doesn't matter, as the data has been typed in using different keyboard models but ultimately with the same purpose.
    – Kingk32
    Jul 29, 2015 at 15:05
  • In case of you only want to replace a small number of differenz letters, you could write an prcedure (e.g. as trigger) that is doing the replacement
    – frlan
    Jul 29, 2015 at 15:11
  • ok, i didnt know about procedures in mysql, im gonna look into it.
    – Kingk32
    Jul 29, 2015 at 15:13
  • But doing it as an update/insert trigger it will change the data provided by the user and you could also do it with your application one layer above. But you could do it also on selecting data.
    – frlan
    Jul 29, 2015 at 15:18

1 Answer 1

1

You want these to be treated as equal?

D989       1609=x0649  [ى]   AL  ARABIC LETTER ALEF MAKSURA
DB8C       1740=x06CC  [ی]   AL  ARABIC LETTER FARSI YEH

I checked COLLATE utf8_unicode_ci; they compare unequal:

mysql> SELECT CONVERT(UNHEX('D989') USING utf8) =
    ->        CONVERT(UNHEX('DB8C') USING utf8) COLLATE utf8_unicode_ci AS x;
+------+
| x    |
+------+
|    0 |
+------+

I don't think MySQL will compare them as equal unless you create a custom collation. See http://dev.mysql.com/doc/refman/5.6/en/adding-collation.html .

You could do set of REPLACE()s to change ى to ی before doing the comparison. But that would be messy and slow.

1
  • i agree custom collation, seems right enough, thanks alot.
    – Kingk32
    Aug 3, 2015 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.