214

Error message on MySql:

Illegal mix of collations (utf8_unicode_ci,IMPLICIT) and (utf8_general_ci,IMPLICIT) for operation '='

I have gone through several other posts and was not able to solve this problem. The part affected is something similar to this:

CREATE TABLE users (
    userID INT UNSIGNED NOT NULL AUTO_INCREMENT,
    firstName VARCHAR(24) NOT NULL,
    lastName VARCHAR(24) NOT NULL,
    username VARCHAR(24) NOT NULL,
    password VARCHAR(40) NOT NULL,
    PRIMARY KEY (userid)
) ENGINE = INNODB CHARACTER SET utf8 COLLATE utf8_unicode_ci;

CREATE TABLE products (
    productID INT UNSIGNED NOT NULL AUTO_INCREMENT,
    title VARCHAR(104) NOT NULL,
    picturePath VARCHAR(104) NULL,
    pictureThumb VARCHAR(104) NULL,
    creationDate DATE NOT NULL,
    closeDate DATE NULL,
    deleteDate DATE NULL,
    varPath VARCHAR(104) NULL,
    isPublic TINYINT(1) UNSIGNED NOT NULL DEFAULT '1',
    PRIMARY KEY (productID)
) ENGINE = INNODB CHARACTER SET utf8 COLLATE utf8_unicode_ci;

CREATE TABLE productUsers (
    productID INT UNSIGNED NOT NULL,
    userID INT UNSIGNED NOT NULL,
    permission VARCHAR(16) NOT NULL,
    PRIMARY KEY (productID,userID),
    FOREIGN KEY (productID) REFERENCES products (productID) ON DELETE RESTRICT ON UPDATE NO ACTION,
    FOREIGN KEY (userID) REFERENCES users (userID) ON DELETE RESTRICT ON UPDATE NO ACTION
) ENGINE = INNODB CHARACTER SET utf8 COLLATE utf8_unicode_ci;

The stored procedure I'm using is this:

CREATE PROCEDURE updateProductUsers (IN rUsername VARCHAR(24),IN rProductID INT UNSIGNED,IN rPerm VARCHAR(16))
BEGIN
    UPDATE productUsers
        INNER JOIN users
        ON productUsers.userID = users.userID
        SET productUsers.permission = rPerm
        WHERE users.username = rUsername
        AND productUsers.productID = rProductID;
END

I was testing with php, but the same error is given with SQLyog. I have also tested recreating the entire DB but to no good.

Any help will be much appreciated.

1
  • 1
    I have faced the same issue when try to join records of 3 tables. When I checked my database, I found that, one of the table has utf8_general but other are have utf8_unicode_ci. So I changed utf8_general to utf8_unicode_ci and my problem solved. Oct 28, 2020 at 10:28

9 Answers 9

283

The default collation for stored procedure parameters is utf8_general_ci and you can't mix collations, so you have four options:

Option 1: add COLLATE to your input variable:

SET @rUsername = ‘aname’ COLLATE utf8_unicode_ci; -- COLLATE added
CALL updateProductUsers(@rUsername, @rProductID, @rPerm);

Option 2: add COLLATE to the WHERE clause:

CREATE PROCEDURE updateProductUsers(
    IN rUsername VARCHAR(24),
    IN rProductID INT UNSIGNED,
    IN rPerm VARCHAR(16))
BEGIN
    UPDATE productUsers
        INNER JOIN users
        ON productUsers.userID = users.userID
        SET productUsers.permission = rPerm
        WHERE users.username = rUsername COLLATE utf8_unicode_ci -- COLLATE added
        AND productUsers.productID = rProductID;
END

Option 3: add it to the IN parameter definition (pre-MySQL 5.7):

CREATE PROCEDURE updateProductUsers(
    IN rUsername VARCHAR(24) COLLATE utf8_unicode_ci, -- COLLATE added
    IN rProductID INT UNSIGNED,
    IN rPerm VARCHAR(16))
BEGIN
    UPDATE productUsers
        INNER JOIN users
        ON productUsers.userID = users.userID
        SET productUsers.permission = rPerm
        WHERE users.username = rUsername
        AND productUsers.productID = rProductID;
END

Option 4: alter the field itself:

ALTER TABLE users CHARACTER SET utf8 COLLATE utf8_general_ci;

Unless you need to sort data in Unicode order, I would suggest altering all your tables to use utf8_general_ci collation, as it requires no code changes, and will speed sorts up slightly.

UPDATE: utf8mb4/utf8mb4_unicode_ci is now the preferred character set/collation method. utf8_general_ci is advised against, as the performance improvement is negligible. See https://stackoverflow.com/a/766996/1432614

8
  • 2
    It is also possible to add COLLATE utf8_unicode_ci to string constants: SET @EMAIL = '[email protected]' COLLATE utf8_unicode_ci;. It is especially useful if you are running a script from a console, where the console default encoding applies to your string constants' collation.
    – gaborsch
    May 5, 2016 at 10:35
  • Or drop database and create new with utf8_general_ci; collation. Apr 1, 2017 at 22:05
  • 3
    For future reference, don't change all your tables to utf8_general_ci unless you understand the differences between the two collations.
    – Manatax
    Jul 31, 2017 at 19:10
  • 2
    @GaborSch Adding collate to string variables was the solution for me, I wrote a detailed answer about it before I noticed your comment.
    – nkatsar
    Oct 10, 2017 at 16:49
  • im getting the same error, except (utf8mb4_unicode_ci, IMPLICIT) instead of (utf8_unicode_ci, IMPLICIT). i am scraping data off the web using python, then creating an CSV file with the scraped data, which i then process with a PHP file on my server that uploads the data to my database. all my MySQL tables/columns are collated as utf8mb4_unicode_ci. might the issue be arising because i encode the data as utf8 in python/csv?
    – oldboy
    Nov 7, 2018 at 8:29
38

I spent half a day searching for answers to an identical "Illegal mix of collations" error with conflicts between utf8_unicode_ci and utf8_general_ci.

I found that some columns in my database were not specifically collated utf8_unicode_ci. It seems mysql implicitly collated these columns utf8_general_ci.

Specifically, running a 'SHOW CREATE TABLE table1' query outputted something like the following:

| table1 | CREATE TABLE `table1` (
`id` int(11) NOT NULL,
`col1` varchar(4) CHARACTER SET utf8 NOT NULL,
`col2` int(11) NOT NULL,
PRIMARY KEY (`col1`,`col2`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci |

Note the line 'col1' varchar(4) CHARACTER SET utf8 NOT NULL does not have a collation specified. I then ran the following query:

ALTER TABLE table1 CHANGE col1 col1 VARCHAR(4) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL;

This solved my "Illegal mix of collations" error. Hope this might help someone else out there.

5
  • 13
    thanks. 'SHOW CREATE TABLE' is the easiest way to understand & fix the root cause of the problem.
    – JRr
    Jan 25, 2016 at 11:44
  • 4
    Also note that specifying COLLATE for the whole table (i.e. ALTER TABLE table1 CHARSET utf8 COLLATE utf8_unicode_ci) won't fix the issue, it has to be done for each (problematic) column. Aug 23, 2019 at 8:15
  • @SkippyleGrandGourou I dropped the table and recreated it with CHARSET utf8 COLLATE utf8_unicode_ci and it worked for me.
    – veritas
    Jun 15, 2021 at 8:47
  • 1
    @veritas Yes, my comment was about altering the table, not recreating it from scratch. Jun 16, 2021 at 13:51
  • also, for whatever reason, the mysql bench, will have misleading answers for the values of the fields as well as the table. using the above manual alter statements was the only way to ensure it was set correctly for me too.
    – G-Force
    Jan 7, 2022 at 23:38
9

I had a similar problem, but it occurred to me inside procedure, when my query param was set using variable e.g. SET @value='foo'.

What was causing this was mismatched collation_connection and Database collation. Changed collation_connection to match collation_database and problem went away. I think this is more elegant approach than adding COLLATE after param/value.

To sum up: all collations must match. Use SHOW VARIABLES and make sure collation_connection and collation_database match (also check table collation using SHOW TABLE STATUS [table_name]).

1
  • 1
    Same issue happened to me, I avoided changing the collation_YYY variables by setting the collation directly in the variable declaration. SET @my_var = 'string1,string2' COLLATE utf8_unicode_ci;
    – nkatsar
    Oct 10, 2017 at 16:44
6

A bit similar to @bpile answer, my case was a my.cnf entry setting collation-server = utf8_general_ci. After I realized that (and after trying everything above), I forcefully switched my database to utf8_general_ci instead of utf8_unicode_ci and that was it:

ALTER DATABASE `db` CHARACTER SET utf8 COLLATE utf8_general_ci;
1
  • 1
    It's strange that the configurations are spread around so much. All collation defaults should be set on the same place.
    – Manatax
    Dec 7, 2016 at 19:35
4

Answer is adding to @Sebas' answer - setting the collation of my local environment. Do not try this on production.

ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_unicode_ci;

ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;

Source of this solution

1

Despite finding an enormous number of question about the same problem (1, 2, 3, 4) I have never found an answer that took performance into consideration, even here.

Although multiple working solutions has been already given I would like to do a performance consideration.

EDIT: Thanks to Manatax for pointing out that option 1 does not suffer of performance issues.

Using Option 1 and 2, aka the COLLATE cast approach, can lead to potential bottleneck, cause any index defined on the column will not be used causing a full scan.

Even though I did not try out Option 3, my hunch is that it will suffer the same consequences of option 1 and 2.

Lastly, Option 4 is the best option for very large tables when it is viable. I mean there are no other usage that rely on the original collation.

Consider this simplified query:

SELECT 
    *
FROM
    schema1.table1 AS T1
        LEFT JOIN
    schema2.table2 AS T2 ON T2.CUI = T1.CUI
WHERE
    T1.cui IN ('C0271662' , 'C2919021')
;

In my original example, I had many more joins. Of course, table1 and table2 have different collations. Using the collate operator to cast, it will lead to indexes not being used.

See sql explanation in the picture below.

Visual Query Explanation when using the COLLATE cast

On the other hand, option 4 can take advantages of possible index and led to fast queries.

In the picture below, you can see the same query being run after applied Option 4, aka altering the schema/table/column collation.

Visual Query Explanation after the collation has been changed, and therefore without the collate cast

In conclusion, if performance are important and you can alter the collation of the table, go for Option 4. If you have to act on a single column, you can use something like this:

ALTER TABLE schema1.table1 MODIFY `field` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
1
  • Thank you for your contribution Raffaele, but I do believe the option 1 would use index, because you are not casting the table, but the comparing value before you even pass it to the SP.
    – Manatax
    Oct 23, 2018 at 17:46
0

In my own case I have the following error

Illegal mix of collations (utf8_general_ci,IMPLICIT) and (utf8_unicode_ci,IMPLICIT) for operation '='

$this->db->select("users.username as matric_no, CONCAT(users.surname, ' ', users.first_name, ' ', users.last_name) as fullname") ->join('users', 'users.username=classroom_students.matric_no', 'left') ->where('classroom_students.session_id', $session) ->where('classroom_students.level_id', $level) ->where('classroom_students.dept_id', $dept);

After weeks of google searching I noticed that the two fields I am comparing consists of different collation name. The first one i.e username is of utf8_general_ci while the second one is of utf8_unicode_ci so I went back to the structure of the second table and changed the second field (matric_no) to utf8_general_ci and it worked like a charm.

0

This happens where a column is explicitly set to a different collation or the default collation is different in the table queried.

if you have many tables you want to change collation on run this query:

select concat('ALTER TABLE ', t.table_name , ' CONVERT TO CHARACTER 
SET utf8 COLLATE utf8_unicode_ci;') from (SELECT table_name FROM 
information_schema.tables where table_schema='SCHRMA') t;

this will output the queries needed to convert all the tables to use the correct collation per column

1
  • It also happens when (as in my case) your default collation for SP is different than the collation used for the table queried.
    – Manatax
    Dec 5, 2018 at 21:09
0

I was also facing a problem while upload excels file in MySql Database form Laravel. In excel file some addresses contain characters like PeterÕs that Error was

SQLSTATE[HY000]: General error: 1267 Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8mb4_unicode_ci,COERCIBLE) for operation '=' (SQL: select count(*) as aggregate from `.....` where `e....` = [email protected] and `first_name` = Gill and `surname` = Harries and `address` = 6 St.PeterÕs Close,,,Woodbridge,Suffolk,IP12 4EJ and `status` = 1 and `client`.`deleted_at` is null)

I try many solutions that is mentioned above but unfortunately, no one works for me. so I post the solution that works for me. so maybe it will not work in some situations. Exception can be different. so you have to find a solution based on that. please vote Up if some finds this is useful.

I try 2 types of solutions.

  1. I putted my exceptional code into try{} Catch() block for handling. because it was giving an error while executing select() query. so that I use TRY{} for this. using that uploaded record stored in the database. But when charset issue comes on that place it is put ?. that sample image is given below.

enter image description here

  1. you can detect the charset that gives an issue while operating and change the MySql configuration by this code
 \Config::set('database.connections.mysql.charset', 'latin1');
  \Config::set('database.connections.mysql.collation', 'latin1_bin');
  \DB::purge('mysql');

Both solution works for me.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.