306

I have a MySQL database configured with the default collation utf8mb4_general_ci. When I try to insert a row containing an emoji character in the text using the following query

insert into tablename 
(column1,column2,column3,column4,column5,column6,column7) 
values 
('273','3','Hdhdhdh😜😀😊😃hzhzhzzhjzj 我爱你 ❌',49,1,'2016-09-13 08:02:29','2016-09-13 08:02:29');

MySQL is raising the following error

1366 Incorrect string value: '\xF0\x9F\x98\x83\xF0\x9F...' for column 'comment' at row 1

4
  • 1
    How are your saving your data? Can you show us that code? Sep 13, 2016 at 7:05
  • 6
    Thanks for your comment. I have find solution for this Change Database default collection as **utf8mb4 ** and also Change Table collection as ** CHARACTER SET utf8mb4 COLLATE utf8mb4_bin **. ALTER TABLE Tablename CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin Sep 13, 2016 at 7:52
  • 5
    Code : insert into tablename (column1,column2,column3,column4,column5,column6,column7) values ('273','3','Hdhdhdh😜😀😊😃hzhzhzzhjzj 我爱你 ❌',49,1,'2016-09-13 08:02:29','2016-09-13 08:02:29' Set utf8mb4 in database connection : $database_connection = new mysqli($server, $user,$password,$database_name); $database_connection->set_charset("utf8mb4"); Sep 13, 2016 at 8:05
  • What client are you using? You could also specify utf8mb4 when connecting.
    – Rick James
    Jul 27, 2020 at 18:18

24 Answers 24

364

1) Database: Change Database default collation as utf8mb4.

2) Table: Change table collation as CHARACTER SET utf8mb4 COLLATE utf8mb4_bin.

Query:

ALTER TABLE Tablename CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin

3) Code:

INSERT INTO tablename (column1, column2, column3, column4, column5, column6, column7)
VALUES ('273', '3', 'Hdhdhdh😜😀😊😃hzhzhzzhjzj 我爱你 ❌', 49, 1, '2016-09-13 08:02:29', '2016-09-13 08:02:29')

4) Set utf8mb4 in database connection:

  $database_connection = new mysqli($server, $user, $password, $database_name); 
  $database_connection->set_charset('utf8mb4');
21
  • 10
    Is it possible without changing database default collection ?
    – AliN11
    Oct 10, 2016 at 9:31
  • 32
    This is not working for me. I am getting "???" instead of smilies. only "☺" this smily made it to the database safely. Mar 17, 2017 at 16:10
  • 13
    May need to update not only the table to utf8mb4, but ALSO the columns themselves, otherwise they can still appear as ?? instead of 💙.
    – Ael
    Jun 25, 2017 at 21:12
  • 4
    Worked for me, but don't forget to restart MySQL.
    – Ravi Misra
    Sep 12, 2017 at 10:58
  • 14
    I need to run SET NAMES utf8mb4; to start saving emoticons; before that command it was saving them as ??
    – cubbuk
    Apr 7, 2018 at 14:03
160

Step 1, change your database's default charset:

ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;

if the db is not created yet, create it with correct encodings:

CREATE DATABASE database_name DEFAULT CHARSET = utf8mb4 DEFAULT COLLATE = utf8mb4_unicode_ci;

Step 2, set charset when creating table:

CREATE TABLE IF NOT EXISTS table_name (
...
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE utf8mb4_unicode_ci;

or alter table

ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE table_name MODIFY field_name TEXT CHARSET utf8mb4;
10
  • 1
    I followed these queries and stopped and restarted mysql server, but when I try to insert emojis into my table I still get the same error. All the commands passed successful except the INSERT. INSERT INTO Entries (date, time, caption) VALUES (2018-05-20', '12:38:00', 'Testing description with emoji: 😊❤️'); The column settings are Collation: utf8mb4_0900_ai_ci Definition: description text
    – user6004404
    May 20, 2018 at 17:18
  • 30
    Your connection also needs to be utf8mb4 not utf8 for it to work. Jun 26, 2018 at 11:00
  • 3
    @ospider, in step 2 you use utfmb4_general_ci instead of unicode - any reason why?
    – Warren
    Dec 19, 2018 at 23:52
  • @Warren - "Unicode" and "utf8" are related, but "unicode" is not correct in that context.
    – Rick James
    Jul 27, 2020 at 17:55
  • 2
    No need to modify database. Just altering table is enough. The people that say "it didnt work" Probably, you try it from your application layer. In Your application layer, you also have to change your db connection's charset and collate.
    – hkaraoglu
    Jul 31, 2021 at 10:50
39

The command to modify the column is:

ALTER TABLE TABLE_NAME MODIFY COLUMN_NAME TYPE;

And we need to use type = BLOB

Example to modify is as under:-

ALTER TABLE messages MODIFY content BLOB;

I checked that latest mySQL and other databases don't need '' to use in command on table_name, column_name etc.

Fetch and Save data: Directly save the chat content to column and to retrieve data, fetch data as byte array (byte[]) from db column and then convert it to string e.g. (Java code)

new String((byte[]) arr) 
5
  • 7
    Yes. If you just need to store unicode like emojis in a certain field, the accepted answer is way too intrusive, Simply change the text / varchar field to a blob and you are done. Madness to convert charset and collation on the entire DB just for that :) Dec 30, 2019 at 7:13
  • 2
    I tried changing collations but only blob type fixed my issue(PHP).
    – m9m9m
    Nov 1, 2021 at 10:14
  • It's probably worth pointing out that the "right" way is to migrate our DBs to utf8mb4. But for a quick hack for one or two fields, this is the best approach.
    – rinogo
    Mar 25, 2022 at 16:54
  • If I only needed to store a single emoji, Could this be done with CHAR as default ? As the length is based on characters and not bytes ?
    – theMyth
    Apr 4 at 19:58
  • this should be the correct answer Jul 7 at 4:14
30

Both the databases and tables should have character set utf8mb4 and collation utf8mb4_unicode_ci.

When creating a new database you should use:

CREATE DATABASE mydb CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

If you have an existing database and you want to add support:

ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;

You also need to set the correct character set and collation for your tables:

CREATE TABLE IF NOT EXISTS table_name (
    ...
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE utf8mb4_unicode_ci;

or change it if you've got existing tables with a lot of data:

ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Note that utf8_general_ci is no longer recommended best practice. See the related Q & A:

What's the difference between utf8_general_ci and utf8_unicode_ci on Stack Overflow.

5
  • i have a database and table containing data.and when executing second alter statement, says that : ERROR 1833 (HY000): Cannot change column 'id': used in a foreign key constraint 'FK12njtf8e0jmyb45lqfpt6ad89' of table 'lizbazi.post' Aug 29, 2019 at 15:34
  • 1
    @SeyyedMahdiyarZerehpoush – you might be able to get away with limiting your update to specific columns that require it, as described here: stackoverflow.com/a/15781925/1247581 e. g. ALTER TABLE mytable MODIFY my_emoji_friendly_text_column VARCHAR(2000) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; Dec 17, 2019 at 20:04
  • 2
    any diff when using utf8mb4_bin vs utf8mb4_unicode_ci for the columns? May 13, 2020 at 12:37
  • thanks @SeyyedMahdiyarZerehpoush, I just alter the column table when i need the emojis and thats all. There's no need to change the collate and charset at database or table level
    – lombardo
    May 16, 2022 at 21:43
  • 1
    Both the database and tables do NOT have to be utf8mb4. I have a database with the default charset = latin1, a table with default charset = utf8 and a column within that table with charset = utf8mb4, and I can save emojis in that column no problem. Even the fox, the avocado and the snowman emojis are no problem with this configuration.
    – Vincent
    Feb 17 at 1:56
17

If you are using Solr + Mysql + Java, you can use:

This can be Used :

  • case1: When you don`t want to alter DB.
  • case2: when you have to import emoticons from your Mysql to Solr core.

In above case this is one of the solutions to store your emoticons in your system.

Steps to use it:

Library used: import java.net.URLDecoder; import java.net.URLEncoder;

  1. Use urlEncoder to encode your String having emoticons.
  2. Store it in DB without altering the MysqlDB.
  3. You can store it in solr core(decoded form)if you want or you can store encoded form.
  4. When fetching these emoticons from DB or Solr core you can now decode it Using urlDecoder.

Code example:

import java.net.URLDecoder;
import java.net.URLEncoder;

public static void main(String[] args) {
    //SpringApplication.run(ParticipantApplication.class, args);
    System.out.println(encodeStringUrl("🇺🇸🇨🇳🇯🇵🇩🇪🔳🔺🆔🆔🆑3⃣5⃣3⃣‼〽➗➗🎦🔆🎦🔆♋♍♋♍⬅⬆⬅⬅🛂🚹🛂🛄🚳🚬💊🔧💊🗿     "));
    System.out.println(decodeStringUrl("Hello+emoticons%2C%2C%F0%9F%98%80%F0%9F%98%81%F0%9F%98%8A%F0%9F%98%8B%F0%9F%98%8E%F0%9F%98%8A%F0%9F%98%8D%E2%98%BA%F0%9F%98%98%E2%98%BA%F0%9F%98%91%F0%9F%98%87%F0%9F%98%98%F0%9F%98%8B%F0%9F%90%84"));
}

public static String encodeStringUrl(String url) {
    String encodedUrl =null;
    try {
         encodedUrl = URLEncoder.encode(url, "UTF-8");
    } catch (UnsupportedEncodingException e) {
        return encodedUrl;
    }
    return encodedUrl;
}

public static String decodeStringUrl(String encodedUrl) {
    String decodedUrl =null;
    try {
         decodedUrl = URLDecoder.decode(encodedUrl, "UTF-8");
    } catch (UnsupportedEncodingException e) {
        return decodedUrl;
    }
    return decodedUrl;
}
3
  • Thank you for this code snippet, which might provide some limited, immediate help. A proper explanation would greatly improve its long-term value by showing why this is a good solution to the problem, and would make it more useful to future readers with other, similar questions. Please edit your answer to add some explanation, including the assumptions you've made. Oct 4, 2017 at 8:39
  • 1
    Encode/decode function calls tend to cause trouble. Instead fix the charset settings in the various places.
    – Rick James
    Feb 26, 2018 at 4:04
  • 1
    This is not solving the problem, this is bypassing it. And you'll run into a host of problems with this method, for example you will slow down your application because you have to decode and encode everything. Also if you enter the character %, your decoding will break. Jun 9, 2018 at 12:51
17

I have updated my database and table to upgraded from utf8 to utf8mb4. But nothing works for me. Then I tried to update column datatype to blob, luckily it worked for me and data has been saved. Even my database and table both are CHARACTER SET utf8 COLLATE utf8_unicode

1
  • This is good, but it should be possible to use CHAR, VARCHAR types for emojis ?
    – theMyth
    Apr 4 at 19:59
13

My answer only adds to Selvamani P answer.

You might also need to change any SET NAMES utf8 queries with SET NAMES utf8mb4. That did the trick for me.

Also, this is a great article to port your website from utf8 to utf8mb4. In particular the article makes 2 good points on indexes and repairing tables after converting them to utf8mb4:

INDEXES

When converting from utf8 to utf8mb4, the maximum length of a column or index key is unchanged in terms of bytes. Therefore, it is smaller in terms of characters, because the maximum length of a character is now four bytes instead of three. [...] The InnoDB storage engine has a maximum index length of 767 bytes, so for utf8 or utf8mb4 columns, you can index a maximum of 255 or 191 characters, respectively. If you currently have utf8 columns with indexes longer than 191 characters, you will need to index a smaller number of characters when using utf8mb4.

REPAIRING TABLES

After upgrading the MySQL server and making the necessary changes explained above, make sure to repair and optimize all databases and tables. I didn’t do this right away after upgrading (I didn’t think it was necessary, as everything seemed to work fine at first glance), and ran into some weird bugs where UPDATE statements didn’t have any effect, even though no errors were thrown.

Read more about the queries to repair tables on the article.

1
  • 1
    REPAIR TABLE and OPTIMIZE TABLE should be unnecessary -- an ALTER has the effect of doing them.
    – Rick James
    Feb 26, 2018 at 4:06
13

I have a good solution to save your time. I also meet the same problem but I could not solve this problem by the first answer.

Your defualt character is utf-8. But emoji needs utf8mb4 to support it. If you have the permission to revise the configure file of mysql, you can follow this step.

Therefore, do this following step to upgrade your character set ( from utf-8 to utf8mb4).

step 1. open your my.cnf for mysql, add these following lines to your my.cnf.

[mysqld]
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci
init_connect='SET NAMES utf8mb4'

[mysql]
default-character-set = utf8mb4


[client]
default-character-set = utf8mb4

step2. stop your mysql service, and start mysql service

mysql.server stop
mysql.server start

Finished! Then you can check your character are changed into utf8mb4.

mysql> SHOW VARIABLES LIKE 'character_set%';
+--------------------------+----------------------------------------------------------+
| Variable_name            | Value                                                    |
+--------------------------+----------------------------------------------------------+
| character_set_client     | utf8mb4                                                  |
| character_set_connection | utf8mb4                                                  |
| character_set_database   | utf8mb4                                                  |
| character_set_filesystem | binary                                                   |
| character_set_results    | utf8mb4                                                  |
| character_set_server     | utf8mb4                                                  |
| character_set_system     | utf8                                                     |
| character_sets_dir       | /usr/local/Cellar/[email protected]/5.7.29/share/mysql/charsets/ |
+--------------------------+----------------------------------------------------------+
8 rows in set (0.00 sec)
1
  • One small difference for anyone using MariaDB - the config line needs to be init-connect='SET NAMES utf8mb4' (with a - instead of a _)
    – SpoonNZ
    Apr 12, 2022 at 22:58
11

There are two ways--> # Way one The simplest is to follow below steps:

Step 1:

SET NAMES utf8mb4;

Step 2:

ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;

Step 3:

ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Step 4:

ALTER TABLE table_name CHANGE column column VARCHAR(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL;

That's all!!

#Way Two (For Python)

This is a hack and its work like a charm

Step 1: Encode your string and decode it in ASCII, and save it to your database.

content = '🥳🥳 Content to be save in 🥳🥳 Database 🥳🥳'
encoded_content = content.encode('unicode-escape').decode('ASCII'))

This simply store encoded_content string in DB

Step 2: While fetch this column data to show your user, simply convert it, here content is the data, fetched from the database.

c = bytes(encoded_content, 'utf-8')
original_content = c.decode('unicode-escape')

Done!!

1
  • or in Ruby... "\xF0\x9F\x91\x8F".dump.undump where dumpescapes the unicode and undump restores it Feb 13, 2022 at 23:06
10

Well, you need not to change the Whole DB Charset. Instead of that you can do it by changing column to blob type.

ALTER TABLE messages MODIFY content BLOB;

8

Emoji support for application having tech stack - mysql, java, springboot, hibernate

Apply below changes in mysql for unicode support.

  1. ALTER DATABASE <database-name> CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
  2. ALTER TABLE <table-name> CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

DB Connection - jdbc url change:

jdbc:mysql://localhost:3306/<database-name>?useUnicode=yes&characterEncoding=UTF-8

Note - If the above step is not working please update mysql-connector version to 8.0.15. (mysql 5.7 works with connector version 8.0.15 for unicode support)

6

The main point hasn't been mentioned in the above answers that,

We need to pass query string with the options "useUnicode=yes" and "characterEncoding=UTF-8" in connection string

Something like this

mysql://USERNAME:PASSWORD@HOSTNAME:PORT/DATABASE_NAME?useUnicode=yes&characterEncoding=UTF-8
4

The simplest solution what works for me is to store the data as json_encode.

later when you retrieve just make sure you json_decode it.

Here you don't have to change the collation or the character set of the database and the table.

1
  • It isn't a solution but a vary bad workaround because it brings some drawbacks like the impossibility to compare data and/or check the content of a column because of the json format, more useless data stored in the database, every read/write will be slower, etc Jan 19 at 7:32
3

For anyone trying to solve this on a managed MySQL instance (in my case on AWS RDS), the easiest way was to modify the parameter group and set the server character set and collation to be utf8mb4 and utf8mb4_bin, respectively. After rebooting the server, a quick query verifies the settings for system databases and any newly created ones:

SELECT * FROM information_schema.SCHEMATA S;
3

For Rails, next to the accepted answer, don't forget to add:

encoding: utf8mb4
collation: utf8mb4_bin

to your database.yml

3

I'm facing this issue when upgrading MySQL 5.0 to MySQL 8.0 AWS RDS, trying many things finally what works for me share with you folks.

Error:

Warning: PDOStatement::execute(): SQLSTATE[HY000]: General error: 3988 Conversion from collation utf8_unicode_ci into utf8mb4_general_ci impossible for parameter in /var/www/html/pdo_con.php on line 87

Array ( [0] => HY000 [1] => 3988 [2] => Conversion from collation utf8_unicode_ci into utf8mb4_general_ci impossible for parameter )

Backend: PHP5/php7 + PDO is giving trouble.

Solution: only two thing needs to do

  1. Add a code in line after your pdo connection

    $conn->exec("set names utf8mb4");
    

    where $conn is connection handler in PDO

  2. Alter the table and set charset utf8mb4 and collate utf8mb4_unicode_ci.

    ALTER TABLE mytable CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
    

    This command will change your every column charset to utf8mb4 and collation too.

2
  • This does not work.
    – Čamo
    Feb 28 at 11:05
  • This solved my issue. It was a pain to figure it out
    – Zortext
    Sep 20 at 7:31
2

If you use command line interface for inserting sql file to database.

Be sure your table charset utf8mb4 and column collation utf8mb4_unicode_ci or utf8mb4_bin

mysql -u root -p123456 my_database < profiles.sql

ERROR 1366 (HY000) at line 1679: Incorrect string value: '\xF0\x9F\x98\x87\xF0\x9F...' for column 'note' at row 328

we can solve the problem with this parameter --default-character-set=name (Set the default character set)

mysql -u root -p123456 --default-character-set=utf8mb4 my_database < profiles.sql
2

If you are inserting using PHP, and you have followed the various ALTER database and ALTER table options above, make sure your php connection's charset is utf8mb4.

Example of connection string:

$this->pdo = new PDO("mysql:host=$ip;port=$port;dbname=$db;charset=utf8mb4", etc etc

Notice the "charset" is utf8mb4, not just utf8!

2

Today I am facing the same question, but solutions in other answers don't work for me. Here is my solution.

First of all, changing charset in mysql/my.ini, database, and the table is necessary, as described in other answers.

Second, if you have created your tables before you want to saving emoji, you can use

SHOW FULL COLUMNS FROM `yourcolumn`;

To check whether the column you want to save emoji is set in utf8mb4. You can find that most of your columns are still in utf8 charset.

Use

ALTER TABLE `yourtable` CHANGE `yourcolumn` `yourcolumn` VARCHAR(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
2

I tried different methods and approaches and found a way that worked for me.

The SQL for the update query:

ALTER DATABASE YOUR_DB_NAME_HERE CHARACTER SET = utf8mb4 COLLATE = 
utf8mb4_unicode_ci;

You can see in the table, the emoji's are present

enter image description here

And if you go to this page: https://www.thecookingcat.com/recipes/thai-green-curry.php#comments

You can see the emojis in the comments.

I also have an RSS feed on the site and the emojis are included in the RSS feed XML code.

1

Hi my friends This is how I solved this problem and I was happy to teach it to you as well I am in the Android application I encrypt a string containing text and emoj and send it to the server and save it in the mysql table and after receiving it from the server I decrypt it and display it in the textview. encoded and decoded my message before request and after response: I send Android app messages to mysql via pdo through this method and receive them with pdo. And I have no problem. I think it was a good way. Please like Thankful

 
 public void main()
 {
    String message="hi mester ali moradi 🌦️🌦️ how are you ?";
    String encoded_message=encodeStringUrl(message);
    String decode_message=decodeStringUrl(encoded_message);
 }
 public static String encodeStringUrl(String message) {
        String encodedUrl =null;
        try {
            encodedUrl = URLEncoder.encode(message, "UTF-8");
        } catch (UnsupportedEncodingException e) {
            return encodedUrl;
        }
        return encodedUrl;
    }

    public static String decodeStringUrl(String message) {
        String decodedUrl =null;
        try {
            decodedUrl = URLDecoder.decode(message, "UTF-8");
        } catch (UnsupportedEncodingException e) {
            return decodedUrl;
        }
        return decodedUrl;
    }
message : hi mester ali moradi 🌦️🌦️ how are you ? encoded : ghgh%F0%9F%98%AE%F0%9F%A4%90%F0%9F%98%A5 decoded :hi mester ali moradi 🌦️🌦️ how are you ?

1

Actually i'm using mysql Ver 8.0.23 I had created the both Database and the Table, without Altering them :

mysql> CREATE DATABASE tp2;

Query OK, 1 row affected (0.30 sec)

mysql> INSERT INTO tweetsRep(username, content) VALUES ('ibrahim', '🤣 oh my god');

Then after select, i thing it just worked fine !

I don't know if it is requested to enter Emoji as a hexadecimal or other encoding string or just copy it as it is... just correct me if i'm wrong, thank you !

1

Put this right before your database call:

    mysqli_set_charset($db, "utf8mb4");

This will allow you to input emojis directly into the database table that has been set to Collation: utfmb4_bin. Make sure to set your column to utfmb4 as well.

1

If anyone searching this in 2022 just follow these steps and no need to do any modification on Database

Name Space

using System.Web;

Your normal text like this :

String encode = "thank you 😊"
encode = HttpUtility.UrlEncode(encode);

It will store in Database like this : "thank+you+%f0%9f%98%8a"

And next fetch that data form your Database and do UrlDecode like this

DataSet ds = "Fetch your Encoded data form your Database";
String decode =  HttpUtility.UrlDecode(ds.Tables[0].Rows[i]["YourColumnName"].ToString().Trim());

And your output is :-

decode = "thank you 😊".

It is working fine for me and saved time.

Not the answer you're looking for? Browse other questions tagged or ask your own question.