28

I have read that mysql >= 5.5.3 fully supports every possible character if you USE the encoding utf8mb4 for a certain table/column http://mathiasbynens.be/notes/mysql-utf8mb4

looks nice. Only I noticed that the mb_functions in php does not! I cannot find it anywhere in the list: http://php.net/manual/en/mbstring.supported-encodings.php

Not only have I read things but I also made a test.

I have added data to a mysql utf8mb4 table using a php script where the internal encoding was set to UTF-8: mb_internal_encoding("UTF-8");

and, as expected, the characters looks messy once in the db.

Any idea how I can make php and mysql talk the same encoding (possibly a 4 bytes one) and still have FULL support to any world language?

Also why is utf8mb4 different from utf32?

2

4 Answers 4

41

MySQL's utf8 encoding is not actual UTF-8. It's an encoding that is kinda like UTF-8, but only supports a subset of what UTF-8 supports. utf8mb4 is actual UTF-8. This difference is an internal implementation detail of MySQL. Both look like UTF-8 on the PHP side. Whether you use utf8 or utf8mb4, PHP will get valid UTF-8 in both cases.

What you need to make sure is that the connection encoding between PHP and MySQL is set to utf8mb4. If it's set to utf8, MySQL will not support all characters. You set this connection encoding using mysql_set_charset(), the PDO charset DSN connection parameter or whatever other method is appropriate for your database API of choice.


mb_internal_encoding just sets the default value for the $encoding parameter all mb_* functions have. It has nothing to do with MySQL.

UTF-8 and UTF-32 differ in how they encode characters. UTF-8 uses a minimum of 1 byte for a character and a maximum of 4. UTF-32 always uses 4 bytes for every character. UTF-16 uses a minimum of 2 bytes and a maximum of 4.
Due to its variable length, UTF-8 has a little bit of overhead. A character which can be encoded in 2 bytes in UTF-16 may take 3 or 4 in UTF-8; on the other hand, UTF-16 never uses less than 2 bytes. If you're storing lots of Asian text, UTF-16 may use less storage. If most of your text is English/ASCII, UTF-8 uses less storage. UTF-32 always uses the most storage.

7
  • "What you need to make sure is that the connection encoding between PHP and MySQL is set to utf8mb4". Right ... I have not done that! That must be the reason.
    – nourdine
    Jun 3, 2013 at 10:34
  • Is it up to four or six? Joel Spolsky's article on this says 6: joelonsoftware.com/articles/Unicode.html
    – Tommy
    Aug 20, 2015 at 19:49
  • @Tommy That's old and deprecated information. 6 isn't used in practice and has been discontinued.
    – deceze
    Aug 20, 2015 at 19:52
  • so UTF-8 changed? It wasn't a standard? (I'm trying to understand this mess all now)
    – Tommy
    Aug 20, 2015 at 19:54
  • The problem is with this is that if we're stuck with mysql UTF8, are we not unable to validate strings in PHP for MySQL without using the database? Wont a stringthat reads as valid UTF8 in PHP potentially have unrecognised sequences nuked in mysql?
    – jgmjgm
    Oct 10, 2017 at 9:40
12

This is what i used, and worked good for my problem using euro € sign and conversion for json_encode failure.

php configurations script( api etc..)

header('Content-Type: text/html; charset=utf-8');
ini_set("default_charset", "UTF-8");
mb_internal_encoding("UTF-8");
iconv_set_encoding("internal_encoding", "UTF-8");
iconv_set_encoding("output_encoding", "UTF-8");

mysql tables / or specific columns

utf8mb4

mysql PDO connection

$dsn = 'mysql:host=yourip;dbname=XYZ;charset=utf8mb4';

(...your connection ...)

before execute query (might not be required):

$dbh->exec("set names utf8mb4");
3
  • 3
    This isnt accurate though. If you are using 'set names utf8' then your connection is speaking to mysql in just a subset of real utf8. You need to 'set names utf8mb4' to speak in the full utf8 character set (including passing emojis). Otherwise mysql will often truncate a string at the point it encounters a utf character which is > 3 bytes
    – carpii
    Jan 17, 2016 at 22:30
  • Super helpful. All I had to do was to set the charset to utf8mb4 instead of utf8. Now my mysql database reads and writes emojis with no issues 😊 Jul 1, 2017 at 19:29
  • setting iconv encoding seems to be deprecated now. Feb 9, 2018 at 16:03
4
  • utf-32: This is a character encoding using a fixed 4-bytes per characters
  • utf-8: This is a character encoding using up to 4 bytes per characters, but the most frequent characters are coded on only 1, 2 or 3 characters.

MySQL's utf-8 doesn't support characters coded on more than 3 characters, so they added utf-8mb4, which is really utf-8.

3

Before running your actual query, do a mysql_query ('SET NAMES utf8mb4')

Also make sure your mysql server is configured to use utf8mb4 too. For more information on how, refer to article: https://mathiasbynens.be/notes/mysql-utf8mb4#utf8-to-utf8mb4

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.