2

As described in the title, I have a problem with inserting emojis into a MariaDB database using the mysql.connector module in python.

When I try to insert the data containing emojis into the database with python, all emojis are simply replaced by question marks. I have also made sure that the emojis work in python by outputting them to the console before inserting them into the database.

Inserting the emojis via phpMyAdmin is no longer a problem after I changed the collation in the database to utf8mb4_general_ci. So I could make sure that there is no problem with the database.

If it helps in any way the data I'm inserting comes from the steam API and consists of nicknames.

I would now like to know if anyone knows how I can fix this problem.

I hope I have not forgotten any important information.

1
  • Nice job with your first question! Thanks.
    – O. Jones
    Mar 30, 2021 at 23:49

1 Answer 1

3

You are correct that emoji are representable in Unicode. Did you remember to set the connection character set and collation in your Python program?

Something like this might help right as you open the connection.

cnx = mysql.connector.connect(user='JonaWe', database='test',
                              use_unicode=True,
                              charset='utf8mb4,
                              collation='utf8mb4_general_ci',
                              etcetera)
cnx.set_charset_collation('utf8mb4', 'utf8mb4_general_ci')

The connection between Python and MySQL might be garbling the utf8mb4 contents of your emoji.

4
  • I didn't know I had to change the charset collation in pyhton too. I tried it and using 'utf8mb4' and 'utf8mb4_general_ci' as a parameter resulted in the following error: unknown encoding: utf8mb4. Then I tried to remove the first parameter, but without that it crashed imidiatly.
    – JonaWe
    Mar 30, 2021 at 23:57
  • Inserting works now, but as soon as I am trying to select some date I get the unknown encoding: utf8mb4 error.
    – JonaWe
    Mar 31, 2021 at 1:07
  • It seems possible you have a version of MySQL that predates the availability of utf8mb4. Upgrade?
    – O. Jones
    Apr 25, 2022 at 11:49
  • 2
    You are right. My problem back them was that I installed the wrong version of the package. I installed pip install mysql.connector instead of pip install mysql-connector-python. After switching to the correct version, I had no more problems. @O.Jones
    – JonaWe
    May 9, 2022 at 7:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.