1

I have a problem here which looks like a bug in MariaDB to me, but before posting it to the MariaDB bug database I wanted to post it here, maybe I am doing something wrong.

The problem can easily be reproduced with the following code:

public static void main(String[] args) throws ReplicationException, SQLException {
  byte[] cb3 = {-61, -92, 98, -29, -125, -70, -61, -87}; //equals to "äb?é", with “?” being a japanese character
  String corrupt = new String(cb3, Charsets.UTF_8);
  MariaDB db = new MariaDB();
  Connection conn = db.getConnection();

  //byte[] latinBytes =  corrupt.getBytes(Charset.forName("ISO-8859-1")); //workaround
  //corrupt = new String(latinBytes, Charset.forName("ISO-8859-1")); //workaround

  PreparedStatement sqlInsert = conn.prepareStatement("insert into prep values (?)");
  sqlInsert.setString(1, corrupt);
  sqlInsert.execute();
  conn.commit();
  System.out.println("insert ok");

  PreparedStatement sqlSelect = conn.prepareStatement("select * from prep where text=? ");
  sqlSelect.setString(1, corrupt);
  sqlSelect.execute();
}

On my MariaDB (which has character set latin1!), there is a table prep: "create table prep (text varchar(10));" When i run this code, the insert works perfectly: the japanese character gets converted to "?", as expected. However, the sqlSelect doesnt work at all and yields the feared error message "Illegal mix of collations (latin1_general_cs,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '='".

If I change the sql to "select * from prep where text=? collate latin1_general_cs", I get the error message "COLLATION 'latin1_general_cs' is not valid for CHARACTER SET 'utf8'". Therefore i concluded that my bind variable "corrupt" was not converted to latin1 for the where clause. However, I expected this conversion because it worked with the insert.

There is a working workaround: if you activate the two lines "//workaround", the String will be converted in java before giving it to the database. However, I feel that this should not be neccessary.

Running the same code on an oracle database yields no error. Can somebody run this on a mysql database?

So, does this look like a bug to you, or am I missing something.

Additional infos:
MariaDB version 10.0.14, MariaDB Client Library for java: 1.1.7
The jdbc connection string looks like this:

"jdbc:mariadb://localhost:3306/myDb?rewriteBatchedStatements=true&useServerPrepStmts=false"  

To start the database I use the default "my-large.ini".
I created the database with this code:

create database myDb
DEFAULT CHARACTER SET latin1 
COLLATE latin1_general_cs;

Update: I submitted a bug report to the mariaDB team: https://mariadb.atlassian.net/browse/CONJ-117

4
  • I just tested your code against MySQL 5.6.13 using MySQL Connector/J 5.1.26 and it ran fine without having to enable the workaround. Oct 15, 2014 at 19:15
  • ...and just now I tested it with MariaDB 10.0.13 and MariaDB Java Client 1.1.7 but the sqlInsert.execute(); statement fails with "java.sql.SQLDataException: Incorrect string value: '\xE3\x83\xBA\xC3\xA9' for column 'text' at row 1". prep.text is varchar(10) with collation latin1_general_ci (as it was for my MySQL test). If you can suggest how I might tweak my MariaDB test case to get past that please let me know. Oct 15, 2014 at 19:47
  • Thanks for trying this out. I am not sure why the MariaDB test case doesnt work. So I am providing more info about that testcase by editing my original post...
    – Sektat
    Oct 16, 2014 at 8:15
  • It does not work, because what you are trying to insert "non-latin1" characters into latin1 column. MariaDB JDBC uses UTF8. Server notices it cannot convert. this is what it is telling. That ConnectorJ succeeds a pure luck, and if you would run it with UTF8, you'll get the same result. You'd need a UTF8 column Nov 6, 2014 at 22:59

1 Answer 1

1

After experimenting with your test code using both MariaDB and MySQL I can confirm that they do seem to behave differently. However, I wouldn't necessarily call it a "bug" in MariaDB.

It appears to be a difference in the response from the database server itself. I tried applying the changes to a MariaDB 10.0.13 server using both the MariaDB Java Client 1.7 and the MySQL Connector/J 5.1.27 and they both gave the same result: MariaDB was able to automagically translate the first, second and fourth characters from UTF-8 to Latin1, but it choked on the third (Japanese) character U+30FA (0xE3 0x83 0xBA in UTF-8) because there is no Latin1 equivalent.

That makes sense. In some cases it might be convenient for MySQL to simply (and silently) map all un-translatable characters to '?' but that may not always be desirable. What you have devised as a workaround is telling Java to explicitly force non-Latin1 characters to '?' and then passing that to MariaDB. You feel that it should not be necessary, but I'm not sure I agree.

So, whether it is a "bug" in MariaDB really depends on whether MariaDB operating "as designed". In other words, just because MariaDB behaves differently than MySQL does not necessarily make it a bug in MariaDB. For example in MariaDB 5.5.31 they changed the behaviour of CAST() to fix a problem in the interpretation of binary string literals, the result being

This introduces an incompatibility with previous versions of MariaDB, and all versions of MySQL

That said, you could always report your issue as a MariaDB bug and see how they respond.

1
  • Thanks a lot, I will report the issue and provide the link.
    – Sektat
    Oct 17, 2014 at 8:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.