10

I have a table with a field a using encoding utf8 and collation utf8_unicode_ci:

CREATE TABLE dictionary (
    a varchar(128) NOT NULL
) DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

The collation utf8_unicode_ci is required for an efficient case insensitive search with extensions and ligations. For this purpose i have the index:

CREATE INDEX a_idx on dictionary(a);

Problem: Additionally i must ensure that all stored values of the field a are unique but in a case sensitive way. German example: "blühen" and "Blühen" must both be stored in the table. But adding "Blühen" a second time should not be possible.

Is there a build-in functionality in MySQL to have both?

Unfortunately it seems not to be possible to set the collation for the index in MySQL 5.1.

Solutions to this problem include a uniqueness check before insert or a trigger. Both are far less elegant than using a unique index.

4
  • Unfortunately, MySQL lacks features like indexes/materialised views or computed columns or function based indexes that other RDBMS have. I'd be interested to see how it's done of course...
    – gbn
    Jan 2, 2012 at 15:21
  • Would adding another column with a case sensitive collation and uniquness constraint work for you? Jan 2, 2012 at 15:22
  • I think this topic is a good help. stackoverflow.com/questions/4945349/…
    – MahanGM
    Jan 2, 2012 at 16:23
  • Thanks for your comments! Adding another column would be an option. For my particular purpose I think that adding a uniqueness-check before any of the few update or insert operation is the best option. My original post is a simplification. Actually, I have multiple columns, many read operations, and am afraid that adding more columns will become a performance issue due to limited memory. Jan 2, 2012 at 18:57

3 Answers 3

5

Well, there are 2 ways to accomplish this:

  1. using _bin collation
  2. change your datatype to VARBINARY

Case 1: using _bin collation

Create your table as follows:

CREATE TABLE `dictionary` (
 `a` VARCHAR(128) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
 UNIQUE KEY `idx_un_a` (`a`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

Please note:

  1. the datatype of the column a
  2. the UNIQUE index on column a

Case 2: using VARBINARY dataype

Create your table as follows:

CREATE TABLE `dictionary` (
 `a` VARBINARY(128) NOT NULL,
 UNIQUE KEY `idx_uniq_a` (`a`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

Please note:

  1. the new datatype VARBINARY
  2. the UNIQUE index on column a

So, both the above will solve your purpose. That is, they both will allow values like 'abc', 'Abc', 'ABC', 'aBc' etc but not allow the same value again if the case matches.

Please note that giving an "_bin" collation is different than using the binary datatype. So please feel free to refer to the following links:

  1. The BINARY and VARBINARY datatypes
  2. The _bin and binary Collations

I hope the above helps!

6
  • Thanks for the reply! I can not see how with this solution there will be an efficient (O(log(n)) and case insensitive search. Jan 2, 2012 at 18:43
  • @user1091141, ofcourse you can do case insensitive searches by changing the collation, e.g. query like SELECT * FROM dictionary WHERE a COLLATE utf8_general_ci = 'abc'. Sorry if my response wasn't clear on it but I guessed you could figure it out, here is a link - "dev.mysql.com/doc/refman/5.0/en/case-sensitivity.html". Regarding O(log(n)), I'm sorry but my maths ain't that strong, but I dont see why would the search be inefficient. Alternatively you can keep 2 a columns - one with general collation for case-insensitive searches and another with _bin for case-sensitive inserts
    – Abhay
    Jan 3, 2012 at 2:11
  • 2
    if i specify a different collation in the where-clause than what is defined in the table definition, MySQL will not use the index but does a full table scan. Full table scans can take a long time for large tables, thats why they can be regarded as inefficient. Doing EXPLAIN SELECT * FROM dictionary WHERE a COLLATE utf8_general_ci = 'abc' shows that all rows in the table are read. This is at least for my version of MySQL (5.0 and 5.1). Would be nice if it would be different. Jan 3, 2012 at 11:34
  • There could be 2 other things then that I could think of. One is that you keep your original table structure (without collation) and prevent the case-sensitive inserts using a BEFORE INSERT trigger. Second is that you may add my definition of "a" as a separate column to your original table
    – Abhay
    Jan 7, 2012 at 2:05
  • I agree with these two solutions. Unfortunately i was not able to write the required trigger, because I did not find out how to throw error messages with the trigger in MySQL 5.1. How is this possible? I would prefer the trigger solution, because I notice significant performance degradation when adding more columns. Jan 15, 2012 at 11:13
1

You can achieve this by adding additinal column 'column_lower'.

CREATE TABLE `dictionary` (
  `a` VARCHAR(128) NOT NULL,
  `a_lower` VARCHAR(128) NOT NULL,
  UNIQUE KEY `idx_un_a_lower` (`a_lower`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

Insert that goes like this:

insert into dictionary set a = x, a_lower = lower(x);

Select can now be case-insensitive:

select * from dictionary where a_lower like lower('search_term%')

Note that column which has index on it, can store at max 191 characters. MySQL can have at max 767 bytes long index, that is 767 / 4 (unicode can take up to 4 bytes if you use utf8mb4 collation) = 191.75 = 191 characters. If you use utf8 collation that takes up at max 3 bytes per character column can store at max 767 / 3 = 255 characters.

0
SELECT * FROM dictionary WHERE a COLLATE utf8_general_ci = 'abc'

Try this It will work .. it worked for me.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.