7

First, I apologize if this has been asked before - indeed I'm sure it has, but I can't find it/can't work out what to search for to find it.

I need to generate unique quick reference id's, based on a company name. So for example:

Company Name                Reference
Smiths Joinery              smit0001
Smith and Jones Consulting  smit0002
Smithsons Carpets           smit0003

These will all be stored in a varchar column in a MySQL table. The data will be collected, escaped and inserted like 'HTML -> PHP -> MySQL'. The ID's should be in the format depicted above, four letters, then four numerics (initially at least - when I reach smit9999 it will just spill over into 5 digits).

I can deal with generating the 4 letters from the company name, I will simply step through the name until I have collected 4 alpha characters, and strtolower() it - but then I need to get the next available number.

What is the best/easiest way to do this, so that the possibility of duplicates is eliminated?

At the moment I'm thinking:

$fourLetters = 'smit';
$query = "SELECT `company_ref`
            FROM `companies`
          WHERE
            `company_ref` LIKE '$fourLetters%'
          ORDER BY `company_ref` DESC
          LIMIT 1";
$last = mysqli_fetch_assoc(mysqli_query($link, $query));
$newNum = ((int) ltrim(substr($last['company_ref'],4),'0')) + 1;
$newRef = $fourLetters.str_pad($newNum, 4, '0', STR_PAD_LEFT);

But I can see this causing a problem if two users try to enter company names that would result in the same ID at the same time. I will be using a unique index on the column, so it would not result in duplicates in the database, but it will still cause a problem.

Can anyone think of a way to have MySQL work this out for me when I do the insert, rather than calculating it in PHP beforehand?

Note that actual code will be OO and will handle errors etc - I'm just looking for thoughts on whether there is a better way to do this specific task, it's more about the SQL than anything else.

EDIT

I think that @EmmanuelN's suggestion of using a MySQL trigger may be the way to handle this, but:

  • I am not good enough with MySQL, particularly triggers, to get this to work, and would like a step-by-step example of creating, adding and using a trigger.
  • I am still not sure whether this will will eliminate the possibility of two identical ID's being generated. See what happens if two rows are inserted at the same time that result in the trigger running simultaneously, and produce the same reference? Is there any way to lock the trigger (or a UDF) in such a way that it can only have one concurrent instance?.

Or I would be open to any other suggested approaches to this problem.

8
  • Have you thought about using trigger and generate that id inside a trigger?
    – Emmanuel N
    Commented Nov 5, 2011 at 15:22
  • No, but SQL is definitely my weakest subject in terms of this type of project, I don't really know how to use triggers effectively - could you give me a basic run down of how this would be done? If you could point me in the direction of some relevant reading material I'd be happy :-)
    – DaveRandom
    Commented Nov 5, 2011 at 15:27
  • 1
    This article talks about auto generation on GUIDs using mysql, I think it will give you somewhere to start
    – Emmanuel N
    Commented Nov 5, 2011 at 15:30
  • Thanks, I'll have a play around and see what I can come up with
    – DaveRandom
    Commented Nov 5, 2011 at 15:43
  • This seems like it would reduce the possibility of duplicates being generated (by being faster), but not eliminate it entirely - what happens if two rows are inserted at the same time that result in the trigger running simultaneously, and produce the same reference? Is there any way to lock the trigger (or a UDF) in such a way that it can only have one concurrent instance? Or some other way to get around this?
    – DaveRandom
    Commented Nov 5, 2011 at 15:57

6 Answers 6

7
+100

If you are using MyISAM, then you can create a compound primary key on a text field + auto increment field. MySQL will handle incrementing the number automatically. They are separate fields, but you can get the same effect.

CREATE TABLE example (
company_name varchar(100),
key_prefix char(4) not null,
key_increment int unsigned auto_increment,
primary key co_key (key_prefix,key_increment)
) ENGINE=MYISAM;

When you do an insert into the table, the key_increment field will increment based on the highest value based on key_prefix. So insert with key_prefix "smit" will start with 1 in key_inrement, key_prefix "jone" will start with 1 in key_inrement, etc.

Pros:

  • You don't have to do anything with calculating numbers.

Cons:

  • You do have a key split across 2 columns.
  • It doesn't work with InnoDB.
4
  • This looks to be a good option and a good candidate for the accepted answer - presumably doing this I just INSERT the alphas into the key_prefix column, and key_increment generates a numeric part? It will make the SELECT logic slightly more complicated when searching by a user-input reference, but aside from that minor point this does fulfill all the requirements... many thanks!
    – DaveRandom
    Commented Nov 8, 2011 at 21:23
  • 1
    My storage engine is current InnoDB (although I am not particularly attached to this) - out of interest, why can you only do this with MyISAM?
    – DaveRandom
    Commented Nov 8, 2011 at 21:25
  • I wondered that too - from the mysql manual under Innodb Limits "For an AUTO_INCREMENT column, you must always define an index for the table, and that index must contain just the AUTO_INCREMENT column. In MyISAM tables, the AUTO_INCREMENT column may be part of a multiple-column index." Commented Nov 9, 2011 at 16:06
  • 1
    MyISAM uses table locking on modifications. So the table locks, the highest ID is retrieved, incremented, then does the insert. InnoDB uses record locking (like most "modern" DBs). On startup, InnoDB figures out the highest auto increment, then caches it and uses that for all new inserts IDs. Commented Nov 10, 2011 at 2:25
3

How about this solution with a trigger and a table to hold the company_ref's uniquely. Made a correction - the reference table has to be MyISAM if you want the numbering to begin at 1 for each unique 4char sequence.

DROP TABLE IF EXISTS company;
CREATE TABLE company (
  company_name varchar(100) DEFAULT NULL,
  company_ref char(8) DEFAULT NULL
) ENGINE=InnoDB

DELIMITER ;;
CREATE TRIGGER company_reference BEFORE INSERT ON company
FOR EACH ROW BEGIN
   INSERT INTO reference SET company_ref=SUBSTRING(LOWER(NEW.company_name), 1, 4), numeric_ref=NULL;
   SET NEW.company_ref=CONCAT(SUBSTRING(LOWER(NEW.company_name), 1, 4), LPAD(CAST(LAST_INSERT_ID() AS CHAR(10)), 4, '0'));
END ;;
DELIMITER ;

DROP TABLE IF EXISTS reference;
CREATE TABLE reference (
company_ref char(4) NOT NULL DEFAULT '',
numeric_ref int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (company_ref, numeric_ref)
) ENGINE=MyISAM;

And for completeness here is a trigger that will create a new reference if the company name is altered.

DROP TRIGGER IF EXISTS company_reference_up;
DELIMITER ;;
CREATE TRIGGER company_reference_up BEFORE UPDATE ON company
FOR EACH ROW BEGIN
   IF NEW.company_name <> OLD.company_name THEN
      DELETE FROM reference WHERE company_ref=SUBSTRING(LOWER(OLD.company_ref), 1, 4) AND numeric_ref=SUBSTRING(OLD.company_ref, 5, 4);
      INSERT INTO reference SET company_ref=SUBSTRING(LOWER(NEW.company_name), 1, 4), numeric_ref=NULL;
      SET NEW.company_ref=CONCAT(SUBSTRING(LOWER(NEW.company_name), 1, 4), LPAD(CAST(LAST_INSERT_ID() AS CHAR(10)), 4, '0'));
   END IF;
END;
;;
DELIMITER ;
1
  • I am going to accept Brent's answer as he has provided a good explaination of why this is the right approach, but if I could accept 2 answers you would get the other accept as it is a combination of your answers that has helped me find the right solution to my problem, as you provided a nice example of the creation of the trigger. The problem I was having is the DELIMITER change while running the creation query - I somehow missed that when I initially tried to do it. You definitely deserve (at least) the +1 I gave you...
    – DaveRandom
    Commented Nov 14, 2011 at 15:24
3

Given you're using InnoDB, why not use an explicit transaction to grab an exclusive row lock and prevent another connection from reading the same row before you're done setting a new ID based on it?

(Naturally, doing the calculation in a trigger would hold the lock for less time.)

mysqli_query($link, "BEGIN TRANSACTION");
$query = "SELECT `company_ref`
            FROM `companies`
          WHERE
            `company_ref` LIKE '$fourLetters%'
          ORDER BY `company_ref` DESC
          LIMIT 1
          FOR UPDATE";
$last = mysqli_fetch_assoc(mysqli_query($link, $query));
$newNum = ((int) ltrim(substr($last['company_ref'],4),'0')) + 1;
$newRef = $fourLetters.str_pad($newNum, 4, '0', STR_PAD_LEFT);
mysqli_query($link, "INSERT INTO companies . . . (new row using $newref)");
mysqli_commit($link);

Edit: Just to be 100% sure I ran a test by hand to confirm that the second transaction will return the newly inserted row after waiting rather than the original locked row.

Edit2: Also tested the case where there is no initial row returned (Where you would think there is no initial row to put a lock on) and that works as well.

1
  1. Ensure you have an unique constraint on the Reference column.
  2. Fetch the current max sequential reference the same way you do it in your sample code. You don't actually need to trim the zeroes before you cast to (int), '0001' is a valid integer.
  3. Roll a loop and do your insert inside.
  4. Check affected rows after the insert. You can also check the SQL state for a duplicate key error, but having zero affected rows is a good indication that your insert failed due to inserting an existing Reference value.
  5. If you have zero affected rows, increment the sequential number, and roll the loop again. If you have non-zero affected rows, you're done and have an unique identifier inserted.
4
  • This approach is slightly ineffective, as you will reattempt inserts until you finally manage to avoid conflict. Still, you're starting from the current maximum Reference value + 1, and unless you have a LOT of inserts, you are very likely to succeed at the first try, and practically guaranteed to succeed at the second try.
    – lanzz
    Commented Nov 8, 2011 at 18:05
  • This was what I was thinking I would end up doing until Brent Baisley's answer, I only don't like it because of the very remote possibility of a lengthy loop trying to find the next available ID in the event of a high volume of independent inserts, as you mention in your comment. I know this is very much programming around a situation that should never occur, but I would still like to avoid if possible.
    – DaveRandom
    Commented Nov 8, 2011 at 21:31
  • Brent's solution is truly better than mine, I actually didn't know myisam tracks the autoincrement for the whole primary key instead of the actual column.
    – lanzz
    Commented Nov 9, 2011 at 8:34
  • Here's an interesting MySQL feature I wasn't aware of before: The ON DUPLICATE KEY UPDATE syntax will actually handle doing the loop in the database layer. dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html Commented Nov 10, 2011 at 16:30
0

Easiest way to avoid duplicate values for the reference column is to add a unique constraint. So if multiple processes try to set to the same value, MySQL will reject the second attempt and throw an error.

ALTER TABLE table_name ADD UNIQUE KEY (`company_ref`);

If I were faced with your situation, I would handle the company reference id generation within the application layer, triggers can get messy if not setup correctly.

0

A hacky version that works for InnoDB as well.

Replace the insert to companies with two inserts in a transaction:

    INSERT INTO __keys
      VALUES (LEFT(LOWER('Smiths Joinery'),4), LAST_INSERT_ID(1))
    ON DUPLICATE KEY UPDATE
      num = LAST_INSERT_ID(num+1);

    INSERT INTO __companies (comp_name, reference)
    VALUES ('Smiths Joinery',
            CONCAT(LEFT(LOWER(comp_name),4), LPAD(LAST_INSERT_ID(), 4, '0')));

where:

    CREATE TABLE  `__keys` (
      `prefix` char(4) NOT NULL,
      `num` smallint(5) unsigned NOT NULL,
      PRIMARY KEY (`prefix`)
    ) ENGINE=InnoDB COLLATE latin1_general_ci;

    CREATE TABLE  `__companies` (
      `comp_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
      `comp_name` varchar(45) NOT NULL,
      `reference` char(8) NOT NULL,
      PRIMARY KEY (`comp_id`)
    ) ENGINE=InnoDB COLLATE latin1_general_ci;

Notice:

  • latin1_general_ci can be replaced with utf8_general_ci,
  • LEFT(LOWER('Smiths Joinery'),4) would better become a function in PHP

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.