5

I have a lot of tables and because of some reasons I need to adjust auto increment value for this tables on application startup.

I try to do this:

mysql> select max(id) from item;
+----------+
| max(id)  |
+----------+
| 97972232 |
+----------+
1 row in set (0.05 sec)

mysql> alter table item auto_increment=1097972232;

In another session:

afrolov@A1-DB1:~$ mysql -u root -e "show processlist" | grep auto_increment
472196  root    localhost       test    Query   39      copy to tmp table       alter table item auto_increment=1097972232

MySQL is starting to rebuild table! Why MySQL need to do it? How can I avoid rebuilding huge tables while adjusting auto_increment value?

MySQL 5.0, InnoDB.
Table definition:

 CREATE TABLE `item` (
      `id` bigint(20) NOT NULL auto_increment,
      `item_res_id` int(11) NOT NULL default '0',
      `stack_count` int(11) NOT NULL default '0',
      `position` int(11) NOT NULL default '0',
      `place` varchar(15) NOT NULL default '',
      `counter` int(11) NOT NULL default '-1',
      `is_bound` tinyint(4) NOT NULL default '0',
      `remove_time` bigint(20) NOT NULL default '-1',
      `rune_res_id` int(11) default NULL,
      `rune_id` bigint(20) default NULL,
      `avatar_id` bigint(20) NOT NULL,
      `rune_slot_res_id` int(11) default NULL,
      `is_cursed` tinyint(4) NOT NULL,
      PRIMARY KEY  (`id`),
      UNIQUE KEY `avatar_id` (`avatar_id`,`place`,`position`),
      UNIQUE KEY `rune_id` (`rune_id`),
      KEY `idx_item_res_id` (`item_res_id`)
    ) ENGINE=InnoDB AUTO_INCREMENT=97972233 DEFAULT CHARSET=utf8;

About why I have to do this. Long story short i want to workaround mysql innodb issue about reseting auto_increment value on server restart. Sometimes we copy rows from our tables to another tables and we must keep rows id unchanged. When we add one row (with id=1 for example) to table1, copy row to table2 , delete row from table1 and restart MySQL, then when we create a new one row in table1 this row will get id=1 too. So if we will have to copy row to table2 we get unique constraint violation. We already have a lot of code and it will be hard to rewrite it all. Adjusting autoincrement value seems the easiest way to fix this problem.

Added:

MySQL 5.5 - all the same :(

3
  • MySQL 5.0 is not a version, its a whole family of versions. Please provide all three digits of the version. If you don't know, do show variables like '%version%'
    – derobert
    Commented Nov 10, 2010 at 17:33
  • 3
    Also, I need to adjust auto increment values ... on application startup strikes me as you're doing it wrong.
    – derobert
    Commented Nov 10, 2010 at 17:54
  • 2
    Why on each forum there is a guy, who exactly know than i am doing it wrong??? :) Commented Nov 10, 2010 at 20:13

5 Answers 5

17
+100

simply add a temporary record that has desired auto_increment_id-1 to each table, and remove the record after that, quick and easy, but probably too dirty

example:

insert into item set id=1097972232-1;

after the execution, the next auto_increment will be 1097972232, which is what you desired

this can avoid slowness

6
  • And then probably delete from item where id=1097972232-1 so you don't have a junk row. But this does work. I've used this trick to allow active inserts to continue while leaving enough room to move in some rows from another (such as a backup, etc). Commented Nov 14, 2010 at 1:16
  • Wow! Really good idea. :) Thank you very much! I will have some troubles with foreign keys but this is minor troubles. Commented Nov 14, 2010 at 10:34
  • 1
    Actually insert + rollback is enough. Commented Nov 15, 2010 at 8:59
  • @Andrew Frolov: bingo! hopefully it won't rollback the auto_increment as well?
    – ajreal
    Commented Nov 15, 2010 at 9:02
  • 1
    auto_increment won't rollback. I checked this. Thank you again :) Commented Nov 15, 2010 at 9:16
7

This is a documented "feature" of MySQL:

If you use any option to ALTER TABLE other than RENAME, MySQL always creates a temporary table, even if the data wouldn't strictly need to be copied (such as when you change the name of a column). For MyISAM tables, you can speed up the index re-creation operation (which is the slowest part of the alteration process) by setting the myisam_sort_buffer_size system variable to a high value.

http://dev.mysql.com/doc/refman/5.0/en/alter-table.html

MySQL 5.1 and 5.5 support a few more alter table operations w/o a temporary table, but changing the auto_increment is not documented to be one of those.

Why do you need to change the auto_increment value, anyway? This isn't something you should be doing routinely.

2
  • Thank you for advice. Tomorrow I will check this behavior on 5.1 and 5.5. I updated the first post to answer why I have to do it. Commented Nov 10, 2010 at 20:11
  • Checked on 5.5. All the same :( Commented Nov 13, 2010 at 10:32
3

There is no simple way to get around the AUTO_INCREMENT attribute default behavior in MySQL, and even if you find a way, I wouldn't recommend you to do so, since it is the best way to run into problems in the short-term. AUTO_INCREMENT values are not meant to be adjusted or reseted in a production environment.

One possible solution to your problem could be to denormalize your model a little. The idea is to move the AUTO_INCREMENT field to a side table where you don't have to copy or delete rows. All you have to do then is to get a new id value from this side table when you create a new item, and keep the existing id value when you copy rows from one table to another.

To achieve this we will use a trigger that will create a new id for us and assign it to our item record. The id field of the item table has to be nullable for this to work, so we have to replace the primary key by a unique index.

This model change would be completely transparent for your application, so you would have no change to make in your application code.

Here are some example scripts. Let's say we have two item tables in our database, with some common rows, and some rows that need to be moved from first table to second table:

CREATE TABLE `item1` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `item_res_id` int(11) NOT NULL DEFAULT '0',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `item2` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `item_res_id` int(11) NOT NULL DEFAULT '0',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO item1 (item_res_id) VALUES (1);
INSERT INTO item1 (item_res_id) VALUES (2);
INSERT INTO item2 (item_res_id) VALUES (1);

If we try to move some data from one table to another and then restart your server, we will encounter the issue of AUTO_INCREMENT value reset. So we will slightly modify our model as follows:

New model with side table

We will proceed in several steps to migrate our data model. The DDL statements in the following migration scripts have been generated using neXtep Designer IDE.

  • First we create a new item_keys table that will hold the AUTO_INCREMENT field:
-- Creating table 'item_keys'
CREATE TABLE item_keys ( 
   id BIGINT(20) UNSIGNED NOT NULL
  ,key_ctime TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
) Engine=InnoDB default charset=utf8;

-- Creating Primary Key constraint 'PRIMARY' on table 'item_keys'
ALTER TABLE item_keys ADD CONSTRAINT PRIMARY KEY (id);
  • But before activating the AUTO_INCREMENT attribute, we must insert existing ids into our new table:
-- Initializing item_keys with existing ids
INSERT INTO item_keys (id)
    SELECT i1.id
    FROM item1 i1
        LEFT JOIN item_keys ik ON ik.id = i1.id
    WHERE ik.id IS NULL
;

INSERT INTO item_keys (id)
    SELECT i2.id
    FROM item2 i2
        LEFT JOIN item_keys ik ON ik.id = i2.id
    WHERE ik.id IS NULL
;
  • We can now activate the AUTO_INCREMENT attribute, and initialize its value for future inserts:
-- Activating auto_increment constraint...
ALTER TABLE item_keys MODIFY id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT;

-- Initializing auto_increment value
SELECT @inc_value := MAX(id) FROM item_keys;
SET @alter_query = CONCAT('ALTER TABLE item_keys AUTO_INCREMENT=',@inc_value); 
PREPARE alter_query FROM @alter_query; 
EXECUTE alter_query; 
DEALLOCATE PREPARE alter_query; 
  • Then we can alter the item1 and item2 tables to replace the primary key by a unique index, and reference the primary key of the item_keys table:
-- De-activating auto_increment constraint...
ALTER TABLE item1 MODIFY id BIGINT(20) UNSIGNED NOT NULL;
-- Dropping constraint 'PRIMARY'...
ALTER TABLE item1 DROP PRIMARY KEY;
ALTER TABLE item1 MODIFY id BIGINT(20) UNSIGNED NULL;
-- Creating index 'item1_uk'...
CREATE UNIQUE INDEX item1_uk ON item1 (id);
-- Creating Foreign Key constraint 'item1_keys_fk' on table 'item1'
ALTER TABLE item1 ADD 
   CONSTRAINT item1_keys_fk FOREIGN KEY item1_keys_fk
      (id) REFERENCES item_keys
      (id)
;
-- De-activating auto_increment constraint...
ALTER TABLE item2 MODIFY id BIGINT(20) UNSIGNED NOT NULL;
-- Dropping constraint 'PRIMARY'...
ALTER TABLE item2 DROP PRIMARY KEY;
ALTER TABLE item2 MODIFY id BIGINT(20) UNSIGNED NULL;
-- Creating index 'item2_uk'...
CREATE UNIQUE INDEX item2_uk ON item2 (id);
-- Creating Foreign Key constraint 'item2_keys_fk' on table 'item2'
ALTER TABLE item2 ADD 
   CONSTRAINT item2_keys_fk FOREIGN KEY item2_keys_fk
      (id) REFERENCES item_keys
      (id)
;
  • Finally, we just have to create the triggers that will manage the ids creation for us:
-- Creating trigger 'tr_item1_bi' on table 'item1'...
DELIMITER |;
CREATE TRIGGER tr_item1_bi BEFORE INSERT ON item1
FOR EACH ROW
BEGIN
   IF (NEW.id IS NULL) THEN

        -- If no item id has been specified in the INSERT statement, it
        -- means we want to create a new item. We insert a new record
        -- into the item_keys table to get an item id.
        INSERT INTO item_keys (
            key_ctime
          )
        VALUES (NOW());

        SET NEW.id = LAST_INSERT_ID();
    END IF;
END;
|;
-- Creating trigger 'tr_item2_bi' on table 'item2'...
DELIMITER |;
CREATE TRIGGER tr_item2_bi BEFORE INSERT ON item2
FOR EACH ROW
BEGIN
   IF (NEW.id IS NULL) THEN

        -- If no item id has been specified in the INSERT statement, it
        -- means we want to create a new item. We insert a new record
        -- into the item_keys table to get an item id.
        INSERT INTO item_keys (
            key_ctime
          )
        VALUES (NOW());

        SET NEW.id = LAST_INSERT_ID();
    END IF;
END;
|;

Now we can move data from one table to another, keeping the ids unchanged, and if we restart the server, the AUTO_INCREMENT value in the item_keys will stay the same.

--------------
INSERT INTO item2
    SELECT i1.*
    FROM item1 i1
        LEFT JOIN item2 i2
            ON i2.id = i1.id
    WHERE i2.id IS NULL
--------------
Query OK, 1 row affected (0.04 sec)
Records: 1  Duplicates: 0  Warnings: 0

--------------
DELETE FROM item1
--------------
Query OK, 2 rows affected (0.00 sec)

--------------
INSERT INTO item1 (item_res_id) VALUES (3)
--------------
Query OK, 1 row affected (0.00 sec)

--------------
SELECT * FROM item1
--------------

+------+-------------+
| id   | item_res_id |
+------+-------------+
|    3 |           3 |
+------+-------------+
1 row in set (0.00 sec)

--------------
SELECT * FROM item2
--------------

+------+-------------+
| id   | item_res_id |
+------+-------------+
|    1 |           1 |
|    2 |           2 |
+------+-------------+
2 rows in set (0.00 sec)

--------------
SELECT * FROM item_keys
--------------

+----+---------------------+
| id | key_ctime           |
+----+---------------------+
|  1 | 2010-11-14 10:31:21 |
|  2 | 2010-11-14 10:31:21 |
|  3 | 2010-11-14 10:31:46 |
+----+---------------------+
3 rows in set (0.00 sec)
5
  • Thank you for advice. This is good idea, but this approach will cause some troubles with perfomance. Commented Nov 14, 2010 at 10:35
  • @Andrew, what level of performance do you need for what type of operation (items insertion, items mass insertion, items deletion, items selection)? Can you provide some metrics? Have you run a benchmark so you can say that this is not good enough? Commented Nov 14, 2010 at 10:55
  • @Andrew, I would understand if you found this solution too complicated compared to the temporary record insertion, but I do not understand the argument of the performance. Commented Nov 14, 2010 at 11:02
  • Now I already have perfomance problems. And about 60% of sql statements works with items table. If I add trigger and insertion into item_keys table my troubles only become bigger. I don't think that benchmarking are needed here. Your approach is very interesting too and probably I will use it next time, when I will not able to change application. Commented Nov 14, 2010 at 12:22
  • @Andrew, there is indeed an overhead (although minor in my opinion) with this solution, and I understand you don't want to take time to measure it, especially when you have the option to change the application code with a simple solution. I'm glad I could bring you a different perspective on this problem. Commented Nov 14, 2010 at 14:40
1

If you need to maintain unique IDs between two or more servers, don't use this alter table method of resetting the auto_increment each time. It'd be easier to change the increment's increment so that each server will generate unique IDs without intervention. For two servers, you set one to start at 0 and one to start at 1, with an increment of 2 - afterward one will generate even IDs, the other will generate odds. With 3 or more servers, you just set the initial values to 0/1/2 with increments of 3, for four it's 0/1/2/3 with inc of 4, etc...

Details on the server-side settings here:

http://dev.mysql.com/doc/refman/5.1/en/replication-options-master.html#sysvar_auto_increment_increment

This way you only have to reset the auto_increment once per table per server, and afterward they'll take care of the uniqueness problem automatically.

3
  • This doesn't make sense for me. I am copying rows inside one database. Commented Nov 12, 2010 at 21:40
  • You can set the different increments on different tables within the same DB as well. If table A starts at 0 and increments by2, and table B starts at 1 and increments by 2, then you can copy rows back and forth at will without having worry about key collisions.
    – Marc B
    Commented Nov 13, 2010 at 4:48
  • In my case ids are generating only in table A. Commented Nov 13, 2010 at 10:50
-1

Isn't it :

ALTER TABLE item AUTO_INCREMENT=1;

?

Source

3
  • What do you mean "rebuild" the table ? It just changes the auto_increment default value. Is your table already "full" ? Are you changing for an increment already used ? Are you using a multi-column index in this table ?
    – Shikiryu
    Commented Nov 8, 2010 at 15:51
  • 2
    almost all ALTER TABLE statements cause a rebuild of the table.
    – nos
    Commented Nov 12, 2010 at 21:11
  • @nos and the -1 : I answered his question loonnng before his edit making my answer, now, inadequate but at first, it helps. Wasn't there for 4 days. But thanks for the downvote. Anyway, as you said almost all ALTER TABLE will do, but in my experience, a modification of a AUTO_INCREMENT won't take long (did it on a million with no problem whatsoever) unless you use it on an existent value, etc.
    – Shikiryu
    Commented Nov 15, 2010 at 9:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.