25

Everytime I use MySQL's CREATE TABLE AS SELECT ... all the tables/indexes being selected from are locked for the duration of the query. I do not really understand why? Is there any way around this?

Using: MySQL 5.1.41 and InnoDB

Added Example:

For example, the following query might take up to 10 minutes to complete:

CREATE TABLE temp_lots_of_data_xxx AS 
SELECT
    a.*
    b.*
    c.*
FROM a
LEFT JOIN b ON a.foo = b.foo
LEFT JOIN c ON a.foo = c.foo

Trying to update values in tables a, b or c during the above query will wait for the above query to finish first. I want to avoid this lock, as I am not interested in the most complete data in the created temp table.

p.s. SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED; yields no change in behavior.

1
  • Question: Are tables a,b, and c all InnoDB? Is the default storage engine for your setup InnoDB? Mar 21, 2012 at 3:18

7 Answers 7

20
+50

See also http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/

if not using replication, can change innodb_locks_unsafe_for_binlog to change this locking behaviour.

Or can dump the data to a file, then reload the data from a file. This also avoids the locks.

3
  • I think this answer is to the heart of the problem. Mar 23, 2012 at 16:14
  • I consider this answer to have a problem; the cited resource is in an external link and may be lost at any time.
    – user17161735
    Oct 21, 2021 at 20:54
  • Well the external link goes into much detail. Far more than I could explain without just paraphrasing. Did copy the exact things to change or even the workaround, which doesn need the explanation. It seems to be a archive link anyway Oct 24, 2021 at 16:53
17

Have you tried to do the operation in 2 phases (first Create the table, then Insert the values) and having set the lowest isolation level?:

CREATE TABLE temp_lots_of_data_xxx AS 
    SELECT
        a.*
        b.*
        c.*
    FROM a
        LEFT JOIN b ON a.foo = b.foo
        LEFT JOIN c ON a.foo = c.foo
    WHERE FALSE

INSERT INTO temp_lots_of_data_xxx
    SELECT
        a.*
        b.*
        c.*
    FROM a
        LEFT JOIN b ON a.foo = b.foo
        LEFT JOIN c ON a.foo = c.foo
1
  • I do like this, but do lots of smaller inserts. SELECT MIN(id),MAX(id) FROM a; CREATE TABLE ... ; for (id = min;id < max; id+1000) { INSERT INTO ... WHERE a.id BETWEEN $id AND $id+999 } ; Sort of thing. that way the lock is only held for each small insert. Put each insert into a seperate transaction so lock is released in between. Mar 23, 2012 at 15:38
4

If your engine is InnoDB than it uses automatic row-level locking. Update statements have higher priority then select statements so that's why you are having this problem.

In order to workaround this issue you could SET LOW_PRIORITY_UPDATES=1 and then you should be able to run your command. But this does not fully fit your case. So you could also give higher priority to a SELECT statement as well. To give a specific SELECT statement higher priority, use the HIGH_PRIORITY attribute.

CREATE TABLE temp_lots_of_data_xxx AS 
SELECT HIGH_PRIORITY
    a.*
    b.*
    c.*
FROM a
LEFT JOIN b ON a.foo = b.foo
LEFT JOIN c ON a.foo = c.foo

For details please refer to this page table-locking-issues this page select-syntax and also this page: option_mysqld_low-priority-updates

5
  • Unfortunately this does not help. While the temp table is created I cannot update a,b or c.
    – clops
    Mar 22, 2012 at 13:07
  • Moreover, as I see form the docs, --low-priority-updates does not affect InnoDB Tables.
    – clops
    Mar 22, 2012 at 13:10
  • Sorry I could not get what you mean. Do you need to update a,b,c? It does not matter whether you create temp or permanent table. The problem is create locks tables for select. So you need to run select before create and SELECT HIGH_PRIORITY gives you this opportunity. Did you have a chance to give it a try?
    – huzeyfe
    Mar 22, 2012 at 13:12
  • I cannot run select before create, as the overall data quantity is above the available memory, so I have to use CREATE TABLE AS SELECT ... I am not interested in how accurate the data is, I do not want the SELECT statement to lock any tables.
    – clops
    Mar 22, 2012 at 13:56
  • 1
    OK thin I got it. You are trying to update a,b,c in another query while this query is running? If so, I am sorry I misunderstood the problem as this is not related with your case but it is related with mysql lock issues. Hmm.. Let me think about this further..
    – huzeyfe
    Mar 22, 2012 at 14:12
3

I didn't test this, but you might have a try with

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
CREATE TABLE ...
COMMIT ; /*See comment by Somnath Muluk*/

But be aware:

Select statements are performed in a nonlocking fashion, but a possible earlier version of a row might be used. Thus, using this isolation level, such reads are not consistent. This is also called a “dirty read.”

Read more about it here:

MySQL SET TRANSACTION manual entry

EDIT: added the COMMIT ;

3
  • tried this as the very first action, unfortunately no joy in MySQL 5.1 :(
    – clops
    Mar 19, 2012 at 11:09
  • Add COMMIT ; after SET TRANSACTION..CREATE TABLE .... Mar 19, 2012 at 12:58
  • No Joy. Unfortunately no change at all.
    – clops
    Mar 19, 2012 at 14:30
2

My psychic debugging skills suggest that you're trying to access the tables/indexes while you're debugging the query that uses them.

In general, I'd not be too surprised if a CREATE TABLE query locks all the tables and indexes from which it is reading.

If my psychic premonition is right, I'd suggest letting the query finish before accessing the tables and indexes it is using.

(Please correct me if I made any wrong assumptions.)

7
  • 1
    That is what the purpose of the question is -- how do I avoid the locks? I am created a temp data table for a high-load environment and waiting for a lock is not an option
    – clops
    Mar 19, 2012 at 9:43
  • @clops: Maybe you'd be better off creating a permanent VIEW instead of a temporary table? It could be much more encompassing, and you could just query the the VIEW for the relevant information at any given moment. That way, you avoid the pitfalls associated with a CREATE TABLE. Heck, you might even see a performance boost. (But I doubt that.) Mar 19, 2012 at 10:04
  • sorry, views are out of question for performance reasons. The queries I am running to create temporary tables often take more than 10 minutes to complete
    – clops
    Mar 19, 2012 at 11:08
  • 1
    More than 10 minutes? Something else is probably going wrong. You may want to re-examine your indexes to make sure they match up with your queries. I remember the first time I got it right - I was joining two tables of one million rows each. Before the indexes, it took about 25 minutes on my 100Mhz/32MB RAM computer. After I got it right, the same query took about 2 seconds. Ah... memories :) Mar 19, 2012 at 11:22
  • Unfortunately we are talking ERP data and reports being generated on that data by the Customers, they are not limited in the variation of data they may require and having an index for everything is hardly possible
    – clops
    Mar 19, 2012 at 12:51
2

All InnoDB locks held by a transaction are released when the transaction is committed or aborted. Thus, it does not make much sense to invoke LOCK TABLES on InnoDB tables in autocommit=1 mode because the acquired InnoDB table locks would be released immediately.

As read here

EDIT and this:

You cannot lock additional tables in the middle of a transaction because LOCK TABLES performs an implicit COMMIT and UNLOCK TABLES.

0
2

I'm not as experienced in mysql but I ran into the same problem in mssql. The solution there was to run the "create table as select..." with zero rows so that it only creates the table with the appropriate structure and releases the locks immediately. Then use "insert into" with the same select statement to populate the table, which won't hold any schema locks while it runs.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.