2

Recently I encountered a deadlock when deleting records (Note that the isolation level is REPEATABLE READ, MySQL 5.7)

Here is the repro steps

1 Create a new table

CREATE TABLE `t` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `name` varchar(32) NOT NULL,
  PRIMARY KEY (`id`),
  KEY `p_name` (`name`)
) ENGINE=InnoDB CHARSET=utf8;

2 Prepare 3 records

insert into t (name) value ('A'), ('C'), ('D');

3

+====================================+============================================================+
|             Session A              |                         Session B                          |
+====================================+============================================================+
| begin;                             |                                                            |
+------------------------------------+------------------------------------------------------------+
|                                    | begin;                                                     |
+------------------------------------+------------------------------------------------------------+
| delete from t where name = 'C';    |                                                            |
+------------------------------------+------------------------------------------------------------+
|                                    | delete from t where name = 'C';  --Blocked!                |
+------------------------------------+------------------------------------------------------------+
| insert into t (name) values ('B'); |                                                            |
+------------------------------------+------------------------------------------------------------+
|                                    | ERROR 1213 (40001): Deadlock found when trying to get lock |
+------------------------------------+------------------------------------------------------------+

The result of show engine innodb status as below shown (LATEST DETECTED DEADLOCK section)

LATEST DETECTED DEADLOCK
------------------------
*** (1) TRANSACTION:
TRANSACTION 3631, ACTIVE 21 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 13, OS thread handle 123145439432704, query id 306 localhost root updating
delete from t where name = 'C'
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 69 page no 4 n bits 72 index p_name of table `jacky`.`t` trx id 3631 lock_mode X waiting
Record lock, heap no 4 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
 0: len 1; hex 43; asc C;;
 1: len 8; hex 8000000000000018; asc         ;;

*** (2) TRANSACTION:
TRANSACTION 3630, ACTIVE 29 sec inserting
mysql tables in use 1, locked 1
5 lock struct(s), heap size 1136, 4 row lock(s), undo log entries 2
MySQL thread id 14, OS thread handle 123145439711232, query id 307 localhost root update
insert into t (name) values ('B')
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 69 page no 4 n bits 72 index p_name of table `jacky`.`t` trx id 3630 lock_mode X
Record lock, heap no 4 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
 0: len 1; hex 43; asc C;;
 1: len 8; hex 8000000000000018; asc         ;;

*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 69 page no 4 n bits 72 index p_name of table `jacky`.`t` trx id 3630 lock_mode X locks gap before rec insert intention waiting
Record lock, heap no 4 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
 0: len 1; hex 43; asc C;;
 1: len 8; hex 8000000000000018; asc         ;;

As the Innodb status shown, session B is waiting next-key lock C, and session A hold a record lock C and waiting gap lock on C;


As we all know that

DELETE FROM ... WHERE ... sets an exclusive next-key lock on every record the search encounters

A next-key lock is a combination of a record lock on the index record and a gap lock on the gap before the index record.

Q1: I guess if session B firstly got the gap lock (part of next-key), and then waiting for the record lock. Thereby, the latter insert in session A was blocked by session B (due to the gap lock), and eventually result in a dead lock. Right?

Q2: As the C is purged from an index, does the gap lock hold by session B should be ('A', 'D')? If so, why the session A is waiting the insert intension lock on range (, 'C')?

Q3: Why session B has 1 row lock(s), and session A has 4 row lock(s)?


Q4: When change index p_name to a unique index, we still get the deadlock due to gap lock, it's weird. It behaves different from official doc which states only record lock is required.

DELETE FROM ... WHERE ... sets an exclusive next-key lock on every record the search encounters. However, only an index record lock is required for statements that lock rows using a unique index to search for a unique row.


However, it is okay when using primary key id to execute delete (steps as below shown). Is this a bug in MySQL?

1 Prepare data

delete from t;
insert into t (id, name) value (1, 'A'), (3, 'C'), (5, 'D');

2

+-------------------------------------------+--------------------------------------+
|                 Session A                 |              Session B               |
+-------------------------------------------+--------------------------------------+
| begin;                                    |                                      |
|                                           | begin;                               |
| delete from t where id = 3;               |                                      |
|                                           | delete from t where id = 3; Blocked! |
| insert into t (id, name) values (2, 'B'); |                                      |
|                                           |                                      |
| commit;                                   |                                      |
+-------------------------------------------+--------------------------------------+
1
  • I don't know the details, but it seems that messing around in a "gap" can lead to worse locks than would seem optimal. In this case, the "gap" is probably all of the range between "A" and "D". Live with it. Write your code to restart the transaction. The second try will probably go through without a deadlock.
    – Rick James
    Commented Jul 11, 2020 at 21:24

2 Answers 2

1
+50

From "WAITING FOR THIS LOCK TO BE GRANTED" part of transaction 3631, we could see:

RECORD LOCKS space id 69 page no 4 n bits 72 index p_name of table `jacky`.`t` trx id 3631 lock_mode X waiting
Record lock, heap no 4 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  1. 3631 is waiting a record lock. The corresponding index content is {"name":"C", "id": 24}.
  2. The index name is p_name in table t.
  3. lock's mode is "lock_mode X"

From "WAITING FOR THIS LOCK TO BE GRANTED" part of transaction 3630, we could see:

*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 69 page no 4 n bits 72 index p_name of table `jacky`.`t` trx id 3630 lock_mode X
Record lock, heap no 4 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
 0: len 1; hex 43; asc C;;
 1: len 8; hex 8000000000000018; asc         ;;

*** (2) WAITING FOR THIS LOCK TO BE GRANTED:

RECORD LOCKS space id 69 page no 4 n bits 72 index p_name of table `jacky`.`t` trx id 3630 lock_mode X locks gap before rec insert intention waiting
Record lock, heap no 4 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
 0: len 1; hex 43; asc C;;
 1: len 8; hex 8000000000000018; asc         ;;
  1. 3630 is waiting a record lock. The corresponding index content is {"name":"C", "id": 24}. waiting lock's mode is "lock_mode X locks gap"
  2. 3630 is holding a record lock. The corresponding index content is {"name":"C", "id": 24}. Holding lock's mode is "lock_mode X locks"
  3. The index name is p_name in table t.
  4. This deadlock is caused by executing "insert into t (name) values ('B')"

According to your reproduce step, session A will send a delete from t where name = 'C'; first, this will lock:

  1. ('A', 'C'] and ('C', 'D'): next-key lock 'C' and gap lock before 'D';

DELETE FROM ... WHERE ... sets an exclusive next-key lock on every record the search encounters. However, only an index record lock is required for statements that lock rows using a unique index to search for a unique row.

  1. add a record lock for 'C' corresponding primary index id. here id value should be "26".

Then session B will start and delete from t where name = 'C'; will be executed again. However. For session B, because session A hadn't been committed, 'C' had been locked by session A. However, if execute a delete sql, session B will try to add lock in the following sequence:

  1. gap lock before 'C': Success, because innodb could add multi gap lock in the same position.
  2. record lock 'C': Blocked, because session A have holded that lock. session B have to wait it released by session A.
  3. gap lock before 'D':

At last, session A send insert into t (name) values ('B');. For table t, there are 2 indexes, which are id and name. id is a auto increasement primary integer key, and for name, this sql will try to add a insert intention lock. However, there have been a gap lock which is holded by session B, therefor session A have to wait session B for releasing that gap lock. Now we could see how this dead lock occur. Innodb will choose a session to rollback base on cost. Here session B will be rolled back.

For Q1, the anwser is yes. For Q2, actually, the deleted record wouldn't be purged from an index before its session commit. For Q3, row lock number is equal to trx_rows_locked, and in mysql website, its:

TRX_ROWS_LOCKED

The approximate number or rows locked by this transaction. The value might include delete-marked rows that are physically present but not visible to the transaction.

From this article, we could know:

  1. For non-clustered unique index filtering, due to the need to return tables, the number of filtered rows is locked as the unique index plus the number of returned rows.

  2. For non-clustered non-unique index filtering, the gap lock is involved, so more records are locked.

So, trx_rows_locked (gap lock + next-key lock + return table) is 3 after delete in session A. final trx_rows_locked value should be 3 + 1 (insert key lock) after trying to insert.


The following are for the new update questions: I didn't notice delete primary key and unique secondary key before.

After some investigating, I found:

  1. When deleting a primary key, which have been deleted and not commited yet, the new delete operation will only require record lock instead of next-key lock.
  2. When deleting a secondary unique key, which have been deleted and not commited yet, the new delete operation will require next-key lock.

You could use set GLOBAL innodb_status_output_locks=ON; show engine innodb status to see the detail lock status for running trasactions.

17
  • For session B, because of MVCC in mysql, 'C' could be seen at that moment. ==> I think 'C' could be seen is not because of MVCC( Session B could see it because session A has not commit the delete yet.). In my opinion, MVCC works for consistent read, and for UPDATE/DELETE statements, InnoDB does a “semi-consistent” read, such that it returns the latest committed version to MySQL.
    – Jacky Wang
    Commented Jul 16, 2020 at 1:47
  • @Jacky1205 yes, you are right. UPDATE/DELETE statement will read newest commited version of data. I will update my answer.
    – Zhiyong
    Commented Jul 16, 2020 at 3:42
  • When I change index p_name to unique index, I still got deadlock with the same steps. Why? The mysql doc has said > only an index record lock is required for statements that lock rows using a unique index to search for a unique row
    – Jacky Wang
    Commented Jul 20, 2020 at 2:20
  • For what in mysql doc, "only an index record lock is required for statements that lock rows using a unique index to search for a unique row", it's correct. However, in your situation, the delete operation in session B is not just "search for a unique row". It's "searching for a unique row which is marked deleted.". In this situation, mysql will add gap lock. If you change delete operation in Session A to update operation, you will found that delete operation in session B will still be blocked, however there will not be a deadlock after insert in Session A.
    – Zhiyong
    Commented Jul 20, 2020 at 4:05
  • 1
    I have found a statement If such a record is delete-marked, it may then become purged, and lock_rec_inheirt_to_gap will be called to decide the fate of each lock on it: either it will be inherited as gap lock, or discarded. in Line 2382, lock0lock.cc source file. But, I am not clear the precise rule of when will be inherited as gap lock (e.g. use secondary unique key?), or discarded (use primary key)?
    – Jacky Wang
    Commented Jul 23, 2020 at 1:47
1

For Q4, I finally found a comment in MySQL 5.7 source code that explain why using next-key locks, just for reference.

In a search where at most one record in the index may match, we can use a LOCK_REC_NOT_GAP type record lock when locking a non-delete-marked matching record.

Note that in a unique secondary index there may be different delete-marked versions of a record where only the primary key values differ: thus in a secondary index we must use next-key locks when locking delete-marked records

Note above that a UNIQUE secondary index can contain many rows with the same key value if one of the columns is the SQL null. A clustered index under MySQL can never contain null columns because we demand that all the columns in primary key are non-null.

1
  • Great job! Here explain everything! a UNIQUE secondary index can contain many rows with the same key value if one of the columns is the SQL null
    – Zhiyong
    Commented Aug 6, 2020 at 13:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.