I'm frequently having deadlock when 2 transactions are doing:
- entitymanager.find by id, no lock mode in particular
- entitymanager.merge, no lock mode in particular
They are all under @Transactional and the default isolation is repeatable read, under mysql 5.7. The entity as id PK autoincrement as commonly used. No @Version if that matters...
What happens is this:
- txn A finds the row
- txn B finds the row
- txn A tries to update and thus escalate to an exclusive X lock but waits because there seems to be a shared (S) (read) lock on the row from txn B
- txn B tries to update and thus escalate to an exclusive X lock but it is after txn A, which is held back by B itself. Now this is detected as a deadlock so one of these txn will rollback.
The SHOW ENGINE INNODB STATUS (SEIS) reveals the last detected deadlock. It clearly says there were shared (S) locks.
This is my SEIS from prod (with some relabelling for privacy).
*** (1) TRANSACTION:
TRANSACTION 175274419, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 8 lock struct(s), heap size 1136, 3 row lock(s), undo log entries 2
MySQL thread id 627088, OS thread handle 22952098592512, query id 365172765 192.168.1.100 mydatabase updating
update thing set needs_checkup=0 where id=1129
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 361 page no 25 n bits 144 index PRIMARY of table `mydatabase`.`thing` trx id 175274419 lock_mode X locks rec but not gap waiting
*** (2) TRANSACTION:
TRANSACTION 175274420, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
8 lock struct(s), heap size 1136, 3 row lock(s), undo log entries 2
MySQL thread id 627077, OS thread handle 22952240928512, query id 365172766 192.168.1.100 mydatabase updating
update thing set needs_checkup=0 where id=1129
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 361 page no 25 n bits 144 index PRIMARY of table `mydatabase`.`thing` trx id 175274420 lock mode S locks rec but not gap
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 361 page no 25 n bits 144 index PRIMARY of table `mydatabase`.`thing` trx id 175274420 lock_mode X locks rec but not gap waiting
*** WE ROLL BACK TRANSACTION (2)
What is surprising: I enabled the hibernate DEBUG level on org.hibernate.SQL to see the statements, and NONE of them show any "select ... lock in share mode" (nor select ... for update).
(I've gone the extra mile and packet-sniffed the mysql protocol over port 3306 with wireshark, and not a single hint of special locking mode, nor any session variable other than the usual "set session transaction read write" vs "... read only" from time to time, which has no effect on locking).
There is enough time between steps 1 and 3 obviously for txn B to sneak in. So I presume this shared lock is not a momentary effect of the update statement. We wouldn't get deadlocks that easily if that was the case. So I presume the shared lock comes from the "find".
The question is where is this configured? For all the docs I read, the default lock mode is LockMode.NONE.
If I write raw sql in 2 sessions, like below, (and using transaction read write mode, the default) I don't get the deadlock:
- txnA: select * from foo where id = 1;
- txnB: select * from foo where id = 1;
- txnA: update foo set x=x+1 where id = 1;
- txnB: update foo set x=x+1000 where id = 1;
but if I write this, then I get the same deadlock:
- txnA: select * from foo where id = 1 lock in share mode ;
- txnB: select * from foo where id = 1 lock in share mode ;
- txnA: update foo set x=x+1 where id = 1;
- txnB: update foo set x=x+1000 where id = 1;
Now, I don't want to use a X (or U) lock in the find, as mentioned in How do UPDATE locks prevent a common form of deadlock?.
I want to just lock less, as the raw SQL seems to allow. So again, the question is where is this configured? why is this shared lock requested? How does hibernate even get to that point if none of the sql statement I see in the sniffed packets even hint at those shared locks?
Thanks.
UPDATE:
In the last days, I've examined the possibility for an unseen statement in the same transactions prior to the updates above.
a) I do have a foreign key of some child table row inserted and pointing to the 'thing' row. The mysql doc does say something about shared locks on parent of FK. I tried and it didn't lock the parent for share. I still couldn't make the deadlock with those child inserts with "raw" instructions again. The child insert doesn't prevent a parent update (the parent "thing" table is the one having deadlocked statement, remember).
b) I also read on unique keys. They mention something about the statement (failing the unique constraint) taking a shared lock. I'm not clear on the steps to achieve that. While I'm still investigating that, I though I should mention it in case that lights up someone's mind...
entityManager.find(A.class, aid, LockModeType.OPTIMISTIC);