2

My project uses spring-data-mongodb, everything is reactive. There is a bean with a transactional method using declarative transactions. The relevant code fragments follow:

@Configuration
public class Config {

    @Bean
    public ReactiveMongoTransactionManager reactiveMongoTransactionManager() {
        return new ReactiveMongoTransactionManager(reactiveMongoDbFactory());
    }

    ...
}

@Service
public class MyService {
    private final ReactiveMongoOperations mongoOperations;

    ...

    @Transactional
    public Mono<User> saveUser(User user) {
        return mongoOperations.insert(user).then(anotherInsertOnMongoOperations()).thenReturn(user);
    }
}

Nothing unusual here.

I can see in logs that the transactions get started before documents get inserted, and after that they are committed:

DEBUG o.s.d.m.ReactiveMongoTransactionManager -  About to start transaction for session [ClientSessionImpl@62de8058 id = {"id": {"$binary": "fye2h5JkRh6yL3MTqtC0Xw==", "$type": "04"}}, causallyConsistent = true, txActive = false, txNumber = 1, error = d != java.lang.Boolean].
DEBUG o.s.d.m.ReactiveMongoTransactionManager -  Started transaction for session [ClientSessionImpl@62de8058 id = {"id": {"$binary": "fye2h5JkRh6yL3MTqtC0Xw==", "$type": "04"}}, causallyConsistent = true, txActive = true, txNumber = 2, error = d != java.lang.Boolean].

... inserts follow, then ...

DEBUG o.s.d.m.ReactiveMongoTransactionManager -  Initiating transaction commit
DEBUG o.s.d.m.ReactiveMongoTransactionManager -  About to commit transaction for session [ClientSessionImpl@62de8058 id = {"id": {"$binary": "fye2h5JkRh6yL3MTqtC0Xw==", "$type": "04"}}, causallyConsistent = true, txActive = true, txNumber = 2, error = d != java.lang.Boolean].

But sometimes, as I can see by the contents of the database, only the first of the inserts is persisted, and the second is lost. After trying to model the situation, I found that this 'loss' happens when the whole reactive pipeline gets cancelled (not every time, but I was able to produce a test that reproduced the situation with high probability).

I added .doOnSuccessOrError() and .doOnCancel() with some logging after the final operator of my method. In the 'normal' case (without a cancel), doOnSuccessOrError logs successfully. But when a cancellation happens, sometimes the sequence of the events in the log is like this:

  1. A transaction is initiated
  2. An insert happens
  3. A cancel occurs
  4. nothing gets logged by the final doOnSuccessOrError(), and something is logged in the onCancel() there (so the cancellation seems to happen 'right in the middle' of the execution of the business method)
  5. ... but the transaction still gets committed!

TransactionAspectSupport.ReactiveTransactionSupport contains the following code (used for this case):

                            return Mono.<Object, ReactiveTransactionInfo>usingWhen(
                                    Mono.just(it),
                                    txInfo -> {
                                        try {
                                            return (Mono<?>) invocation.proceedWithInvocation();
                                        }
                                        catch (Throwable ex) {
                                            return Mono.error(ex);
                                        }
                                    },
                                    this::commitTransactionAfterReturning,
                                    (txInfo, err) -> Mono.empty(),
                                    this::commitTransactionAfterReturning)

The last argument is onCancel handler.

This means that on cancel, the transaction actually gets committed.

The question is: why? When a cancellation happens due to reasons external to a reactive pipeline, there is a probability that some operations inside a transaction have completed and some haven't (and never will). Committing at such a moment produces a partial commit which violates the atomicity requirement.

It seems more logical to initiate a rollback instead. But I suppose that the authors of spring-tx did this choice on purpose. I wonder, what is the reason for this?

P.S. To verify my point, I patched spring-tx 5.2.3 (by the way, that's the version used by the project) so that the code looks like this:

                            return Mono.<Object, ReactiveTransactionInfo>usingWhen(
                                    Mono.just(it),
                                    txInfo -> {
                                        try {
                                            return (Mono<?>) invocation.proceedWithInvocation();
                                        }
                                        catch (Throwable ex) {
                                            return Mono.error(ex);
                                        }
                                    },
                                    this::commitTransactionAfterReturning,
                                    (txInfo, err) -> Mono.empty(),
                                    this::rollbackTransactionDueToCancel)

    private Mono<Void> rollbackTransactionDueToCancel(@Nullable ReactiveTransactionInfo txInfo) {
        if (txInfo != null && txInfo.getReactiveTransaction() != null) {
            if (logger.isDebugEnabled()) {
                logger.debug("Rolling transaction back for [" + txInfo.getJoinpointIdentification() + "] due to cancel");
            }
            return txInfo.getTransactionManager().rollback(txInfo.getReactiveTransaction());
        }
        return Mono.empty();
    }

(basically, just changed the on-cancel behavior to rollback), and with this patch my tests do not produce any inconsistent data anymore.

5
  • Which class calls the saveUser method? May 15, 2020 at 15:39
  • @ModusTollens I cannot show the caller code here, it's a closed-source and contains too much irrelevant information. But, in the end, the business method (it is even named differently, here all the names are changed to protect the innocent) is called from a code that does a concatMap() on a KafkaReceiver.receive() from reactor-kafka. The cancellations occur due to retries (retryBackoff() is used). But all this seemed to be irrelevant for the question. May 15, 2020 at 16:03
  • Just wanted to make sure it wasn't called by the same class that defines it (MyService). That would lead to problems. May 15, 2020 at 16:05
  • Can you show the stacktrace? May 16, 2020 at 18:13
  • @ModusTollens the problem is about a 'commit on cancel'. The cancellation may be caused by different reasons. I'm not even sure that it is actually caused by retryBackoff() behavior, so I don't have a stack trace (and it is not relevant here). May 16, 2020 at 18:17

1 Answer 1

2

It turns out that indeed there is a possibility for a reactive Spring transaction to be committed half the way due to an unexpected cancel: https://github.com/spring-projects/spring-framework/issues/25091

The problem is due to the 'commit-on-cancel' policy. Spring guys are planning to switch it to 'rollback-on-cancel' policy in Spring 5.3. For now, the options are:

  1. If some your transactions contain more than one write, use a custom build of spring-tx library with a fix like the following https://github.com/rpuch/spring-framework/commit/95c2872c0c3a8bebec06b413001148b28bc78f2a that switches to 'rollback-on-cancel' policy to avoid such unleasant surprises. But this would mean that perfectly valid Reactor operators (that use cancellation signal as a part of their normal functioning) would become unusable downstream from transactional operators (as a cancellation routinely issued by them will roll back a transaction).
  2. If all your transactions have maximum one write each, then you are safe to stay with an unpatched Spring version. But please note that Spring guys are (currently) going to flip the policy in 5.3.

Here is an article on the matter: https://blog.rpuch.com/2020/05/25/spring-reactive-transactions-atomicity-violation.html (disclaimer: I'm the author of the article).

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.