50

I try to simulate the behaviour of a class, using Mockito. This worked using Mockito 1.x. Migrating to JUnit 5 and Mockito 2 it seems not to work anymore.

@ExtendWith(MockitoExtension.class)
public class MockitoExample {

  static abstract class TestClass {
    public abstract int booleanMethod(boolean arg);
  }

  @Mock
  TestClass testClass;

  @BeforeEach
  public void beforeEach() {
    when(testClass.booleanMethod(eq(true))).thenReturn(1);
    when(testClass.booleanMethod(eq(false))).thenReturn(2);
  }

  @Test
  public void test() {
    assertEquals(1,testClass.booleanMethod(true));
    assertEquals(2,testClass.booleanMethod(false));
  }
}

The expectation is, that the mocked TestClass shows the behaviour as tested in the test-method.

The error I get is:

org.mockito.exceptions.misusing.PotentialStubbingProblem: 

  Strict stubbing argument mismatch. Please check:
   - this invocation of 'booleanMethod' method:
      testClass.booleanMethod(false);
      -> at org.oneandone.ejbcdiunit.mockito_example.MockitoExample.beforeEach(MockitoExample.java:30)
   - has following stubbing(s) with different arguments:
      1. testClass.booleanMethod(false);
        -> at org.oneandone.ejbcdiunit.mockito_example.MockitoExample.beforeEach(MockitoExample.java:29)
  Typically, stubbing argument mismatch indicates user mistake when writing tests.
  Mockito fails early so that you can debug potential problem easily.
  However, there are legit scenarios when this exception generates false negative signal:
    - stubbing the same method multiple times using 'given().will()' or 'when().then()' API
      Please use 'will().given()' or 'doReturn().when()' API for stubbing.
    - stubbed method is intentionally invoked with different arguments by code under test
      Please use default or 'silent' JUnit Rule (equivalent of Strictness.LENIENT).
  For more information see javadoc for PotentialStubbingProblem class.

In both cases, the argument false seems to be matched, though I clearly matched with true.

Is that a bug in Mockito 2.17 or a misunderstanding. How should/can I use Mockito 2.x to simulate calls with different boolean arguments?

The example can also be found on github. But surefire will start the test only using

mvn test -Dtest=MockitoExample

Executing the test using Mockito 2.21 leads to the same results.

0

4 Answers 4

54

Since Mockito 2.20 it is also possible, to add lenient() locally

@ExtendWith(MockitoExtension.class)
public class MockitoExample {

  static abstract class TestClass {
    public abstract int booleanMethod(boolean arg);
  }

  @Mock
  TestClass testClass;

  @BeforeEach
  public void beforeEach() {
    lenient().when(testClass.booleanMethod(eq(true))).thenReturn(1);
    lenient().when(testClass.booleanMethod(eq(false))).thenReturn(2);
  }

  @Test
  public void test() {
    assertEquals(1,testClass.booleanMethod(true));
    assertEquals(2,testClass.booleanMethod(false));
  }
}
2
  • 1
    Your answer appears to give the better advice. I wrote an answer to complete your approach (+1). I lost more 1 hour today to guess the issue with a very complex code...
    – davidxxx
    Nov 9, 2018 at 23:02
  • 3
    I also agree this should be accepted one, maybe with some explanation why its required Feb 13, 2020 at 12:20
46

With strict stubs (the default behavior of Mockito) calling several whens on the same method will reset that mock. The solution is to call when once and have the logic in an Answer:

@BeforeEach
public void beforeEach() {
    when(testClass.booleanMethod(anyBoolean())).thenAnswer(invocationOnMock -> {
        if ((boolean) invocationOnMock.getArguments()[0]) {
            return 1;
        }
        return 2;
    });
}

Alternatively, you can use lenient mocking, but that's not always a good idea - lenient mocking allows redundant stubbing, and makes it easier for you to make mistakes in your test, which may lead to unnoticed bugs in the "production" code:

@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
public class MockitoExample {
5
  • 13
    Really? I have to use Answers, interpret the arguments there for which originally a very fine DSL was implemented and make the stubs the more prone for mistakes? In that case it seems to be better to drop mockito usage and inject real simulations (at least, if you are using spring or weld)
    – aschoerk
    Sep 2, 2018 at 20:03
  • 1
    sorry, thank you very much for the answer. That seems to be a way to go forward.
    – aschoerk
    Sep 2, 2018 at 20:03
  • org.mockito.quality.Strictness.LENIENT
    – amdev
    Oct 14, 2019 at 6:03
  • 2
    If you can live with the doReturn().when() syntax, porting to that instead is a decent approach.
    – amoe
    Mar 3, 2020 at 10:50
  • Mock isn’t resetting on multiple calls to when with different arguments
    – Ivan
    Aug 22, 2022 at 10:54
27

Mockito 1 and 2 don't have the same "strictness" level.
Besides by using Mockito 2 with JUnit 4 or 5 the default level will be still different.

To sum up :

3 levels of strictness :

  • LENIENT : minimum strictness
  • WARN : extra warnings emitted to the console
  • STRICT_STUBS : ensures clean tests by throwing exception if potential misuse but may also produce some false positives.

Default effective level according to the APIs used :

  • Mockito 1 : LENIENT
  • Mockito 2 with JUnit 4 : WARN
  • Mockito 2 with JUnit 5 (MockitoExtension.class) : STRICT_STUBS
  • Mockito 3 : planned to be STRICT_STUBS.

More details

The actual Mockito documentation is very clear about that :

The Strictness javadoc states :

Configures the "strictness" of Mockito during a mocking session.A session typically maps to a single test method invocation. Strictness drives cleaner tests and better productivity.The easiest way to leverage enhanced Strictness is usingMockito's JUnit support (MockitoRule or MockitoJUnitRunner).If you cannot use JUnit support MockitoSession is the way to go.

How strictness level influences the behavior of the test (mocking session)?

1.Strictness.LENIENT - no added behavior.The default of Mockito 1.x.Recommended only if you cannot use STRICT_STUBS nor WARN.

2.Strictness.WARN - helps keeping tests clean and improves debuggability.Reports console warnings about unused stubsand stubbing argument mismatch (see org.mockito.quality.MockitoHint).The default behavior of Mockito 2.x when JUnitRule or MockitoJUnitRunner are used. Recommended if you cannot use STRICT_STUBS.

3.Strictness.STRICT_STUBS - ensures clean tests, reduces test code duplication, improves debuggability.Best combination of flexibility and productivity. Highly recommended.Planned as default for Mockito v3.See STRICT_STUBS for the details.

But whatever the thrown exception associated to the message

"has following stubbing(s) with different arguments"

seems to be a excessively strict check. The exception message proves that in a some way :

However, there are legit scenarios when this exception generates false negative signal:

...

  • stubbed method is intentionally invoked with different arguments by code under test

So forbidding it by default seems to be too much.
So if you use JUnit 5, as alternative to STRICT_STUBS you could use WARNING but you generally want to avoid LENIENT that is too quiet.

In addition to MockitoExtension, the mockito-junit-jupiter library provides @MockitoSettings that may be used at the method level as well as at the class level.

Here is an example :

import java.util.List;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;

@ExtendWith(MockitoExtension.class)
public class FooTest {

    @MockitoSettings(strictness = Strictness.WARN)
    @Test
    void foo() throws Exception {
        List<String> strings = Mockito.mock(List.class);
        Mockito.when(strings.add("a"))
               .thenReturn(true);
        Mockito.when(strings.add("b"))
               .thenReturn(false);
    }

    @Test
    void fooKo() throws Exception {
        List<String> strings = Mockito.mock(List.class);
        Mockito.when(strings.add("a"))
               .thenReturn(true);
        Mockito.when(strings.add("b"))
               .thenReturn(false);

    }

}

fooKo() throws the misuse Mockito exception while foo() is successful but provides helpful warnings :

[MockitoHint] FooTest (see javadoc for MockitoHint):
[MockitoHint] 1. Unused -> at FooTest.foo(FooTest.java:19)
[MockitoHint] 2. Unused -> at FooTest.foo(FooTest.java:21)

As other alternative you can also use Mockito.lenient() very well described by aschoerk to apply the lenient strictness for a specific invocation. As well as you can set every mock invocations as lenient at the mock instantiation :

@Test
void foo() throws Exception {
    List<String> strings = Mockito.mock(List.class, Mockito.withSettings()
                                                           .lenient());
     ....
}
-4

Since the first answer came as a surprise, I checked the following:

interface Poops {
    String get(boolean is);
}

@Test
void test1() {
    Poops a = mock(Poops.class);

    when(a.get(eq(true))).thenReturn("1");
    when(a.get(eq(false))).thenReturn("2");

    Assertions.assertEquals("1", a.get(true));
    Assertions.assertEquals("2", a.get(false));
}

It works with Mockito 2.21.0.

Update: The problem seems to be the Jupiter Mockito extension which changes the default setting to Strictness.STRICT_STUBS.

13
  • If you have a new question, please ask it by clicking the Ask Question button. Include a link to this question if it helps provide context. - From Review
    – Draken
    Sep 3, 2018 at 10:46
  • @Draken It would have been better as a comment but comments don't allow code snippets. And no, I don't have an additional question since everything works as I expect it to. Sep 3, 2018 at 11:36
  • @aschoerk I cloned your repo, CDed to mockito-example, removed the pom dependency to ejb-cdi-unit since I don't have it and voila: mvn test -> BUILD SUCCESS. But it fails in IntelliJ. Hm. Sep 3, 2018 at 11:38
  • @johanneslink I think you've misunderstood how StackOverflow works, it's not a forum. This area is only for answering the OP's question, and not asking further questions. You've asked a question as you've used the word, why? If it's meant to be an answer to the question, please update it so it takes on that form. Answers shouldn't generally end with a question in the final line. If you're trying to get the OP to debug the question and find out through self discovery, then you need to be clearer about it.
    – Draken
    Sep 3, 2018 at 11:39
  • @johanneslink Perhaps you did not use "mvn clean install -Dtest=MockitoExample" or you used an older version using Lenient.
    – aschoerk
    Sep 3, 2018 at 11:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.