Hey, I am testing a service which heavily relies on project reactor.

For many tests I am mocking the return value of the component responsible for API calls. The tests are split over multiple files. When I run the tests of one file, they are green, but when I execute all of the test files at once, some tests fail, with the error message indicating that the mocking did not succeed (Either the injected component returned null, or the implementation of the actual component is invoked). In the logs, there is no information about the mocking failing.

A code example:

interface API {
  Flux<Bird> getBirds();
}

@Component
class BirdWatcher {
  API api;
  BirdWatcher(API api) {
    this.api = api;
  }

  Flux<Bird> getUncommonBirds() {
    return api.getBirds()           // Although this is mocked in the test, in some runs it returns `null` or calls the implementation of the actual component
      .filter(Bird::isUncommon);
  }
}

@SpringBootTest
class BirdWatcherTests {
  @Autowired
  BirdWatcher birdWatcher;
  @MockBean
  API api;

  @Test
  void findsUncommonBirds() {
    // Assemble
    Bird birdCommon = new Bird("Sparrow", "common");
    Bird birdUncommon = new Bird("Parrot", "uncommon");
    Mockito.when(api.getBirds()).thenReturn(Flux.just(birdCommon, birdUncommon));

    // Act
    Flux<Bird> uncommonBirds = birdWatcher.getUncommonBirds();

    // Assert
    assertThat(uncommonBirds.collectList().block().size(), equalTo(1));
  }
}

For me the issue seems like a race condition, but I don't know where and how this might happen, and how I can check and fix this.

I am using spring-boot-test:2.7.8, pulling in org.mockito:mockito-core:4.5.1 org.mockito:mockito-junit-jupiter:4.5.1, and org.junit.jupiter:junit-jupiter:5.8.2, with gradle 7.8. For reactor, spring-boot-starter-webflux:2.7.8, depending on reactor:2.7.8.

Comment From: wilkinsona

I think a race condition could only occur when running multiple tests in parallel which isn't supported in this case. If you are not using parallel test execution then I do not understand the cause of the race condition and I'd need to see a minimal example of the problem to be able to investigate further.

Comment From: spring-projects-issues

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Comment From: spring-projects-issues

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

Comment From: AntonOellerer

Hey, sorry for not following up! I am not able to provide a MRE, but for anyone encountering this issue, I fixed this by annotating each class with @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS) I don't really know what the problem was though

Comment From: innokenty

I have encoutered what I believe must be the same issue (running under Spring Boot 3.5.3, java 24)

I have a pretty standard setup, no parallel running, no fancy-shmancy, just a regular Spring Boot project. It's not even a real production project in fact, it's an example project. This one: https://github.com/feature-tracker/feature-service

One of my test classes declares a @MockitoSpyBean for a given service in the context and does a simple verification of "a method was called". When I run this test class separately – everything works perfect.

However, when I run multiple tests (seems to be irrelevant which ones, just as long as the test with the mock isn't the first one in the suite) – the mock / spy simply doesn't get injected in the target bean inside the context, where it should be injected to! There are no multiple beans of the same type, no qualifiers needed, nothing like that – very simple setup.

I debugged it now for a couple of days: made sure the context is recreated during my test suite, tried various types of mock creation with @TestConfiguration, added @DirtiesContext(BEFORE_CLASS) to the failing test with the mock, as well as to all the other tests in the project – no effect. Tests are run, mock is crated but not injected, I see in the logs the Spring context is recreated correctly and all the business logic is invoked, but I get "zero interactions with the mock" error. Once again: this only happens when multiple test classes are run, all this time the problem test works perfectly in isolation!

I also have no idea why, but (like @AntonOellerer suggests) adding DirtiesContext.ClassMode.AFTER_CLASS) to all the other tests indeed fixes the issue!! BEFORE_CLASS on all tests (including the failing one) doesn't, but adding AFTER_CLASS just to the single test that is run before the problem test with the mock – does!

Comment From: wilkinsona

Thanks for the analysis, @innokenty. @MockitoSpyBean is a Spring Framework feature so you may want to open an issue in their repository.