What is the difference between Maven Surefire and Maven Failsafe plugins?
I have searched all over web, but did not get the answer.
3 Answers
In simple words, the Failsafe plugin is designed to run integration tests while Surefire to run unit tests.
This is further explained in Maven FAQ:
maven-surefire-plugin
is designed for running unit tests and if any of the tests fail then it will fail the build immediately.
maven-failsafe-plugin
is designed for running integration tests, and decouples failing the build if there are test failures from actually running the tests.The name "failsafe" was chosen both because it is a synonym of surefire and because it implies that when it fails, it does so in a safe way.
The Failsafe Plugin has two goals:
failsafe:integration-test
runs the integration tests of an application,failsafe:verify
verifies that the integration tests of an application passed.
See also:
-
15The interesting question is why. Many build tools work perfectly without making this arguably rather artificial and limited distinction. Configuring both plugins together and absorbing their reports in consistent ways is quite the PITA.– RaphaelCommented Mar 31, 2020 at 15:32
-
1And no, the differences stated in the other answers do not provide sufficient reason for having multiple tools for what should be filters and flags.– RaphaelCommented Mar 31, 2020 at 15:33
-
5I think this answer fails to make a specific case for Failsafe. It also uses two more lifecycle phases within maven,
pre-integration-test
andpost-integration-test
. This is generally used to bring up an environment, or do some sort of external setup. If a test fails when running in failsafe, it is guaranteed thatpost-integration-test
will run and tear things down. This is because, as stated in the answer, it does not fail until the verify phase. That is the real benefit of the plugin, it will ALWAYS runpost-integration-test
Commented Nov 2, 2020 at 18:46 -
Technically, the plugin itself does not execute these phases, but it does adhere to the convention. Surefire will fail immediately when a test failure is detected and no subsequent maven lifecycle phases are executed. Commented Nov 2, 2020 at 18:51
From https://maven.apache.org/surefire/maven-failsafe-plugin/, I would say that the difference between Surefire and Failsafe is the way they fail:
If you use the Surefire Plugin for running tests, then when you have a test failure, the build will stop at the integration-test phase and your integration test environment will not have been torn down correctly.
The Failsafe Plugin is used during the integration-test and verify phases of the build lifecycle to execute the integration tests of an application. The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute.
-
9Then the question becomes: why ever use Surefire if they are telling us it can leave stuff in an incorrect state? Commented Apr 12, 2020 at 17:22
In my country its the second google result when searching for "maven failsafe maven surefire" to get to this FAQ: Difference between maven-failsafe-plugin and maven-surefire-plugin which states:
maven-surefire-plugin is designed for running unit tests and if any of the tests fail then it will fail the build immediately.
maven-failsafe-plugin is designed for running integration tests, and decouples failing the build if there are test failures from actually running the tests."