Maven Failsafe Plugin
Requirements: Maven 3.2.5 and JDK 1.8 or higher.
This is the road map of the development.
Versions | Release Targets |
---|---|
3.0.0-M1 | Maven API 3.0 and Java 1.7 Maven Plugin API 3.0 Java 1.7 as minimum @Component is deprecated. @Parameter should be used instead Surefire manifest jar classloading broken on latest Debian/Ubuntu Java8 See the Release Notes for the version 3.0.0-M1 |
3.0.0-M2 | Fixed: JDK9 and Windows Class-Path issues, 3.0 and Legacy Report XSD, 3.0.0-M2 shadefire Fixed JDK9+ (Jigsaw) modular paths (module-info.java) having white spaces on file system Windows slashes appear in relative paths of Class-Path in MANIFEST.MF (Boot Manifest-JAR) Surefire fails loading class ForkedBooter when using a sub-directory pom file Plugin fails if used Toolchains together with JDK9+ and (Jigsaw) modular paths (module-info.java) 3.0 and Legacy Report XSD 3.0.0-M2 shadefire Feature: Option to switch-off Java 9 modules Option to switch-off Java 9 modules See the Release Notes for the version 3.0.0-M2 |
3.0.0-M3 | Fixed: maven-surefire-report-plugin supports JDK11, JUnit5 issues, deprecated skipTests in Failsafe plugin, used ShadeFire 3.0.0-M2 maven-surefire-report-plugin fails on JDK 11 JUnit Runner that writes to System.out corrupts Surefire's STDOUT when using JUnit's Vintage Engine Smart stacktrace in test summary should not print JUnit5 assertion exception type Deprecate skipTests in Failsafe Plugin See the Release Notes for the version 3.0.0-M3 |
3.0.0-M4 | Provided 3 extensions of reporters which can be used to customize XML report, console and file reporters. It is very useful for JUnit5 users. The internal implementation was reworked so that new commands and events can be easily added. The impl is located in a center point and it is a prerequisite in next versions. Provided bug fixes for Docker Alpine/BusyBox Linux, JUnit5 and 43 more. ForkClient attempts to consume unrelated lines |
3.0.0-M5 | Test Report tasks (prerequisite: SUREFIRE-1222 in 3.0.0-M4) New interprocess communication with TCP/IP which fixed current blocker and critical bugs. Provided extensions which can be used to customize this interprocess communication in plugin configuration. It is internally used to switch over the pipes and TCP connector. TCP/IP Channel for forked Surefire JVM. Extensions API and SPI. Polymorphism for remote and local process communication. |
3.0.0-M6 | ReportEntry contains new fields testRunId:long and RunMode which help identifying the tests and logs. The plugin supports Java 1.8 and Maven Plugin API 3.2.5. Many bug fixes. ASF Release versions |
3.0.0-M7 | SurefireProvider interface, TestSetRunListener, StatelessXmlReporter, and related code will be reworked. Currently, Java classes are executed as tests and we will support more. ASF Release versions |
3.0.0 | Breaking backwards compatibility with system properties in configuration parameters, removing deprecated configuration parameters and removing deprecated code, etc. ASF Release versions |
The Failsafe Plugin is designed to run integration tests while the Surefire Plugin is designed to run unit tests. The name (failsafe) was chosen both because it is a synonym of surefire and because it implies that when it fails, it does so in a safe way.
The Maven lifecycle has four phases for running integration tests:
pre-integration-test
for setting up the integration test environment.integration-test
for running the integration tests.post-integration-test
for tearing down the integration test environment.verify
for checking the results of the integration tests.
If you use the Surefire Plugin for running tests, then when you have a test failure, the build will stop at the integration-test
phase and your integration test environment will not have been torn down correctly.
The Failsafe Plugin is used during the integration-test
and verify
phases of the build lifecycle to execute the integration tests of an application. The Failsafe Plugin will not fail the build during the integration-test
phase, thus enabling the post-integration-test
phase to execute.
NOTE: when running integration tests, you should invoke Maven with the (shorter to type too)
mvn verify
rather than trying to invoke the integration-test
phase directly, as otherwise the post-integration-test
phase will not be executed.
The Failsafe Plugin generates reports in two different file formats:
- Plain text files (
*.txt
) - XML files (
*.xml
)
By default, these files are generated in ${basedir}/target/failsafe-reports/TEST-*.xml
.
The schema for the Failsafe XML reports is available at Failsafe XML Report Schema.
The XML reports generated by legacy plugins (versions up to 2.22.0) should be validated by Legacy Failsafe XML Report Schema.
Two plugin versions (2.22.1 and 3.0.0-M1) however generate 3.0 XML reports they still refer to legacy schema (see noNamespaceSchemaLocation
in XML Report). The projects which expect XML Reports validated by XSD Schema, tools e.g xUnit, should not use versions 2.22.1 and 3.0.0-M1 of the Failsafe plugin.
For an HTML format of the report, please see the Maven Surefire Report Plugin.
By default this plugin generates summary XML file at ${basedir}/target/failsafe-reports/failsafe-summary.xml
and the schema is available at Failsafe XML Summary Schema.
Goals Overview
The Failsafe Plugin has only two goals:
- failsafe:integration-test runs the integration tests of an application.
- failsafe:verify verifies that the integration tests of an application passed.
Usage
General instructions on how to use the Failsafe Plugin can be found on the usage page. Some more specific use cases are described in the examples listed below. Additionally, users can contribute to the GitHub project.
In case you still have questions regarding the plugin's usage, please have a look at the FAQ and feel free to contact the user mailing list. The posts to the mailing list are archived and could already contain the answer to your question as part of an older thread. Hence, it is also worth browsing/searching the mail archive.
If you feel like the plugin is missing a feature or has a defect, you can file a feature request or bug report in our issue tracker. When creating a new issue, please provide a comprehensive description of your concern. Especially for fixing bugs it is crucial that the developers can reproduce your problem. For this reason, entire debug logs, POMs or most preferably little demo projects attached to the issue are very much appreciated. Of course, patches are welcome, too. Contributors can check out the project from our source repository and will find supplementary information in the guide to helping with Maven.
Examples
The following examples show how to use the Failsafe Plugin in more advanced use cases:
- Using TestNG
- Using JUnit 5 Platform
- Using JUnit
- Using POJO Tests
- Skipping Tests
- Skip After Failure
- Inclusions and Exclusions of Tests
- Running a Single Test
- Re-run Failing Tests
- Class Loading and Forking
- Debugging Tests
- Using System Properties
- Configuring the Classpath
- Selecting Providers
- Fork Options and Parallel Test Execution
- Using Console Logs
- Shutdown of Forked JVM
- Run tests with Java 9
- Run tests in Docker