Maven Surefire Plugin – Using JUnit 5 Platform

Using JUnit 5 Platform

Configuring JUnit Platform

To get started with JUnit Platform, you need to add at least a single TestEngine implementation to your project. For example, if you want to write tests with Jupiter, add the test artifact junit-jupiter-engine to the dependencies in POM:

<dependencies>
    [...]
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.4.0</version>
        <scope>test</scope>
    </dependency>
    [...]
</dependencies>

This will pull in all required dependencies. Among those dependencies is junit-jupiter-api which contains the classes and interfaces your test source requires to compile. junit-platform-engine is also resolved and added.

This is the only step that is required to get started - you can now create tests in your test source directory (e.g., src/test/java).

If you want to write and execute JUnit 3 or 4 tests via the JUnit Platform add the Vintage Engine to the dependencies, which transitively pulls in (and requires) junit:junit:4.12:

<dependencies>
    [...]
    <dependency>
        <groupId>org.junit.vintage</groupId>
        <artifactId>junit-vintage-engine</artifactId>
        <version>5.4.0</version>
        <scope>test</scope>
    </dependency>
    [...]
</dependencies>

For more information on using JUnit 5, that is the JUnit Platform, JUnit Jupiter, and JUnit Vintage, see the JUnit 5 web site and the JUnit 5 User Guide.

Smart Resolution of Jupiter Engine and Vintage Engine for JUnit4

JUnit5 API artifact and your test sources become isolated from engine. In these chapters you will see how you can segregate, combine, select the APIs and Engines miscellaneous way. You can find integration tests with JUnit4/5, with JUnit5/TestNG and with the JUnit4 Runner for Jupiter tests. (See the Maven profiles.)

How to run only one API

Normally, the developer does not want to access internal classes of JUnit5 engine (e.g. 5.4.0). In the next chapters you can find your way to use the Jupiter or JUnit5 API where the plugin would resolve the engine.

Jupiter API in test dependencies

In this example the POM has only Jupiter API dependency in test classpath. The plugin will resolve and download the junit-jupiter-engine with the version related to the version of junit-jupiter-api. Similar principles can be found in the following chapters as well.

<dependencies>
    [...]
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>5.4.0</version>
        <scope>test</scope>
    </dependency>
    [...]
</dependencies>
...
<build>
    <plugins>
        [...]
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            [... configuration or goals and executions ...]
        </plugin>
        [...]
    </plugins>
</build>
...

API-Engine versions segregation

In the following example the engine artifact appears in plugin dependencies and the engine is resolved by the plugin and downloaded from a remote repository for plugins. You may want to update the version of engine with fixed bugs in 5.3.2 but the API version 5.3.0 stays intact!

<dependencies>
    [...]
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>5.3.0</version>
        <scope>test</scope>
    </dependency>
    [...]
</dependencies>
...
<build>
    <plugins>
        [...]
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <dependencies>
                <dependency>
                    <groupId>org.junit.jupiter</groupId>
                    <artifactId>junit-jupiter-engine</artifactId>
                    <version>5.3.2</version>
                </dependency>
            </dependencies>
        </plugin>
        [...]
    </plugins>
</build>
...

JUnit4 API in test dependencies

This is similar example with JUnit4 in test dependencies of your project POM. The Vintage engine artifact has to be in the plugin dependencies; otherwise the plugin would use surefire-junit4 provider instead of the surefire-junit-platform provider.

<dependencies>
    [...]
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.13</version>
        <scope>test</scope>
    </dependency>
    [...]
</dependencies>
...
<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <dependencies>
                <dependency>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                    <version>5.4.0</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>
...

How to run multiple APIs or Engines

In the following example you can use both JUnit4 and JUnit5 tests.

Jupiter API and JUnit4

Once you define any JUnit5 API in the dependencies, the provider surefire-junit-platform is selected and you can always add the JUnit4 dependency.

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.13</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Jupiter API and Vintage Engine

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.vintage</groupId>
        <artifactId>junit-vintage-engine</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Jupiter and Vintage Engine

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.vintage</groupId>
        <artifactId>junit-vintage-engine</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Select engine and use multiple APIs

In these examples you use both API, i.e. Jupiter and JUnit4, in the test dependencies but you want to select the engine via plugin dependencies.

Select Jupiter

Here your tests import the packages from JUnit4 and Jupiter but you want to select only one Maven profile with JUnit4 tests.

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.13</version>
        <scope>test</scope>
    </dependency>
</dependencies>
<profile>
    <id>select-junit5</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <dependencies>
                    <dependency>
                        <groupId>org.junit.jupiter</groupId>
                        <artifactId>junit-jupiter-engine</artifactId>
                        <version>5.6.2</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>
</profile>

Select JUnit4

Here your tests import the packages from JUnit4 and Jupiter but you want to select only one Maven profile with Jupiter tests.

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.13</version>
        <scope>test</scope>
    </dependency>
</dependencies>
<profile>
    <id>select-junit4</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <dependencies>
                    <dependency>
                        <groupId>org.junit.vintage</groupId>
                        <artifactId>junit-vintage-engine</artifactId>
                        <version>5.6.2</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>
</profile>

How to run TestNG tests within Jupiter engine

You can run TestNG tests combined with JUnit5 tests.

For more information see this example.

<dependencies>
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>7.1.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>com.github.testng-team</groupId>
        <artifactId>testng-junit5</artifactId>
        <version>0.0.1</version>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.junit.platform</groupId>
                <artifactId>junit-platform-engine</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
</dependencies>

The Maven does not take any responsibility for broken compatibilities in this case and the responsibility for the dependency com.github.testng-team:testng-junit5.

JUnit Runner

The JUnit4 library has the Runner implemented in the JUnit5's artifact junit-platform-runner.

For more information see this example.

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-runner</artifactId>
        <version>1.6.2</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Provider Selection

If nothing is configured, Surefire detects which JUnit version to use by the following algorithm:

if the JUnit 5 Platform Engine is present in the project
    use junit-platform
if the JUnit version in the project >= 4.7 and the <<<parallel>>> configuration parameter has ANY value
    use junit47 provider
if JUnit >= 4.0 is present
    use junit4 provider
else
    use junit3.8.1

When using this technique there is no check that the proper test-frameworks are present on your project's classpath. Failing to add the proper test-frameworks will result in a build failure.

Running Tests in Parallel

From JUnit Platform does not support running tests in parallel.

Running a Single Test Class

The JUnit Platform Provider supports the test JVM system property supported by the Maven Surefire Plugin. For example, to run only test methods in the org.example.MyTest test class you can execute mvn -Dtest=org.example.MyTest test from the command line.

Filtering by Test Class Names for Maven Surefire

The Maven Surefire Plugin will scan for test classes whose fully qualified names match the following patterns.

**/Test*.java
**/*Test.java
**/*Tests.java
**/*TestCase.java
Moreover, it will exclude all nested classes (including static member classes) by default.

Note, however, that you can override this default behavior by configuring explicit `include` and `exclude` rules in your `pom.xml` file. For example, to keep Maven Surefire from excluding static member classes, you can override its exclude rules.

Overriding exclude rules of Maven Surefire

...
<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <excludes>
                    <exclude>some test to exclude here</exclude>
                </excludes>
            </configuration>
        </plugin>
    </plugins>
</build>
...

Filtering by Tags

You can use JUnit5 Tags and filter tests by tags or tag expressions.

To include tags or tag expressions, use groups.

To exclude tags or tag expressions, use excludedGroups.

...
<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <groups>acceptance | !feature-a</groups>
                <excludedGroups>integration, regression</excludedGroups>
            </configuration>
        </plugin>
    </plugins>
</build>
...

Configuration Parameters

You can set JUnit Platform configuration parameters to influence test discovery and execution by declaring the configurationParameters property and providing key-value pairs using the Java Properties file syntax (as shown below) or via the junit-platform.properties file.

...
<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <properties>
                    <configurationParameters>
                        junit.jupiter.conditions.deactivate = *
                        junit.jupiter.extensions.autodetection.enabled = true
                        junit.jupiter.testinstance.lifecycle.default = per_class
                        junit.jupiter.execution.parallel.enabled = true
                    </configurationParameters>
                </properties>
            </configuration>
        </plugin>
    </plugins>
</build>
...

Surefire Extensions and Reports Configuration for @DisplayName

Since plugin version 3.0.0-M4 you can use fine grained configuration of reports and enable phrased names together with @DisplayName in you tests. This is the complete list of attributes of particular objects. You do not have to specify e.g. disable, version and encoding. The boolean values reach default values false if not specified otherwise.

...
<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <statelessTestsetReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
                    <disable>false</disable>
                    <version>3.0</version>
                    <usePhrasedFileName>false</usePhrasedFileName>
                    <usePhrasedTestSuiteClassName>true</usePhrasedTestSuiteClassName>
                    <usePhrasedTestCaseClassName>true</usePhrasedTestCaseClassName>
                    <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
                </statelessTestsetReporter>
                <consoleOutputReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5ConsoleOutputReporter">
                    <disable>false</disable>
                    <encoding>UTF-8</encoding>
                    <usePhrasedFileName>false</usePhrasedFileName>
                </consoleOutputReporter>
                <statelessTestsetInfoReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoReporter">
                    <disable>false</disable>
                    <usePhrasedFileName>false</usePhrasedFileName>
                    <usePhrasedClassNameInRunning>true</usePhrasedClassNameInRunning>
                    <usePhrasedClassNameInTestCaseSummary>true</usePhrasedClassNameInTestCaseSummary>
                </statelessTestsetInfoReporter>
            </configuration>
        </plugin>
    </plugins>
</build>
...

Default implementations of these extensions are org.apache.maven.plugin.surefire.extensions.SurefireStatelessReporter, org.apache.maven.plugin.surefire.extensions.SurefireConsoleOutputReporter, and org.apache.maven.plugin.surefire.extensions.SurefireStatelessTestsetInfoReporter.

The aim of extensions is to let the users customizing the default behavior. We are keen on listing useful extensions on Apache Maven Surefire site if you propagate your extensions on GitHub.