This web page describes how to setup and run the Derby Compatibility Tests. These tests stress the compatibility of Derby clients and servers running at different rev levels and on different VMs. We currently test the following combinations:
Clients
Derby 10.1 Derby 10.2 DB2JCC 2.4 Servers
Derby 10.1 Derby 10.2 VMs
jdk1.3 jdk1.4 jdk1.5
To run these tests, you will need to download various VMs and Derby releases. You will also need to edit ant.properties in your home directory, adding variables which point at these VMs and releases. For a complete description of this setup, run the following command, invoking testScript.xml (which lives in the same directory as this web page):
ant -buildfile testScript.xml -projecthelp
To run the tests, you will need to do the following:
Something like the following should work:
#! /usr/bin/bash # # Run the compatibility tests. # # $1 directory to hold output, e.g., c:/cygwin/home/aliceBToklas/derby/mainline/testRun # $2 location of derby trunk, e.g., c:/cygwin/home/aliceBToklas/derby/mainline/testRun/trunk outDir=$1 trunkDir=$2 COMPAT_DIR=$trunkDir/java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility rm -rf $outDir mkdir $outDir ant -buildfile $COMPAT_DIR/testScript.xml -DoutDir=$outDir
When the tests startup, they defensively bring down the Derby network server in case it was running. Generally, it won't be, and you can expect to see a couple vacuous diagnostics, which you may ignore:
_stopServer:
[exec] Could not connect to Derby Network Server on host localhost, port 1527.
[exec] Result: 1
If the tests succeed, ant will happily report:
BUILD SUCCESSFUL
Otherwise, ant will report a failure. In this case, the output directory will hold a file called SUMMARY_COMPAT.txt. This file will identify which test run failed. More specifically, SUMMARY_COMPAT.txt will hold the name of an output file which describes the specific test failures.
You can also run the tests directly using JUnit. In that case, you need a properties file that tells which combinations to test. The file should be named compatibilitytest.properties and it must be placed in the directory from which the tests are started (current working directory).
For example, compatibilitytest.properties could look like this:
##
## Example compatibilitytest.properties file
##
##
## Specify which JVMs to test by defining properties named
## jvm.versions, containing the number of JVMs to test, and jvm.N
## (where 0 <= N < ${jvm.versions}), containing a symbolic name for
## the particular JVM to test.
##
## This example will make the test run with Java SE 5 and Java SE 6:
##
jvm.versions=2
jvm.0=j15lib
jvm.1=j16lib
##
## Specify where to find the JVMs. Property names should match the
## symbolic names in the jvm.N properties. The properties should point
## to the jre/lib directory of a JDK.
##
j15lib=/usr/jdk/instances/jdk1.5.0/jre/lib
j16lib=/usr/jdk/instances/jdk1.6.0/jre/lib
##
## Specify which Derby versions to test. derby.versions contains the
## number of versions to test, derby.versionN (where 0 <= N <
## ${derby.versions}) contains the name of a Derby version.
##
## This example tests four versions of Derby: 10.0.2.1, 10.4.1.3,
## 10.5.3.0 and head of trunk:
##
derby.versions=4
derby.version0=10.0.2.1
derby.version1=10.4.1.3
derby.version2=10.5.3.0
derby.version3=Trunk
##
## Specify where to find the jars for the Derby versions specified
## above. Property names should match names used in the derby.versionN
## properties.
##
10.0.2.1=/opt/db-derby-10.0.2.1-bin/lib
10.4.1.3=/opt/db-derby-10.4.1.3-bin/lib
10.5.3.0=/opt/db-derby-10.5.3.0-bin/lib
Trunk=/code/derby/trunk/jars/sane
##
## We also need to specify whether or not the Derby versions tested
## support the -noSecurityManager option when starting the network
## server. 10.2 and earlier versions don't support it, 10.3 and later
## versions do. Use the version name specified in the derby.versionN
## properties and append "_SA" to the name (short for security
## enabled).
##
10.0.2.1_SA=false
10.4.1.3_SA=true
10.5.3.0_SA=true
Trunk_SA=true
##
## Specify the name of the -noSecurityManager option. (Yes, it's
## required, but it will always be the same.)
##
test.securityOption=noSecurityManager
##
## Tell the tests where to find junit.jar.
##
junit_jar=/usr/share/lib/java/junit.jar
##
## Specify where to find the DB2JCC jar files. This property is only
## required if you test Derby 10.0.2.1, which doesn't have its own
## network client driver.
##
db2jcc_lib=/opt/db2jcc/lib
##
## Optional properties to limit the number of combinations to test.
##
## Test only a with a single Derby Network Server version:
## test.singleServer=N
##
## Test only with a single JVM version on the server side:
## test.singleServerVM=N
##
## Test only with a single Derby Network Client version:
## test.singleClient=N
##
## Test only with a single JVM version on the client side:
## test.singleClientVM=N
##
## The N in the property values is a number that matches the N in one
## of the jvm.N or derby.versionN properties.
##
## Example: Only test combinations that use the trunk version of the
## Network Server, running on a Java 6 virtual machine:
##
## test.singleServer=3
## test.singleServerVM=1
##
To start the compatibility tests, add the Derby jar files and junit.jar to your CLASSPATH, and execute the following command:
java -Djava.security.policy="<NONE>" junit.textui.TestRunner \
org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilityCombinations
If all the tests pass, JUnit will report:
OK (2 tests)
Otherwise, JUnit will report a failure. In that case, the file ServerClient_wombat_failed will contain a list of file names with more information about the failed combinations.