6-June-2008
Overview
In order to demonstrate compliance with the Java Data Objects specification, an implementation must successfully run all of the TCK tests that are not on the "excluded" list. The implementation is hereinafter referred to as the IUT (Implementation Under Test).
The results must be posted on a publicly accessible web site for examination by the public. The posting includes the output of the test run, which consists of multiple log files containing configuration information and test results. For an example of the required posting, please see http://db.apache.org/jdo/tck/final.
Prerequisites
In order to run the TCK, you must install maven 1.x.x. Maven http://maven.apache.org/maven-1.x/ is the driver of the test programs. Note that Maven 2 is not supported.
You must test the IUT on all configurations that the IUT supports. This includes different hardware and operating systems, different versions of Java, and different datastores. The TCK supports Java versions 1.5 to 1.6.
Installation
Download the zip file from the distribution location. Unpack the zip file into a directory of your choice. In this directory you will find:
README.html
maven configuration files project.properties and project.xml (common project definition for all Apache JDO projects including the TCK). These files must not be changed.
lib - this directory contains a directory ext that should contain jar files fscontext.jar and providerutil.jar used by the JNDI tests. The jar files can be found at http://java.sun.com/products/jndi/downloads/index.html. Choose "File System Service Provider, 1.2 Beta 3" from the "Download JNDI 1.2.1 & More" page. Unzip the archive and install them into the lib/ext directory. It is permitted to use a different JNDI implementation; see the README.txt for information on how to configure a different JNDI implementation.
the TCK directory, which has a release-specific name (e.g. jdo2-tck-2.1) and contains:
maven.xml, project.properties, project.xml - the maven definition of the project. These files must not be modified.
build.properties - the maven definition for the IUT. This file may be modified to change any of the IUT properties needed.
this RunRules.html
assertions - contains the assertions file identifying the assertions tested by the tests. This is for reference.
target - this directory contains artifacts of compiling and running the tests. It does not exist in the distribution and will be created by the maven build script.
iut_jars - this directory is where the JDO implementation jars are installed. It is empty in the distribution. To use the maven target runtck.iut (required for an implementation to prove compliance), copy the JDO implementation jar files into this directory. Alternatively, update the build.properties file in the TCK directory to refer to an existing location of the IUT jar files.
src - this directory contains the test configuration files and directories:
testdata - this directory contains data (represented as .xml files) loaded into the datastore for tests. These files must not be modified.
sql - this directory contains DDL to define the tables used in the tests. The files distributed must not be modified. Files may be created for databases for which the DDL for the database under test is not provided.
jdo - this directory contains .jdo metadata files for the persistent classes used in the tests. These files must not be modified.
orm - this directory contains .orm metadata files to map the persistent classes to the sql tables. These files must not be modified except to add DDL-generation information (which is not used by the TCK).
java - this directory contains the source code to the TCK tests. These files must not be modified.
conf - this directory contains the configuration information for the test runs. The files iut-pmf.properties, iut-jdoconfig.xml, and iut-persistence.xml in this directory must be changed to provide properties for the IUT persistence manager factory. The file jndi.properties may be changed to use a different jndi provider. Other files must not be modified, except to put a successfully challenged test case into the trunk/tck20/test/conf/exclude.list. Please see below.
Modifying the Configuration
The Implementation Under Test (IUT) can be installed into the iut_jars directory in the TCK directory. Any .jar files in this directory are added to the class path used to run the tests.
There are properties in the build.properties file in the TCK directory that must be changed to configure the IUT execution and enhancement (optional) environment. These properties begin with iut.runtck and iut.enhancer.
There is are three properties files that must be modified to be IUT-specific, all located in the TCK src/conf directory. The iut-pmf.properties file contains information used to construct the PersistenceManagerFactory used in the tests. iut-jdoconfig.xml and iut-persistence.xml also contain PersistenceManagerFactory properties used only in tests in the org.apache.jdo.tck.api.persistencemanagerfactory.config package.
SQL DDL files are provided for the sql table definitions. The existing files must not be changed, but files may be added in the directory in order to provide DDL for additional databases supported by the JDO implementation under test.
Running the Tests
From the installation directory, change to the TCK directory. From the TCK directory, call "maven build" which will build the jar files used in the tests, create the Derby database, install the schema into the Derby database, and run the TCK on the Reference Implementation. Success indicates that the TCK was installed correctly.
Then call "maven runtck.iut" to run the tests on the Implementation Under Test. This will produce console output plus a directory in the TCK/target/logs directory whose name contains the date/time the tests were started. This directory contains the output of the tests. This is the directory to be published.
Some of the TCK tests require the implementation to support up to 20 instances of PersistenceManager with open transactions simultaneously.
Debugging the IUT while Running TCK tests
Execute "maven help" in the TCK directory in order to get information on running the TCK tests with a debugger. In particular, properties jdo.tck.cleanupaftertest, jdo.tck.cfglist, jdo.tck.identitytypes, and jdo.tck.dblist may be useful.
If you make a change to the IUT enhancer while debugging the TCK tests (for implementations that use an enhancer) you must remove the target/classes directory before continuing in order to make sure that the classes are re-enhanced by the changed code.
Publishing the Results of the TCK Tests
With a successful test run, the log directory with the results of the tests must be published on a publicly-available web site. The unmodified directory is the self-certification of the successful TCK test run.
First Level TCK Appeals Process
If any test does not pass on the JDO implementation under test, this may be due to an error in the implementation or in the TCK test. If you believe that the failure is due to an error in the TCK test, you may challenge the test. To do so, send email to: jdo-dev@db.apache.org with a subject line containing "CHALLENGE" and the name of the test program, e.g. org.apache.jdo.tck.api.persistencemanager.ThreadSafe.java; and the body of the email containing the details of the challenge.
The Maintenance Lead will respond within 15 working days with a decision on whether there is an error in the test case. If the issue is found by the Maintenance Lead to be due to an error in the test case, the Maintenance Lead might patch the erroneous test or add the test to the exclude list. The user can obtain the TCK updates by checking out the latest minor version branch, 2.n.1. If a fix is not provided within 15 working days of the receipt of the challenge, then the user may put the test into the TCK file src/conf/exclude.list and it will not be run as part of the TCK.
Decisions of the Maintenance Lead may be appealed to the full expert group. A vote of the full expert group will be conducted by the Maintenance Lead, and a majority of votes cast will decide the issue. The Maintenance Lead has one vote, as does each member of the expert group at the time of the vote.