The org.apache.river.qa.harness package provides a specialized testing infrastructure designed to support testing of Jini(TM) technology-enabled components and services. This overview provides information of general interest both to test developers and to deployers. A partial list of features provided by this environment includes: Tests are always run in a separate VM, allowing per-test assignments for classpath, codebase, etc. Each test has an associated net.jini.config.Configuration object. Other parameter sources are also used by tests, such as property files and command-line arguments. This document occasionally uses the generic term "configuration" to refer to the entire collection of sources available for obtaining test parameters and Configuration objects. The terms Configuration and ConfigurationFile (code font) will be used when referring to an object from the net.jini.config package, and the term ConfigurationFile (normal font) will be used to refer to a file from which a ConfigurationFile object may be derived.

Installation Directories

The test harness and the Jini Technology Starter Kit (starter kit) may be installed in user selected directories. Installation parameters and user-supplied properties that control harness execution are provided in a deployment properties file which is named on the command line used to run the harness. The directory containing the starter kit is identified in this file by the property org.apache.river.jsk.home. The directory containing the test harness kit is identified in this file by the property org.apache.river.qa.home. In property files, and frequently in this document, these directories will be referred to symbolically as ${org.apache.river.jsk.home} and ${org.apache.river.qa.home}. The test suite to be executed by the harness is installed separately and may require additional properties to define the installation environment of the test suite.

Distributed Execution

The test harness supports a distributed mode of execution, in which the test runs on a master host, and services are started on the master host or on a slave host. The master can support an arbitrary number of slaves. The test harness must be started on every participant, and the command line must include a system property that names the participants. For example, the definition:
  -Dorg.apache.river.qa.harness.testhosts=foo,bar
would indicate that two hosts, named foo and bar, are participants. The first name in the list is interpreted as the master. The harness does not begin executing tests until all participants have been started. Any logging output generated on the slave host will be merged into the logging output generated by the master. All harness and test properties are transmitted from the master to the slave at test execution time. However, the slave obtains installation properties from its own deployment properties file. This allows participants to be a mix of platforms and to have different installation characteristics.

When a test starts a service, the master selects the service host based on the current host selection policy. The policy to use is defined by the value of the test property org.apache.river.harness.servicehostpolicy. This property may have one of the following values:

random
host is selected at random from all participants
remoterandom
host is selected at random from slaves
roundrobin
service host is selected in sequence from the ordered list of participants, starting from the first slave. The master is the last host chosen.
remoteroundrobin
service host is selected in sequence from the ordered list of slaves. The master is never selected.
This policy may be overridden on a per-service basis; see the hosts service property for details. The most common use of such an override is to force a test service to always run on the master, due to some assumption built into the test service or the test.

The Kits

There are two versions of the harness kit. The binary kit is a packaging that contains the minimum set of files sufficient to provide full functionality. The harness provides mechanisms to access resources in the harness and test suite JAR files. The packaging of the binary distribution takes advantage of this by placing most resource files supplied in the harness source distribution into the harness JAR file. This includes policy files, ConfigurationFiles, and property files. One resource cannot be accessed via a URL (the JAAS trust store) and is included as file in the kit. The binary kit is targeted for production testing, and is the kit used for production testing by the Jini Technology Group.

There is also a source kit which provides all of the Java source files and resource files needed to build the harness, as well as the javadoc for the harness API. The JAR files from the binary kit are also included.

Accessing resources

Property definitions that refer to resource files may use two special tokens to supply those references. Using these tokens rather than a direct reference to the resource provides several advantages:

A token of the form <file: path> is assumed to refer to a file named by path. The harness will search for that file following a search path described in the next section. If the file is found, the token is replaced by the fully qualified name of the first occurance of the file in the search path. Otherwise a TestException is thrown.

A token of the form <url: path> is assumed to refer to a file named by path that may be embedded in a JAR file. The harness will first search for that file as described above. If the file is found, the token is replaced by a file URL specifying the fully qualified path of the file. Otherwise, the test and harness jar files are examined, resolving the reference first relative to the location of the test description file, then relative to the root of the test JAR file, and finally relative to the root of the harness JAR file. If the file is found, the token is replaced with an appropriate JAR file URL. Otherwise, a TestException is thrown.

Search Path

When the harness processes the <url:filename> or <file:filename> tokens, it searches for filename along a search path defined by the searchPath test property. Regardless of the value of this property, the search path will always include ${org.apache.river.qa.home}. In general, searchPath should be defined self-referentially. For instance, it can be convenient for debugging purposes to modify the search path on the command line as follows:
      -searchPath '${searchPath},/files/suitedir/src'
which would add the /files/suitedir/src directory to the search path. The list must be comma-separated. Whitespace in file and directory names are allowed. The search is performed relative to the base of each searchPath component, and relative to each (component + current test description file directory) so that short names can be used for files that are colocated with the test description file.

ConfigurationFileSets

Each Jini test is run with a selected ConfigurationFileSet. A ConfigurationFileSet is a collection of compatible files used by the activation system, services, and the test. Most of these files are ConfigurationFiles, but the set may include other types of files, such as keystores or property files. In the binary distribution, all of these resources are embedded in the harness JAR file. When a test is run, the ConfigurationFileSet to use is identified by a user specified tag. Supported tags include: The set of tags to use in a test run is specified either on the command line via the option:
  -org.apache.river.qa.harness.configs tag1[,tag2,...]
or in the deployment property file via the following definition:
  org.apache.river.qa.harness.configs=tag1[,tag2,...]
If multiple tags are specified, the same test is run successively with each configuration. The default value is none. The tag is used by the harness to locate the root directory of the ConfigurationFileSet. These root directories are located at:
     <url: harness/configs/jrmp>
     <url: harness/configs/jeri>
     <url: harness/configs/jsse>
     <url: harness/configs/http>
     <url: harness/configs/https>
     <url: harness/configs/kerberos>
The none tag is a special case, indicating that configurations are not to be used. In this case, the default values built into the Jini components and tests are used. There is no configuration tree associated with the none tag.

The files that constitute a ConfigurationFileSet are static data sources and are intended to be reusable to minimize the number of files that need to be supported. Configuration entries that must be defined dynamically are provided as overrides when the ConfigurationFile is loaded by the ConfigurationProvider. The test harness includes built-in support for specifying some overrides, such as the service persistence directory. A mechanism is also provided to allow tests to supply additional overrides when a service is started.

Under each ConfigurationFileSet root directory lies a tree of files. Each tree has the following structure:

 <root>
       /configset.properties (optional)
       /test.config
       /norm
            /norm.config
       /mercury
            /mercury.config
       ...
       other Jini service config directories
       ...
       /sharedgroup
            /sharedgroup.config
       /phoenix
            /phoenix.config
       /starter
            /starter.config
For each Jini service there is a default service ConfigurationFile (e.g. norm.config). There is also a default ConfigurationFile for the service starter, the activation system, the activation group VM, and the administrative service for the shared group (SharedGroupImpl). Each configuration has a default test ConfigurationFile named test.config that is expected to be reusable for most tests. Tests may specify an override for the entries in test.config by naming an optional test configuration file via the testConfiguration test property, who's value would typically be defined via a <url:> or <file:> token. When the test searches for an entry, it will first look in the file named by this property. If the entry is not found, the default test.config file is searched.

Test Properties Values

Tests for Jini services may require access to numerous configuration values. For instance, a simple JoinManager test might require instantiation of several lookup services, all possibly configured in different ways. To meet this requirement, the test environment supports an extensive hierarchy of data sources. Support utilities provide easy, consistent access to the values defined in these sources.

This hierarchy provides various levels of scope for defining test property values. For instance, there are files that apply to all tests, files that are accessed only for all tests for a particular component (e.g. JoinManager), files that may be included by arbitrary groups of tests, and files that are accessed only for a specific test. This makes it possible to establish a hierarchy of default values that can be overridden at different levels of granularity. And, of course, very simple tests can also be written that do not directly access any configuration information.

All of the configuration information available to a test is represented by an instance of the QAConfig class, which is passed to the test during test setup. This object is constructed for a test by (logically) merging the contents of a set of property files, the command-line arguments, and a single test ConfigurationFile. The QAConfig object provides accessors for obtaining the String values provided by these sources. In general, this collection of data will be referred to as the test properties, even though some of the sources are not true Properties objects. The set of sources follows, in order of precedence (highest first):

Each of these sources is described in more detail below. Access to non-string Entries provided by the test ConfigurationFile must be obtained directly from the associated Configuration object, which is available to a test via the getConfiguration() method of the QAConfig object.

Slave Properties Overrides

In a distributed test, it may be necessary for the slave to use a local property value in place of the value supplied by the master. These overrides may be defined in the slave deployment properties files through the property org.apache.river.qa.harness.slaveTestOverrides. The value of this property is always obtained locally, and is a comma or space separated list of property names who's values should also be obtained locally.

Dynamic Properties

On rare occasions, it may be necessary for a test to define a configuration value on the fly. The most likely case is when a value needed for a service property cannot be predefined in a static configuration file. QAConfig provides a setDynamicProperty method to support this. These properties are automatically reset for each test.

Command Line

Test properties can be defined on the harness command line. Any pair of tokens of the form "-foo bar" will be treated identically to the property definition "foo=bar". Such a definition generally will not be defined as a system property for the test VM, but will be accessible via the test property accessor methods defined by QAConfig.

System Properties

System properties are included when searching for test property values. Note, however, that system properties defined when starting the harness are not automatically inherited by the test VM. The use of system properties should be considered an internal implementation detail, employed by the harness VM to establish the environmental context of the test VM. System properties should not be directly used to configure test behavior.

Test Description File

Every test is defined by a test description properties file. These files must be located in the primary test JAR file identified by the -testJar command-line option, and must have an extension of ".td". The name of a test is the pathname to the test description file relative to the root of the test Jar file. Test description files must contain values for the following keys:
testClass
the name of the test implementation class. If the class is in the same directory as the test description file, then the unqualified class name may be used. Otherwise the fully qualified name must be given.
testCategories
a comma-separated list of test category names
These property values cannot be overridden or defined anywhere else. There are also additional keys that are associated with test descriptions, but which may be defaulted in a more global property file:
testCodebase
the codebase for the test
testPolicyfile
the security policy file for the test
testClasspath
the classpath for the test
testConfiguration
the name of the test ConfigurationFile
testjvm
the fully qualified path to the VM executable
testjvmargs
options and properties to be passed to the test VM. Multiple values must be separated by commas. Special characters, such as a leading '-', must be included. Property definitions must be expressed in the usual form: -Dname=value. White space is not removed so that paths containing white space characters are supported.
testOverrideProviders
a white-space or comma separated list of override providers to be registered before executing the test.
testMaxRetries
an integer defining the number of times a test should be rerun on failure. The default value is 1. Test reruns are not performed unless the test property org.apache.river.qa.harness.rerunFailedTests is also defined and has the value true. The default value of this property is false.
The value of testConfiguration is optional and would typically be supplied as a <url:> token referring to a ConfigurationFile included in the primary test JAR file. This file contains entries that override or augment the entries in the default test.config file for the configuration.

The test description file also supports an 'include' mechanism. If an entry of the form

   include=resourceName
is found, where resourceName is expressed as a file name either relative to the location of the test description file, or relatiive to the root of the test JAR file, then the properties provided by the resource are added to the test description entries. If an entry for 'include' is not found, then entries of the form 'include0=resourceName', 'include1=resourceName', ..., 'includeN=resourceName' are searched for and loaded until an entry for includeN is not found. If an included key already exists in the test description file then the include entry is discarded. This prevents included files from overriding entries directly specified in the test description. If multiple includes contain values for the same entry, higher numbered includes superceed lower numbered ones.

Test Configuration Object

Each test has an associated Configuration object that is loaded from a test ConfigurationFile in the selected ConfigurationFileSet. The object may be obtained within the test by calling the getConfiguration() method on the QAConfig object. The Configuration object associated with the test is typically used to obtain objects such as exporters and proxy preparers, and may be passed in the constructors for utilities such as JoinManagers. The harness also includes the Configuration object in the search when resolving a test property value. If the fully qualified entry name is the same as the test property being searched for, and if the entry returns a String, then the entry is considered a match. This makes it possible to pass values to an individual test that are dependent on which configuration is selected.

ConfigurationFileSet Properties File

The root directory of a ConfigurationFileSet may contain a properties file named configSet.properties. This file may be used to provide values that are dependent on which ConfigurationFileSet is selected. These values are applied for all tests using the selected set. One use of this file is to provide system properties for the test VM which are only appropriate for secure configurations.

Deployment Properties File

When the harness is invoked from the command line, the name of a deployment properties file must be provided. The example prototype for this file is located in ${org.apache.river.qa.home}/src/org/apache/river/resources/basicDeployment.prop. in the source distribution kit. Typically, a test suite would define additional properties which must be added to this prototype.

Default Properties File

A default properties file is the last properties file searched to resolve a test properties key. It is referenced via the token <url:org/apache/river/qa/resources/qaDefaults.properties>.

Defaults in the Code

Finally, the test configuration accessors require a default value to be supplied in the test code. If the configuration key is not found, the default value is returned.

Symbolic References in Test Property Values

When a test property value is obtained from QAConfig, simple string substitution is performed. If the value contains a string of the form $foo or ${foo}, then the test properties are searched using the key foo. If a value for foo is found, then that value replaces the original reference. Substitutions are performed recursively. Within a single source, a given property may only be defined once. However, a higher precedence source can reference a lower precedence source self-referentially. For instance, if 'foo' was defined in qaDefaults.properties as:
    foo=bar
and was then redefined in the test description file as:
    foo=${foo} bell
the resulting value for foo would be 'bar bell'.

After all symbolic substitutions are performed, the resulting value is searched for the following tokens:

Following this, the values of the <url:> and <file:> tokens are evaluated. This ordering allows the tokens in the previous list to be embedded in these file based tokens.

For additional details on the string substitution mechanism, see org.apache.river.qa.harness.Resolver.

Custom Security Policy Provider

The test harness kit provides a number of default policy files for tests and for the supported Jini services. In many cases, test suite developers will find that additional grants are required to satisfy testing requirements. Rather than duplicating and tweaking the supplied policy files, a MergedPolicyProvider is supplied in the harness that accepts multiple policy files and produces a policy that represents the union of the policy files. This union is not perfect, but is considered adequate for testing purposes. For example, if policy file A granted read access to foo and policy file B granted write access to foo, then an access check for read,write access to foo would still fail.

The policy provider is ${org.apache.river.qa.home}/lib/mergedpolicyprovider.jar (in both the source and binary distributions) and must be installed in the extensions directory of the JDK used to run the test harness. On startup, the harness will verify that the provider is loaded by the extension class loader and will exit with an error message if this is not the case.

When the harness starts a test or service, it specifies the default policy file to use via the normal java.security.policy property. If additional grants are required, additional policies can be specified by supplying a list of policy files (using <url:> or <file:> tokens) through the system property org.apache.river.qa.harness.policies for the VM containing the test or service. This list may be separated by commas or white space.

Note: due to a known bug, the grants supplied by the MergedPolicyProvider are not applied to services in this release of the harness.

Trust Resources

Both the source and binary harness distributions supply a variety of trust resources. These resources include JAAS login configuration files, keystores, truststores, etc. Because the value of the javax.net.ssl.trustStore property must refer to a file, the trust store supplied by the harness is packaged as a separate file in ${org.apache.river.qa.home}/harness/trust/truststore in both the source and binary distributions.

Starting Jini Services

The AdminManager provides the following method for starting Jini services:
   public Object startService(String serviceName)
where serviceName is a string that identifies the service to start. This method returns the proxy for the service. serviceName is used to construct a set of property names used to obtain service parameters from the test properties. See Service Admins for details on this mechanism. It is possible to override some of these property values (in the test description file, for instance) to affect the behavior of the service. The most common overrides are defined to specify the groups and locators used by the service. Default property values for all of the contributed Jini services are defined in the default property file, <url:org/apache/river/qa/resources/qaDefaults.properties>. By convention, the serviceName for a service is the name of the public interface that the service implements. Therefore, the contributed Jini services have the following serviceNames:
      net.jini.core.lookup.ServiceRegistrar
      net.jini.lease.LeaseRenewalService
      net.jini.discovery.LookupDiscoveryService
      net.jini.event.EventMailbox
      net.jini.core.transaction.server.TransactionManager
      net.jini.space.JavaSpace
If a test implementation requires the creation of a specialized test service, that test service may be started with the startService method as long as all of the mandatory service properties are defined, as described in Service Admins.

Service Configuration Overrides

When a Jini service is started, its associated ConfigurationFile is obtained from the ConfigurationFileSet. These are static files, so Configuration values generated at run-time must be passed as overrides. A set of overrides are passed automatically during startup: A test may also supply additional overrides to a service. The test may register one or more instances of the org.apache.river.qa.harness.OverrideProvider interface with QAConfig by calling its addOverrideProvider method. The testOverrideProviders property may also be defined to identify a list of providers to be registered automatically before starting the test. Whenever the test or a service is started, the set of registered override providers is obtained and the getOverrides method called on each. The string arrays returned by these override providers are merged into the set of overrides passed to the test or service. Any such overrides that have the same entry name as the automatically generated overrides will take precedence.

* the exporter required by an activatable service is different than that required by a nonactivatable service. To avoid duplicating configuration files for activatable and nonactivatable variants, a scheme was developed to allow the specification of the appropriate exporter to use through an override. See Service Configuration Files for more details.

Auxiliary Services

The class servers required to support test execution are, by default, automatically started during test setup. The activation system and any required activation groups (or non-activatable groups) are also started if needed by services. These auxiliary services will be automatically stopped during test teardown. The default behavior of setup in QATest is to: The properties that control these auxiliary services are defined in the example deployment properties file, ${org.apache.river.qa.home}/src/org/apache/river/qa/resources/basicDeployment.prop. supplied in the harness source distribution.

The Starter Kit Class Server

By default, the harness will automatically start a class server on port 8080 to serve JAR files from ${org.apache.river.jsk.home}/lib-dl. This can be inhibited by defining
   org.apache.river.qa.harness.runjiniserver=false
To change the port (for example, to 9000), define the following:
   org.apache.river.jsk.port=9000
   org.apache.river.qa.harness.dldir.9000=${org.apache.river.jsk.jars}
The second line is needed to support generating httpmd URLs.

The QA Class Server

By default, the harness will automatically start a class server on port 8081 to serve JAR files from ${org.apache.river.qa.home}/lib. This can be inhibited by defining
   org.apache.river.qa.harness.runkitserver=false
To change the port (for example, to 9001), define
   org.apache.river.qa.port=9001
   org.apache.river.qa.harness.dldir.9001=${org.apache.river.qa.jars}
The second line is needed to support generating httpmd URLs.

The Activation System

The first time the AdminManager is called to start an activation group on a master or slave host, an activation system will be started automatically to support the group. The harness by default will start phoenix. For more information, see the JavaDoc for ActivationSystemAdmin. If harness management of the activation system must be disabled, the following may be defined:
   org.apache.river.qa.harness.runactivation=false
This would be done only in unusual debugging situations.

The Global Shared Group

The first time the AdminManager is called to start an activatable service on a master or slave host, an activation group will automatically be started to support the service (which in turn will cause the activation system to be started). By default, the harness creates a shared activation group that is used for all other activatable service started by the same AdminManager. This can be inhibited by defining
   org.apache.river.qa.harness.shared=false
which will cause each service to run in a private activation group.

Test Logging

The harness merges the output generated by the activation system and class servers into the logging output from the test. It also merges the output generated from slave tests into the logging output stream of the master test unless the following is defined:
  org.apache.river.qa.harness.slavepipe=false
This typically is only done for debug/demo purposes.

The test harness configuration files are set up to treat the system property java.util.logging.config.file specially. If a logging properties file is specified on the command line when starting the test harness on a master or slave, then that same logging property will automatically be set on the test and service VMs started by the master or slave harness.

Test output is generated by calls to the Java logging API to a logger named org.apache.river.qa.harness. A custom ReportHandler forwards all log records to System.out. A custom formatter will optionally prepend the record with the abbreviated source (last class name token and method name) and level indicator. If the test is running distributed, then the name of the host on which the output was generated will also be prepended. Timestamps are generated at intervals, specified by the logging property org.apache.river.qa.harness.HarnessFormatter.timeinterval. If a log message is generated and the number of seconds specified by this property have elapsed since the last timestamp, a new timestamp is generated on a separate line. The default value is ten seconds. If this property is defined with a value of zero, the timestamp is generated and prepended on every logging record.

There are four boolean logging properties that can be defined to control the display of the source, level, timestamp, and host info:

   org.apache.river.qa.harness.HarnessFormatter.showtime
   org.apache.river.qa.harness.HarnessFormatter.showsource
   org.apache.river.qa.harness.HarnessFormatter.showlevel
   org.apache.river.qa.harness.HarnessFormatter.showhost
These must be defined in the logging properties file rather than in the harness configuration files. If undefined these properties default to true.

A reference to the org.apache.river.qa.harness logger is provided by QATest via the static protected field logger.

Test Timeouts

The test harness can be configured to timeout a test after a fixed period of time. The definition:
   org.apache.river.qa.harness.timeout=<val>
where <val> is an integer, will cause the test VM process to be interrupted val seconds after the test was started. The default value of val is zero, which disables timeouts.

Supporting httpmd Codebases

When a test is configured for security, test and service codebases will automatically be converted to httpmd URLs. This behavior is globally controlled by the test property org.apache.river.qa.harness.integrityhash. If this value is defined and has any value other than "none", then any http URL in a test or service codebase will automatically be converted to an httpmd URL, using the value of the integrityhash property as the name of the hashing function. If the value of this property consists of multiple comma-separated tokens, then each token is assumed to be the name of a hashing function, and the function to use is chosen randomly for each httpmd URL generated. This property would typically be defined in the ConfigurationFileSet property file, located at <url:harness/configs/*/configSet.properties>. This global flag can be overridden on a per-service basis by defining a value for the service property named integrityhash.

In order to perform the conversion, it is necessary to know the document directory associated with the http URL. To obtain this, a test property org.apache.river.qa.harness.dldir.<port> must be defined where <port> is the port number associated with the URL.

API Overview

This section summarizes the classes and methods most often referenced by test implementation code:

org.apache.river.qa.harness.Test

All tests must implement the org.apache.river.qa.harness.Test interface:
    public interface Test {
      public void setup(QAConfig config) throws Exception;
      public void run() throws Exception;
      public void teardown();
    }
All three of these methods are called by the harness in the order listed, unless setup throws an exception. See org.apache.river.qa.harness.QATest for additional details on how the harness interacts with implementations of this interface. A test which does not require the features provided by QATest could implement this interface directly. This document is written with the assumption that test implementations will extend QATest.

org.apache.river.qa.harness.Admin

The Admin interface is implemented by objects that are used to control services. Several implementations of this interface are included with the test harness, as described later. Three methods are defined:
      public void start() throws RemoteException, TestException;
      public void stop() throws RemoteException;
      public Object getProxy();
After the start method is called to start the service, the getProxy method may be called to obtain its proxy. Also, start and stop should only be called once; the behavior of additional calls is undefined.

org.apache.river.qa.harness.AdminManager

AdminManager acts as a factory for service admins, and supports orderly startup/shutdown of services used by a test. Methods are provided for starting and stopping services, and for obtaining the admin of a service for those rare cases where access to the admin is necessary. The most commonly called methods of this class are:

startService(String serviceName) starts a jini service, as described above
startLookupService() calls startService("net.jini.core.lookup.ServiceRegistrar");
getAdmin(Object proxy) returns the admin associated with a given service proxy

In addition, the destroyAllServices() method is always called by the teardown method implemented by QATest. AdminManager maintains a table of all the services that were started with the startLookupService and startService methods. The destroyAllServices method uses the entries in this table to perform an ordered shutdown of the services. Because the teardown method implemented by the QATest base class always calls destroyAllServices, it is generally unnecessary for a test to call explicitly any of the destroy methods provided by AdminManager.

Tests rarely need to directly access the admin object for a service, since the AdminManager object mediates most service related activity. However, occasionally it may be necessary to obtain a service admin in order to obtain information about the parameters that were used to start the service.

org.apache.river.qa.harness.QAConfig

QAConfig exposes a rather large number of methods. However, test writers typically call only a small subset of them, specifically accessors of the form
  get<type>ConfigVal(key, default)
which obtains a value of a test property interpreted as <type>. These methods are used by tests that are written to access property values to control test behavior. The set consists of:

public String getStringConfigVal(String key, String default)
public int getIntConfigVal(String key, int default)
public float getFloatConfigVal(String key, float default)
public double getDoubleConfigVal(String key, double default)
public long getLongConfigVal(String key, long default)
public booleangetBooleanConfigVal(String key, boolean default)

org.apache.river.qa.harness.QATest

This base class is inherited by most tests. It is declared to implement the Test interface, and provides concrete implementations of the setup and teardown methods. Subclasses must implement the abstract
    public void run() throws Exception
method. This method provides the body of the test. A normal return indicates a successful test. An exception return indicates test failure*.

Optionally, the test implementation may override two additional methods:

   public void setup(QAConfig config) throws Exception
   public void teardown()
setup is called before the run() method to perform any pre-run setup, such as starting required services. Because the default implementation starts the class servers, it is important that overrides of this method call super.setup(config) before starting any services. The teardown method is called by the harness after the run() method returns, or after setup is called if setup throws an exception. The default teardown implementation will stop any services started as a result of calling the startService method; it will also shut down the shared groups, the activation system, and any class servers which were started. As a result, it is generally unnecessary to override the default implementation. However, if the test class implementation overrides teardown, the superclass teardown method should be called to ensure that the superclass cleanup code is executed. Exceptions thrown by teardown are silently ignored by the harness.

QATest provides the accessor:

   public QAConfig getConfig()
which returns the QAConfig object passed in the setup call.

Tests written for the test harness must either extend QATest or implement the Test interface, and must provide a no-arg constructor.

* The harness supports the ability to register FailureAnalyzers with tests which can inspect the exception and determine the pass/fail status of a test. However, there are no analyzers implemented at this time, so exceptions are always treated as failures. See the JavaDoc for the FailureAnalyzer interface for more details.

org.apache.river.qa.harness.TestException

TestException is thrown by a test to signal a failure. In most cases, unexpected exceptions (both checked and unchecked) are not caught by tests and are also interpreted as test failures. Test writers are discouraged from catching exceptions other than those which are expected and which result from correct behavior of the item under test.

Running a Test

To run the harness, at least two JAR files must be included in the classpath:
        ${org.apache.river.qa.home}/lib/jiniharness.jar
is the JAR file containing the harness code and resources, and
        ${org.apache.river.jsk.home}/lib/jsk-platform.jar
is the starter kit platform that is typically included in the classpath of Jini clients and services.

The user must also specify the name of a deployment properties file on the command line that defines installation values, such as the value of ${org.apache.river.qa.home}. A prototype of this file is available at ${org.apache.river.qa.home}/src/org/apache/river/qa/resources/basicDeployment.prop in the source distribution. The user must also specify the name of the primary test suite JAR file via the command-line option -testJar foo.jar. In this example, foo.jar contains the test description files for the tests making up the suite. The user must also specify a list of categories or specific tests to run. For the following examples, assume the user has placed a modified version of this file in /tmp/myHarness.prop. Also, assume the starter kit has been installed in /files/jini2_1, the harness kit has been installed in /files/jiniqa2_1_harness, and the test suite is installed in /files/testsuite, with the primary JAR file in /files/testsuite/lib/tests.jar.

To specify a set of test categories to run, use the -categories option, for example:

  java -cp /files/jiniqa2_1_harness/lib/jiniharness.jar:/files/jini2_1/lib/jsk-platform.jar \
       org.apache.river.qa.harness.QARunner \
       /tmp/myharness.prop \
       -testJar /files/testsuite/lib/tests.jar \
       -categories joinmanager,javaspace
This would limit test execution to those tests whose test descriptions included either joinmanager or javaspace in their definitions of testCategories. The configuration used would be none unless the value of org.apache.river.harness.configs had been modified in /tmp/myharness.prop.

To specify an explicit list of tests to execute, list them after the -tests option. For example,

  java -cp /files/jiniqa2_1_harness/lib/jiniharness.jar:/files/jini2_1/lib/jsk-platform.jar \
       org.apache.river.qa.harness.QARunner \
       /tmp/myHarness.prop \
       -testJar /files/testsuite/lib/tests.jar \
       -tests org/apache/river/test/spec/joinmanager/EqualsTest
This would execute only the hypothetical joinmanger EqualsTest test. The following command line would run this test using the jeri ConfigurationFileSet:
  java -cp /files/jiniqa2_1_harness/lib/jiniharness.jar:/files/jini2_1/lib/jsk-platform.jar \
       org.apache.river.qa.harness.QARunner \
       /tmp/myHarness.prop \
       -testJar /files/testsuite/lib/tests.jar \
       -tests org/apache/river/test/spec/joinmanager/EqualsTest \
       -org.apache.river.qa.harness.configs jeri
The -tests argument is optional, and if present must be followed by a comma-separated list of test description file names specified relative to the root of the primary test suite JAR file. This identifies the set of tests to place in the run list, which may be additionally filtered by category. If omitted, all tests are placed in the run list.

The -categories argument is optional, and if present must be followed by a comma-separated list of test categories. When the run list is processed, tests which do not belong to one of the specified categories are discarded from the list. If omitted no category filtering is done.

One or both of these options must be specified.

Service Admins

Admins are an abstraction for controlling services. All admins implement at least two methods: start and stop. There are seven admins provided with the harness. They are:
ActivatableServiceStarterAdmin manages any activatable service compatible with org.apache.river.start.ServiceDescriptor.
NonActivatableServiceStarterAdmin manages any non-activatable service compatible with org.apache.river.start.ServiceDescriptor.
RunningServiceAdmin manages any service which is already running and registered in a lookup service.
SharedGroupAdmin manages a shared activation group
ActivationSystemAdmin manages an activation system
ClassServerAdmin manages an in-process class server
RemoteServiceAdmin manages a remote service - used internally when running distributed
In addition to the start/stop methods, these admins all include accessors for obtaining their initial service parameters. Examples of these accessors include getCodebase, getClasspath, getPolicyFile, etc. The ActivatableServiceStarterAdmin also provides methods for obtaining the ActivationGroup and ActivationID of the service.

In general, it is not necessary for tests to instantiate or access these admins. AdminManager provides utility methods which start/stop services and manage these admins transparently.

Services require a variety of parameters to be specified when they are started. These parameters are generated by the admin based on values obtained from the test properties. When an admin is constructed, it must be provided with a serviceName that is used to search QAConfig for configuration values. By convention, this identifier is formatted as the fully qualified name of the service interface or class to be managed, for example:

  net.jini.discovery.LookupDiscoveryService
Some of these properties refer to files or resources embedded in the harness or test JAR files. Definitions of these property values should use the appropriate token type.

The serviceName is used as a prefix string for generating service property names. The admins accept an instance counter, which is a zero based integer that can be used when multiple instances of the same service are needed for a test. The counter is used as a suffix when generating property names. It is not necessary for test code to manage this counter; the support methods in AdminManager maintain counters for every service. A set of well-known service identifiers are used to generate the set of property names associated with a service. These well-known identifiers, categorized by admin type, are:

ActivatableServiceStarterAdmin

Identifier Usage
type a mandatory property having the value "activatable".
impl the implementation class
host if defined and running distributed, the master or slave host for the service. Valid values are:
master
always run on the master host
slave
modify the selection policy to always select a slave by converting roundrobin to remoteroundrobin or random to remoterandom
slaveN
where "N" is an integer, use the Nth entry in the host list (modulo the host list size). If the index is 0, it is reset to 1.
name
where name is the name of the master or slave host. If name is not in the list, a TestException is thrown. if this property is undefined, the default selection policy is used.
classpath the service classpath
codebase the service codebase
policyfile the service security policy file
component the component name to use when constructing entry names for entries in the service Configuration. For example, reggie has a value of "org.apache.river.reggie" for this property. This property must be defined if the tojoin or membergroups properties are defined, or if log property has any value other than "none".
log a token used in constructing the log directory path. If undefined, a value of "log" is assumed. If set to "none", no persistence log override definition is generated. The component property must be defined unless log is set to "none".
tojoin for Jini services, the groups/locators to join. If this property is defined, then the component property must also be defined
membergroups for lookup services, the member groups. If this property is defined, then the component property must also be defined.
port the multicast port (lookup services only)
serviceConfiguration the name of the service configuration file relative to the root directory of the ConfigurationFileSet. If the value of this property is "-", then no ConfigurationFile is loaded, but command-line overrides are still generated.
starterConfiguration the name of the service starter configuration file relative to the root directory of the ConfigurationFileSet directory. If the value of this property is "-", then no ConfigurationFile is loaded.
serverjvmargs any options or properties to pass to the service VM, separated by commas. Any special characters, such as a leading '-', must be included. Properties must be defined using standard command line syntax, i.e. "-Dfoo=bar". White space is not removed so that paths containing white space are supported.
serverjvm the absolute path name of the VM to use to run the service. The default value is the same VM used to run the test.
activationhost The host name of the activation system to contact. If undefined the local host is contacted.
activationport The port number of the activation system to contact. If undefined or 0, the default activation port is used.
integrityhash the name of the hashing algorithm to use to generate httpmd URLs. If defined and having a value other than "none", http URLs in codebases are automatically converted to httpmd URLs using the hashing algorithm named by this property. This value overrides the value defined by the global property org.apache.river.qa.harness.integrityhash and has the same semantics for supporting multiple values.
restart the value of the restart parameter to pass to the activation system when the service is registered. If undefined, the default value is true.
serverjvmargs is unused if the activatable service is run in the global shared group. However, if the global group is disabled, then the ActivatableServiceStarterAdmin will create a private shared group for the service and the serverjvmargs defined for the service will be merged with those for the private shared group VM.

NonActivatableServiceStarterAdmin

Identifier Usage
impl the implementation class
type a mandatory property having the value "transient" or "persistent"
classpath the service classpath
codebase the service codebase
policyfile the service security policy file
component the component name to use when constructing entry names for entries in the service Configuration. For example, reggie has a value of "org.apache.river.reggie" for this property. This property must be defined if the tojoin or membergroups properties are defined, or if log property has any value other than "none".
log a token used in constructing the log directory path. If undefined, a value of "log" is assumed. If set to "none", no persistence log override definition is generated. The component property must be defined unless log is set to "none".
tojoin for Jini services, the groups/locators to join. If this property is defined, then the component property must also be defined
membergroups for lookup services, the member groups. If this property is defined, then the component property must also be defined.
port the multicast port (lookup services only)
serviceConfiguration the name of the service configuration file relative to the root directory of the ConfigurationFileSet directory
starterConfiguration the name of the service starter configuration file relative to the root directory of the ConfigurationFileSet directory
serverjvm the absolute path name of the VM to use to run the service. The default value is the same VM used to run the test.
serverjvmargs any options or properties to pass to the service VM, separated by commas. Any special characters, such as a leading '-', must be included. Properties must be defined using standard command line syntax, i.e. "-Dfoo=bar". White space is not removed so that paths containing white space are supported.
integrityhash the name of the hashing algorithm to use to generate httpmd URLs. If defined and having a value other than "none", http URLs in codebases are automatically converted to httpmd URLs using the hashing algorithm named by this property. This value overrides the value defined by the global property org.apache.river.qa.harness.integrityhash and has the same semantics for supporting multiple values.
Note that this admin supports both persistent and nonpersistent services. If the type property is "transient", then the nonpersistent service is started and the value of the log property is ignored. If type is "persistent", the persistent variant is started.

RunningServiceAdmin

Identifier Usage
type a mandatory property having the value "running"
impl a mandatory property providing the name of the service class to locate
tojoin a mandatory property providing the set of groups and locators to be used to look up the service
Note that this admin is not used by any test, and the implementation of this admin has not itself been tested.

SharedGroupAdmin

Identifier Usage
type a mandatory property having the value "group"
impl the implementation class of the shared group
classpath the shared group classpath
codebase the shared group codebase
policyfile the policy file for the shared group VM
serverjvmargs any options or properties to pass to the service VM, separated by commas. Any special characters, such as a leading '-', must be included. Properties must be defined using standard command-line syntax, i.e. "-Dfoo=bar". White space is not removed so that paths containing white space are supported.
activationhost The host name of the activation system to be contacted. If undefined, the local host is contacted.
activationport The port number of the activation system to be contacted. If undefined or 0, the default activation port is used.
implPrefix the serviceName identifying the administrative service to load into the shared group
Note that there is no property to specify the shared group persistence log. SharedGroupAdmin always generates a reference to a unique log directory in the default temp directory.

ActivationSystemAdmin

Identifier Usage
type a mandatory property having the value "rmid" or "phoenix"
policyfile the security policy file for the activation system
codebase the codebase for the activation system
log the token used to generate the persistence directory name
logdir the activation system log directory name token
serverjvmargs any options or properties to pass to the service VM, separated by commas. Any special characters, such as a leading '-', must be included. Properties must be defined using standard command-line syntax, i.e. "-Dfoo=bar". White space is not removed so that paths containing white space are supported.

ClassServerAdmin

Identifier Usage
type a mandatory parameter having the value "classServer"
impl a mandatory parameter providing the name of the server class to instantiate
port the TCP port the server is to use (default = 8080)
dir a mandatory parameter providing the directory from which JAR files are to be served
serverjvmargs additional options to supply to the class server. The only supported values are -trees and -verbose. Any properties specified are ignored since the class servers run in the test VM.
In addition there is a RemoteServiceAdmin implementation which is used internally to support communications with Admins on slave systems. There are no identifiers exposed for the admin.

Service Property Resolution

The values of some service properties may have a type dependency. A likely example is the implementation class, which differs for activatable and non-activatable implementations. Service properties may be qualified by appending the type value. For instance, for the hypothetical Jini service net.jini.FooInterface, consider the following definitions:

   net.jini.FooInterface.impl.activatable=org.apache.river.foo.FooActImpl
   net.jini.FooInterface.impl.transient=org.apache.river.foo.FooNonActImpl
   net.jini.FooInterface.impl.persistent=org.apache.river.foo.FooNonActImpl
   net.jini.FooInterface.codebase=${org.apache.river.jsk.home}/lib-dl/foo-dl.jar
If the value of net.jini.FooInterface.type is "activatable" then all attempts to resolve any service property value for net.jini.FooInterface will first search for the ".activatable" variant. If not found, then the unqualified variant will be searched for. For the example above, the search for impl would return org.apache.river.foo.FooActImpl, and the search for codebase would return ${org.apache.river.jsk.home}/lib-dl/foo-dl.jar. However, if net.jini.FooInterface.type is "persistent", then the search for impl would return org.apache.river.foo.FooNonActImpl, and the search for codebase would return ${org.apache.river.jsk.home}/lib-dl/foo-dl.jar. When overriding the definition of a service property, it is therefore important to override all variants for that property. Overriding only the unqualified definition will result in incorrect behavior if any variants are defined elsewhere.

The best way to understand the generation and use of service property names is through an example. Assume that the following call was made for the third time in the same test:

  manager.startService("net.jini.discovery.LookupDiscoveryService");
where manager is an instance of AdminManager. Since AdminManager manages the instance counter automatically, it will initially attempt to use a value of 2 when constructing property names. The startService method would initially look for the following set of configuration values (assuming an ActivatableServiceStarterAdmin is used):
  net.jini.discovery.LookupDiscoveryService.impl.2
  net.jini.discovery.LookupDiscoveryService.classpath.2
  ...
  ...
Any configuration values found by searching with these names would be used. If any of these names were not found, then the instance count would be decremented and a search using the unresolved names would be repeated:
  net.jini.discovery.LookupDiscoveryService.classpath.1
  net.jini.discovery.LookupDiscoveryService.log.1
  ...
  ...
This process is repeated until all names are resolved, or until the instance count reaches zero (a search with value zero is done if necessary). If any names are still unresolved, then the instance count is discarded and a final search is done:
  net.jini.discovery.LookupDiscoveryService.classpath
  net.jini.discovery.LookupDiscoveryService.codebase
  ...
  ...
This makes it easy to define a set of default values, and then to specify overrides on a per-instance basis.

The type of admin created to manage a service is based on the value of the services type property:

Value Admin Created
group SharedGroupAdmin
rmid ActivationSystemAdmin
phoenix ActivationSystemAdmin
classServer ClassServerAdmin
running RunningServiceAdmin
transient NonActivatableServiceStarterAdmin
persistent NonActivatableServiceStarterAdmin
activatable ActivatableServiceStarterAdmin

if type is undefined or has a value not represented in this table, a TestException is thrown. If a slave host is used for the service, then an admin of the specified type is created on the slave host, and a RemoteServiceAdmin is created in the master host to communicate with it. This behavior is transparent to tests.

Service ConfigurationFiles

The ConfigurationFiles for services must follow a few conventions to support the use of standard overrides provided by the harness and default proxy preparers expected by the harness. Each file should contain the following set of well-known entries:
exporter.name accessed by the harness to obtain the name of the service exporter
exporter.activatableExporter defines the exporter for the activatable service
exporter.transientExporter defines the exporter for the nonactivatable, nonpersistent service
exporter.persistentExporter defines the exporter for the nonactivatable, persistent service

An example ConfigurationFile for norm (imports omitted) might look like:

  exporter {
        name = "org.apache.river.norm.exporter";
        activatableExporter = new JrmpExporter((ActivationID) $data, 0);
        transientExorter = new JrmpExporter();
        persistentExporter = new JrmpExporter();
  }

  org.apache.river.norm {
   [standard norm entries, see the javadoc for org.apache.river.norm]
  }
In order to identify the exporter to use, the harness obtains the exporter.name entry and uses it to build an override which indirectly points to the exporter to use. For the activatable variant, the resulting override would be
   "org.apache.river.norm.exporter=exporter.activatableExporter"
When norm obtains the entry for org.apache.river.norm.exporter, it will access exporter.activatableExporter which returns the activatable exporter variant.

User-Specified Test Properties

The user must specify a deployment properties file on the command line used to run the harness. This file defines installation specific values and a small number of configurable parameters, such as timeouts. The prototype for this file is included with the harness at ${org.apache.river.qa.home}/src/org/apache/river/qa/resources/basicDeployment.prop. Properties defined in this file include:
org.apache.river.jsk.home the installation directory of the starter kit. This value is mandatory, and must be specified as an absolute path.
org.apache.river.qa.home the installation directory of the QA 'kit'. This value is mandatory, and must be specified as an absolute path.
org.apache.river.jsk.port the HTTP port for downloading starter kit classes. This parameter is optional, and has the default value 8080.
org.apache.river.qa.harness.dldir.8080 The download directory for the class server at port 8080. This is needed to support generating httpmd URLs.
org.apache.river.qa.port the HTTP port for downloading QA classes. This parameter is optional, and has the default value 8081.
org.apache.river.qa.harness.dldir.8081 The download directory for the class server at port 8081. This is needed to support generating httpmd URLs.
org.apache.river.qa.harness.shared if true (the default), all services are configured to run in a shared group VM. If false, each service runs in a private group.
org.apache.river.qa.harness.runactivation if true (the default), the activation system identified by the service name activationSystem is started when the first activatable service is started, and stopped during test teardown.
org.apache.river.qa.harness.runjiniserver if true (the default), the class server identified by the service name jiniClassServer is started automatically in test setup, and stopped in test teardown.
org.apache.river.qa.harness.runkitserver if true (the default) the class server identified by the service name qaClassServer is started automatically during test setup, and stopped during test teardown.
org.apache.river.qa.harness.timeout the number of seconds to elapse before a test timeout interrupt is fired. If undefined, or if the value is zero (or negative) then timeouts are not enabled. The default value is zero.
org.apache.river.qa.harness.discardOKOutput if true, most output generated by a passing test is discarded. If false, all output is written to the test log. The default value is false.
org.apache.river.qa.harness.rerunFailedTests if true, a failing test may be automatically rerun as determined by the testMaxRetries test property. The default value is false.
org.apache.river.qa.harness.configs a comma separated list of configurations to use when running tests. This list may contain any combination of jrmp, jeri, http, jsse, https, kerberos, or none. The default value is none.
Additional properties associated with kerberos configurations and distributed operations are defined and documented in qaHarness.prop.

Appendix A - Supporting Kerberos Configurations

The test harness includes configuration files that support testing with kerberos as the underlying security infrastructure. However, in order to run with kerberos some additional setup is required.

1. In order to run the Kerberos harness configuration, you need a Kerberos KDC (Key Distribution Center) available in your network.

2. You also need to create principals for reggie, mahalo, outrigger, mercury, norm, phoenix, an executing test (this is a single principal used for all tests), fiddler, an activation group (again a single principal used for all activation groups), a server principal, and a client principal. Refer to your KDC documentation for instructions on how to create principals for that particular KDC.

3. Once the principals needed by the harness have been created, generate an aggregate password file containing the passwords for all the principals created in step 2 and store this password file in a location that is accessible to the harness. Refer to your KDC documentation for instructions on how to create password files for your KDC.

4. Edit the following section in the test suite deployment properties file which was derived from ${org.apache.river.qa.home}/src/org/apache/river/qa/resources/basicDeployment.prop supplied in the harness source distribution:

#
# The following properties are used to specify the kerberos
# environment for tests that need to access a kdc and kerberos principals
#
org.apache.river.qa.harness.kerberos.realm=[add the realm that your KDC is serving]
org.apache.river.qa.harness.kerberos.kdc=[add the name of your KDC host]
org.apache.river.qa.harness.kerberos.clientPasswordFile=[add the location for the password file created in step 3]
org.apache.river.qa.harness.kerberos.serverPasswordFile=[add the location for the password file created in step 3]
org.apache.river.qa.harness.kerberos.aggregatePasswordFile=[add the location for the password file created in step 3]
org.apache.river.qa.harness.kerberos.clientPrincipal=[add the name of the client principal created in step 2]
org.apache.river.qa.harness.kerberos.serverPrincipal=[add the name of the client principal created in step 2]
org.apache.river.qa.harness.kerberos.reggiePrincipal=[add the name of the reggie principal created in step 2]
org.apache.river.qa.harness.kerberos.mahaloPrincipal=[add the name of the mahalo principal created in step 2]
org.apache.river.qa.harness.kerberos.outriggerPrincipal=[add the name of the outrigger principal created in step 2]
org.apache.river.qa.harness.kerberos.mercuryPrincipal=[add the name of the mercury principal created in step 2]
org.apache.river.qa.harness.kerberos.normPrincipal=[add the name of the norm principal created in step 2]
org.apache.river.qa.harness.kerberos.phoenixPrincipal=[add the name of the phoenix principal created in step 2]
org.apache.river.qa.harness.kerberos.testPrincipal=[add the name of the test principal created in step 2]
org.apache.river.qa.harness.kerberos.fiddlerPrincipal=[add the name of the fiddler principal created in step 2]
org.apache.river.qa.harness.kerberos.groupPrincipal=[add the name of the group principal created in step 2]