How to Write a QA Test

Introduction

This document explains in step by step fashion how to write a Jini QA test. A Jini QA test is a class that implements the org.apache.river.tck.harness.Test interface and is thus able to run inside of the TCK (Technology Compatibility Kit) test harness. In general, a Jini QA test will test either a service (such as Lookup Discovery Service) or a utility (such as Client Lookup Manager or Join Manager).

It is recommended that each test be written to test one or a related set of assertions. These assertions will either be specification based or implementation based. In the former case the test is asserting something about conformance to a written Jini specification. In the latter case, the assertion relates to an implementation specific aspect of the entity under test. The requirements for writing these two types of tests differ slightly and that difference is explained later on in this document.

All QA Test directories reside under /vob/qa/src/org/apache/river/qa. A pictorial view of this structure follows:

                        /vob/org/apache/river/qa                                
                                 |
  -------------------------------------------------------------------------
  |      |         |         |      |                   |
  +      +         +         +      +                   +
docs  resource  examples  harness  spec                impl
                                    |                   |
                                    `- lookupservice    `- reggie
                                    |                   |
                                    `- javaspace        `- outrigger
                                    |                   |
                                    `- txnmanager       `- mahalo
                                    |                   |
                                    `- discoveryservice `- fiddler
                                    |                   |
                                    `- renewalservice   `- norm
                                    |                   |
                                    `- eventmailbox     `- mercury
                                    |                   |
                                    `- lookupmanager    `- clientlookupmanager
                                    |                   |
                                    `- renewalmanager   `- leaserenewalmanager
                                    |                   |
                                    `- joinmanager      `- joinmanager
                                    
  

If you are writing tests for a heretofore untested service or utility, then you should create the appropriate directories as necessary.

Make certain that you have the most recent build of the necessary code by executing the following from the command line :

1. cd /vob/tools/src
2. gmake clean
3. gmake
4. gmake jars
5. cd /vob/jive/src
6. gmake clean
7. gmake
8. gmake jars
9. cd /vob/tck/src
10. gmake clean
11. gmake
12. gmake jars
13. cd /vob/qa/src
14. gmake clean
15. gmake
16. gmake jars

Insure that whatever compile process you use includes the following in the CLASSPATH :

1. /vob/jive/lib/qa1.jar

1. Copy the /vob/qa/src/org/apache/river/qa/resource/QATestTemplate.txt file into the package directory of the test you are writing.
2. Rename QATestTemplate.txt to the name of the test class you are writing.
3. Bring the file up in your favorite editor.
4. Search for code buoys and fill in the information as requested.

A code buoy is a tag in the code that acts as a placeholder for information (code or comment) that needs to be supplied in the process of writing a Jini QA test. Each code buoy starts with the string "<XXX" and ends with the text "XXX>". What appears in between is a word or phrase indicating the nature of the required information. For example the place holder for the package name is expressed as <XXXpackage_nameXXX>. Each test requires the following information :

1. <XXXpackage_nameXXX>
2. <XXXclass_descriptionXXX>
3. <XXXauthorXXX>
4. <XXXclass_nameXXX>
5. <XXXdescription_of_the_test_goes_here_XXX>
6. <XXXcategory_in_lower_caseXXX>
7. <XXXa_generous_description_of_test_objectiveXXX>
8. <XXXsuccess_messageXXX>
5. Find the setup() method and include any required setup code.
6. Find the run() method and add code to test the required assertion(s).
7. Find the tearDown() method and add code to clean up test state.
8. Congratulations, you have just written a Jini QA test!

Test writers are encouraged to log all important actions in a test as an aid to diagnosing logic faults during automated test runs. Please be generous in logging important actions by using the log PrintWriter. The log PrintWriter is a public instance variable inherited from the parent QATest class.

All tests are derived from the org.apache.river.qa.harness.QATest class. This parent class provides a protected instance member named util which is an instance of the class org.apache.river.qa.harness.QATestUtil. QATestUtil provides common methods which all Jini QA tests should use as needed to achieve their objectives.

QATestUtil provides methods that can be categorized into three different areas. A summary of these methods are listed in the tables below. For more information see the QATestUtil java doc.

Capturing Test Parameters
boolean	getBooleanArgValue(java.lang.String keyword, boolean defaultValue)           Retrieve an boolean argument from command line or failing that, the the test's properties file.
double	getDoubleArgValue(java.lang.String keyword, double defaultValue)           Retrieve an double argument from command line or failing that, the the test's properties file.
java.io.File	getFileArgValue(java.lang.String keyword, java.io.File defaultValue)           Retrieve an File argument from command line or failing that, the the test's properties file.
float	getFloatArgValue(java.lang.String keyword, float defaultValue)           Retrieve an float argument from command line or failing that, the the test's properties file.
int	getIntArgValue(java.lang.String keyword, int defaultValue)           Retrieve an integer argument from command line or failing that, the the test's properties file.
long	getLongArgValue(java.lang.String keyword, long defaultValue)           Retrieve an long argument from command line or failing that, the the test's properties file.
java.lang.String	getStringArgValue(java.lang.String keyword, java.lang.String defaultValue)           Retrieve an String argument from command line or failing that, the the test's properties file.

Starting/Querying Services
net.jini.core.lookup.ServiceID	getServiceID(java.lang.Object serviceProxy)           Return the service id of the service that publishes the supplied proxy.
java.lang.Object	requestService(java.lang.Class interfaceClass)           Request a service via its interface class.
void	stopAllServices()           Stop all the services started up via the requestService() method
void	stopService(java.lang.Object serviceProxy)           Stop a service that has been obtained via the requestService() method.

Modifying Lookup Services
void	setLookupGroups(net.jini.core.lookup.ServiceRegistrar  serviceProxy, java.lang.String[] groups)           Change the set of groups to which a lookup belongs

File System Support
static java.io.File	createUniqueDirectory(java.lang.String prefix, java.lang.String suffix, java.lang.String path)           Creates a directory (folder) that is guaranteed to be unique on the file system and returns it as a java.io.File object.
static java.lang.String	createUniqueFileName(java.lang.String prefix, java.lang.String suffix, java.lang.String path)           Create a filename that is guaranteed to be unique on the file system.

The best way to see how the above utility methods are used is to look at the example SampleUtilityTest.java.

There are special test parameters that can be used to control various aspects of the testing system. For a detailed description these and other features of the QA Test Methodology see The Conformance Methodology Architecture Document.