How to Run a QA Test

Introduction

This document explains in step by step fashion how to run Jini QA tests. A Jini QA test is a class that implements the org.apache.river.tck.harness.Test interface and is thus able to run inside of the TCK (Technology Compatibility Kit) test harness. In general, a Jini QA test will test either a service (such as Lookup Discovery Service) or a utility (such as Client Lookup Manager or Join Manager). For more information refer to the "How to Write a Test" document.

In order to run tests you will need access to an installation of TCK 1.0B as well as the QA Methodology Framework software. There is an installation of TCK 1.0B located in the installation directory /net/jini/files/web/qa/tck/jinitck1_0B. Alternatively, you may install the TCK software on your own drive and use that. You will need to start up rmid (if your tests require it) as the framework will not do this for you. The Runner (the TCK test harness) starts up a class server on port 8080 which serves up class files from the TCK's /lib directory under the installation directory. It is essential that any class server that you start use a port number other than 8080 otherwise, the harness will throw an exception. The QA Methodology Framework software is located in /vob/qa/jars/QATestUtil.jar. You may need to perform a gmake on the qa vob if you have not already done so. In order to successfully perform a build in the qa vob you must have previously performed successful builds in the jive and tck vobs.

1. Ensure that you have access to the required software as described above.
2. Start up rmid
3. Start up a class server to serve up the Jini jars (core, ext, etc...) in directory /vob/jive/lib on port 8085
4. Copy the contents of /vob/qa/src/org/apache/river/qa/harness/resource to some convenient directory.
   For the purposes of these instructions we will assume $HOME/qatest.
5. cd to $HOME/qatest if you are not already there.
6. Enter the command /vob/qa/src/org/apache/river/qa/harness/examples/run_sampleutilitytest

The qatest.prop file that you copied to the $HOME/qatest directory has a property called qatest.debug which is set to true. This causes additional output to be generated by the framework when starting and stopping services. This extra information is useful for debugging purposes. You can stop this output by either commenting out the assignment of the qatest.debug property or by setting the value of the qatest.debug property to false.

At the end of the output you should see the following text:

  Test Passed: Success -> Proxy joined lookups as expected.
  

7. Enter the command /vob/qa/src/org/apache/river/qa/harness/examples/run_sampleservicetest

At the end of the output you should see the following text:

  Test Passed: Success -> Transaction isolation is verified.
  

The run_qatest shell script is provided to help make running Jini QA test easier. The usage statement for the script follows:

  Usage run_qatest -tests -categories [-prop -policy -cp -jdk]
  

• -tests is the comma delimited list of tests to be run
• -categories is the comma delimited list of test categories
• -prop is the path of the propery file
• -policy is the path of the policy file
• -cp is extra classpath directories to append to default
• -jdk is the fully specified path to java.exe

  qatest -test org.apache.river.test.harness.examples.ServiceUtilityTest 
         -categories joinmanager
  

The following procedure assumes that the environment has been set up correctly and rmid and a class server are running. It also assumes that your test(s) are compiled and reside under /vob/qa/classes

1. Create a Properties file for your test(s). You may create one Properties file for each test or use one for all your tests. See /vob/qa/src/org/apache/river/qa/harness/examples/sampletest.prop for an example.
2. Edit the qatest.prop file in $HOME/qatest and add a line to the end that maps your Properties file to a specific test. You must add one such line for each test. qatest.prop shows examples using SampleUtilityTest and SampleServiceTest.
3. Edit the properties file and include argument information for the services that your test(s) will need to start. Also see /vob/qa/src/org/apache/river/qa/harness/examples/sampletest.prop for an example.
4. cd to $HOME/qatest if you are not already there.
5. Execute the command:

     run_qatest -tests org.apache.river.test.myarea.MyTest -categories myCategory 
                -cp /vob/qa/classes
     

   You may also define any parameters that you might put in a Properties file on the command line. For example:

     run_qatest -tests org.apache.river.test.myarea.MyTest -categories myCategory 
                -cp /vob/qa/classes -qautil.debug true
     

   Will turn on debugging information in the QA Methodology Framework code.