Best practices of writing tests
1) TESTS:
Tests are written under version they apply to. For e.g. all tests related to
v2 should be written under v2/test
The test folder is broken down into
a) src
b) cases
c) tools
Below are the uses for each of the folders.
a) Src:
This is where the actual test cases (in this case junit driver) live. The src
folder is divided by functional areas at the highest level.
Each functional area is further sub divided by smaller functional areas (if
required). If the division is not required then functional area is divided
by importance of the tests. Importance are of 2 types
a) Checkin
b) Detailed.
As the names suggest checkin tests are really important tests for a particular
area while Detailed tests aim is to test more of edge cases/error condition/
various kinds of input etc.
Not each top-level functional area needs to have checkin folder. If that area
is not that widely used in the product there might just be detailed folder.
Since checkin tests have to be run by all developer every time they checkin
the goal is to include the most crucial tests for each functional area of
the product.
Also each top level/sub functional area can have a Common folder. The main
goal here is group smaller utilities used to test functional area in one
place so that both the Checkin/Detailed tests can use. If you feel the tools
you are writing can be share across Cross-functional areas then use the Tools
folder (more to come later in the doc). Any class which is not itself a Junit
test should go into that folder, even if it is a superclass for a Junit test
in the checkin/detailed folder. Same goes for all interfaces.
Below is an e.g. of how one of the top-level functional area's directory
structure may look like.
xmlcursor/
|-checkin/
|-detailed/
|-common/
|-saver/
| |-detailed/
| |-common/
|-xpath/
| |-checkin/
| |-detailed/
| |-common/
In this e.g. the xmlcursor area is broken down by checkin/detailed tests for
smaller functionality that is cursor specific. For functionality in the
cursor that is bigger sub folders are created with its own checkin/detailed/
common areas. Again saver does not have a checkin folder since the tests in
this area are not too important.
Overall when writing a test please consider which areas they fit and only
create sub folder/top level folders only if necessary.
2) CASES:
This contains schema files/xml instances belonging to those schemas and schema
independent xml instances which can be shared within the functional areas.
Cases is further divided into top level areas matching src folder and all
schemas/xml instances used by functional areas are placed within this.
All schemas are compiled with scomp during the build process before compiling
the tests. If you have added a top level folder to src then the schemas & xml
instances for those tests should have their top-level folder within cases/xbean.
If these schemas need to be compiled by scomp you will need to edit the
testbuild.xml file to include the new folder in the build. Add this folder
to the variable "schema.dirs". (More details below)
If you are just adding schemas to existing folders within xbean/cases then
these will be automatically picked up in the build process.
If you have schemas that you don't want to pre compile in the build process
please rename the extension from .xsd to .xsddonotcompile. In this way scomp
will not pick up pre compile the schema.
The best practice for the cases folder is to try to reuse schemas within the
functional areas. So please check the schemas we have before adding more schemas.
Since currently the directory structure is such that process sharing of
schemas exists within top level functional areas (e.g. all tests with
xmlcursor can reuse the schemas) there is some amount of duplication since
schemas are not shared among top level functional areas. (for e.g. xmlcursor
& xmlobject will not have schemas shared among them). Though we recognize this
and are making attempts to revise the directory structure this has
implications on how the tests run so for the immediate future this might stay
as it is.
3) TOOLS:
The tools area is meant for storing tools/utilities which help running the
tests and reporting results. If you feel you have written a tool which could
be used across all functional areas please put the source within tools/src.
However before adding a tool please check the existing tools which we already
have.
4) BUILD FILE CHANGES:
Depending on how tests and cases are added, the build file (testbuild.xml)
will need to be updated.
a) New folder containing schemas added to cases:
Append this folder to the property schema.dirs
e.g:
Old value:
New value:
If your schemas are needed in order to run checkintests please update the
"checkin.schema.dirs" property as well.
b) New top-level area added to test (src/NEWAREA)
Add this folder to the list of areas for incremental compile
* Append the area name to the property area.names
* Add a new target with the name "build.area.NEWAREA"
Properties that need to be set for this target to work
* includes.for.compile - includes spec for javac
Optionally set the following properties
* this.schema.dirs - list of folders for scomp
* this.schema.standalone.dirs - list of individual schemas for scomp
* testcasesjar.update - set to true if testcases.jar should be
updated instead of overwriting
Add a call to target "unit.build" at the end of the target.
e.g: