= Test Suite Adaptors 1. Test Suite Adaptor is an Ant script acting as a glue between BT framework and Test Suite to be integrated. 2. The Adaptor implementatin should be placed in adaptor.xml file under ${root.dir}/adaptors/${suite.name} directory. 3. The adaptor.xml should implement 3 targets: - setup - should implement the all the functionality needed to prepare the test suite for actual run - run - should make the actual execution of the test suite - clean - should clean all the artifacts produced by test suite. = Test Suite Parameters o Common Notions and Use 1. All of the parameters needed to integrate, tune and launch the test suite are specified in parameters.xml placed near the adaptor.xml in ${root.dir}/adaptors/${suite.name}/ directory. 2. The parameters.xml has root element and several sections separating the parameters by their semantics. 3. Currently BT Framework supports the following sections for parameters: * used to describe the parameters 'required' to launch the test suite (example: jvm.under.test). * used to describe the external libraries on which the test suite depends. * used for sharing some values with other suites. * specifies SVN locations used by a suite. * used to describe some parameters used for default Cruise Control configuration (example: use SVN checks on the test suite or not, use trigger file for rerun or not) 4. Sections under TODO list: * to describe some 'optional' configuration parameters for this test suite. o Parameters Values resolution 1. The ways to resolve parameters specified in each section can differ and are specified for each section separately. 2. One common way for all of the sections is use extended name for the parameter which constructed according to Ant's task specification. For example, for parameters.xml for test suite named 'my-test-suite' described in the following form: ---------- adaptors/my-test-suite/parameters.xml ------- -------------------------------------------------------- the following set of properties is defined: my-test-suite.parameters.external.junit-3.8.1.md5=1f40fb782a4f2cf78f161d32670f7a3a my-test-suite.parameters.shared.build.dir=${sources.dir}/target my-test-suite.parameters.shared.sources.dir=${sources.dir} my-test-suite.parameters.required.req.param.description=Required For Something Important my-test-suite.parameters.external.junit-3.8.1.url=http://some.host.org/junit-3.8.1.jar my-test-suite.parameters.required.req.param=Some Parameter Value Such a keys are considered as in an extended form and can be used as a common way for resolving of the parameter values. o Required Parameters o Common Notions and Use 0. Required Parameters are those without which Test Suite won't run (such as: path to tested classlib, jvm and so on) 1. In case of unspecified value for some of the 'required' parameters, BT framework reports about it and doesn't launch the selected test suites 2. All of the required parameters for selected test suites are collected in ${root.dir}/required-parameters.properties file. This file is generated by framework on 'install' stage and should be used for specification of the values for unset required parameters. ----- Note: ${root.dir}/required-parameters.properties file should not be ----- updated manually. All unset required properties should be defined in framework.local.properties file. o Required Parameters Specification and Dereferencing 1. Required Parameters are specified by means of section of parameters.xml file as follows: - Required Parameter named as 'req.param' for test suite 'my-test-suite' should be added into the section in the following form: ---------- adaptors/my-test-suite/parameters.xml ------- ... ... ... ... -------------------------------------------------------- ----- Note: Without description your parameter is not considered as ----- required. 2. Inside the my-test-suite/adaptor.xml this parameter can be accessed by the following ways: - by using of its short name: ${req.param} - by using of its short name with 'my-test-suite' prefix: ${my-test-suite.req.param} - by using of extended parameter name: ${my-test-suite.parameters.required.req.param} 3. Inside the 'Required Parameters Specification File': ${root.dir}/required-parameters.properties this parameter is in extended form. o Default Values for Required Parameters 1. It's possible to provide default values for required parameters based on the shared parameters of other test suites. 2. The parameter holding the value for the required parameter should be the 'shared' value of other suite in extended form: ${some-other-test-suite.parameters.shared.param.key} 3. The form of the required parameter with default value is as follows: ---------- adaptors/my-test-suite/parameters.xml ------- ... ... ... ... -------------------------------------------------------- 4. If the 'some-other-test-suite' is not selected for run, the value for this parameter is unset and requested for specification by framework on 'install' stage. o Shared Parameters 1. Shared parameters are used for sharing of some values between the test suites. 2. Shared parameters are described in section of parameters.xml file. 3. The value specifying and resolutions is made as for required parameters. See [>o Required Parameters Specification and Dereferencing] section for this. o External Libraries Dependencies BT Infratructure offers the common way to fetch, use and share external libraries between the test suites. It is done by the following means: - every external dependency for the test suite is described in defined format in parameters.xml file - BT on 'install' stage fetches all the dependencies according to their descriptions and sets up corresponding properties to access them. - adaptors uses these properties to pass the information about fetched libraries to the test suites. 1. The dependencies on external librarties are described at = Test Execution Modes o Single Run Mode and Test Suite Dependencies 1. The base mode for test suite execution launched by 'run' target is a single run of each of the selected test suites. 2. The order in which the test suites are launched depends on the dependencies between the suites. Such a dependencies are specified in parameters.xml by 'depends' attribute of the root 'parameters' tag. The value of this attribute should be a coma separated list of all the suites on which this suite depends. 3. The test suite depending on other suites is launched for execution only if all of its superior test suites passed (adaptors have not reported failure). o Continuous Run Mode 1. BT framework supports continuous test execution. 2. This support is implemented on the base of the Cruise Control system: http://sourceforge.net/cruisecontrol o Default CC Configuration * By default CC run all if the selected test suites are executed as a separate CC Project with the following configuration: 1. Launching of the suite for execution is made by means of adaptor's run target. 2. If the set of dependencies for this suite is not empty, this suite monitors the build status of these suites is launched for execution in case of succesfull build of superior suites. 3. If there is a 'usesvn' attribute specified for element of parameters.xml file, the path contained as a value of this attribute is monitored for changes in a SVN repository. And if there is some update for this repository, this update is autimatically fetched and test suite is launched for execution. 4. If there is a 'trigger' attribute specified for element of parameters.xml file, the path to file contained as a value of this attribute is monitored for the modifications. And if there is a modification for this file, the test suite is launched for execution. o Custom CC Configuration * If there is a need to extend the functionality provided by default BT CC configuration, there are a means to provide custom configurations for each of selected test suites: 1. Custom CC configuration 'my-cc-config.xml' for suite 'my-test-suite' can be pluged in for use by BT Framework by specifying the following property: my-test-suite.parameters.cc.config=${path.to}/my-cc-config.xml in ${root.dir}/framework.local.properties configuration file. 2. There are some restrictions on content of the custom CC config file: - The name of the project SHOULD be equal to the suite name for which this config is used ('my-test-suite' in our case). To tell to BT Framework to automatically insert this name, the tag @BT.PROJECT@ should be used. - The section of the custom CC-config file MUST contain the following tag: This tag is resolved to the code performing the call to the adaptor run target to launch suite for execution. - Before using, this custom config file is copied by BT to another place and to another name. So the paths to the external files in custom CC-config file SHOULD be absolute names, or use global framework's properties (such as ${root.dir}, or ${cc.log.dir}) specified in scripts/framework.properties file. 3. There are several predefined tags substituted by BT framework. You can see and read about them in ${root.dir}/scripts/templates/cc-project-template.xml file. This file is used as a default CC-config template for test suites. I.e. the default CC behaviour for 'my-test-suite' is equivalent to this template specified as a custom CC config: my-test-suite.parameters.cc.config=${root.dir}/scripts/templates/cc-project-template.xml You can use this template as a starting point for your CC-config implementation.