Writing Traffic Server tests with DEFT

Test scripts

Test scripts are writting in perl though it should possible with some effort to use a different interpretor if necessary. The testing framework is core written in C++ and exports functions to perl for controling test programs. The log parser process are written in perl.

High level concepts

To effectively write tests, we need to understand the high level building blocks of the test framework. There are 5 processes run by the test framework.

Test framework processes

• test_exec - runs the test scripts. Starts up proc_managers on each node used in the test. Pushes test tool packages to proc_managers. Starts up test_log_collate and the log_viewer.pl
• proc_manager - starts up and manages tests processes. There is one proc_manager started on each machine being used in the test.
• test_log_collate - collates the test log from each of the proc_managers. This process is generally invisible to users of the system since it is automatically spawned by test_exec.
• log_viewer.pl - displays the test log in a window.
• parse_test_log.pl - processes the test log at the end of the tests and tallys up errors and warnings.

Test Process Management

Each process run under the test framework has different requirements for config files and start up. For example, Traffic Server has a whole set of configuration files that need to be setup up to. An instantiator script is used to manage test process setup. The instantiator setups up any necessary files and directories, allocates ports and tells the process manager what command line to use to start up the process. The instantiator can be sent test specific configuration through an argument pm_create_instance function. The configuration is treated as an opaque string that is passed to the instantiator.

Variable substiution

It's rather inconveient to hard code a test to specific machines and ports. To avoid hard coding, the test framework will make variable substitutions on various parameters. To have substitution made, use %%(varname). Substitutions are defined by .defs file read in at the beginning of the test, command line agrument to test_exec, and port bindings issued by instantiators. Note: currently is no escaping mechanism to say %%( and not have be a variable sustitution.

In order to make tests easily reusable, the following convention exists for substitutions for machine names:

• For traffic_server machines: ts1, ts2, ts3, ts4
• For load machines: load1, load2 ... load10

Most tests will only use ts1 and load1 to load 4. But more complex tests that either need to generate very high loads or test cache hierachies through parenting or icp may use the entire set.

The default defs file maps all hostnames to localhost. You can generate your own defs files and use '-D' option to test_exec (or run_test.pl which passes through its agruments through to test_exec).

The following characters are reserved and can not be used within a variable name: ':' and '/'.

RAF

RAF (Remote Access Facility) is a simple protocol TCP based protocol that can be used to both query and control test processes. Ideally, test tools are enhanced to include RAF support but RAF is not a requirement. Details on RAF are avaiable here .

An example test

This test test that starts up traffic_server and a jtest and issues an error if the ops per second are less than 5.

"use TestExec" loads the perl module that contains the the bindings to the underyling C++ functions of the testing framework.

The $ts_config var is set through a useful perl quoting syntax that makes everything quoted text until the marker is seen, in this case, EOC .

use TestExec;

$ts_config = <<EOC;
[records.config]

proxy.config.cache.storage_filename   storage.config
proxy.config.proxy_name		      example.com
proxy.config.hostdb.size              50000

[storage.config]

. 700000
EOC

@ts_create_args = ("package", "ts",
 	           "localpath", "%%(ts_localpath)",
		   "config", $ts_config);

@jtest_create_args = ("package", "jtest",
		      "config", "-P %%(ts1) -p %%(ts1:tsHttpPort) -c 5");

@jtest_start_args = ("args", "-C");
@empty_args = ();

# Fire up Traffic Server
$ts_inst_name = "ts1";
TestExec::pm_create_instance("ts1", "%%(ts1)", \@ts_create_args);
TestExec::pm_start_instance("ts1", \@empty_args);

sleep(5);


# Fire up jtest
TestExec::pm_create_instance("jtest1", "%%(load1)", \@jtest_create_args);
TestExec::pm_start_instance("jtest1", \@jtest_start_args);

sleep(5);

# Query jtest via RAF to get ops per second value
@raf_args1 = "/stats/client_sec";
@raf_result = TestExec::raf_instance("jtest1", "query", \@raf_args1);

# Must do at five ops per second
if ($raf_result[2] < 5) {
   TestExec::add_to_log("Error: insufficent jtest ops: $raf_result[2]"); 
} else {
   TestExec::add_to_log("Status: jtest ops: $raf_result[2]"); 
}


TestExec::pm_stop_instance("jtest1", \@empty_args);
TestExec::pm_destroy_instance("jtest1", \@empty_args);

TestExec::pm_stop_instance("ts1", \@empty_args);
TestExec::pm_destroy_instance("ts1", \@empty_args);

Function calls for managing test processes

Tools and products already running under DEFT

• Traffic Server
• Jtest
• Syntest

More on variable substitution

There are two additional topics for understanding variable substitution.

The first is how to substitute results from port bindings. To substitute a port binding, the following form is used:

%%(instance_name:port_binding)

The second topic is modifiers to variable substitution. The form for modifiers is:

%%(var_name/modifier)

A modifier is specified by a single character following the slash. Currently, the only modifier is /r which says the result of the substitution should be passed to gethostbyname() and the resulting ip address should be used.

What's next

The next step is learning how about test groups and result reporting options. Read Managing Tests to learn how.

Helpful Links

• Remote Access Facility (RAF)

Back to getting started

All rights Reserved.