Flood manualJacekPrucia2003-2004Apache Software FoundationForeword
This manual describes Flood. A software developed by Apache
Software Foundation.
Introduction to Flood
Flood is a profile-driven HTTP load tester. It is a software that is
capable of generating large amount of web traffic, so you can measure
performance of your web application. Flood can also postprocess web
server responses, so you also can check your web application for
correct behaviour.
Being profile-driven means, that nearly all actions related to creating
and performing a request, are controlled by a set of rules. Those rules
together create a profile. By using different profiles and the same set
of URLs, you might get quite different results. Moreover, particular
profiles are usefull in conjunction with different types of tests. As
far as web applications are concerned, we can distinguish three types
of tests. These are:
Performance test. This test measures response time of web server.
Desired result is usually a Requests Per Second for given resource
or average for all pages hit.
Regression test. This test doublechecks behaviour of web
application, by simulating several user actions (so called URL
paths). It is supposed to give answer to question: Is my web
application working correctly?
Web capacity test. This test performs numerous parallel requests,
simulating really heavy ussage of web application. It is supposed
to answer question: does my web application and web server handle
given load?
With a little bit of tweaking flood can be useful, regardless of which
type of test you are going to perform.
Flood aims to be modular and extensible. It is fairly easy to write
flood extension providing new functionality. Hence flood can be easily
extended to suit just about everybody needs. Right now it can be used
in most enviroments without the need for writting external modules.
Current flood feature list follows:
HTTP/1.0 and HTTP/1.1 supportHTTPS supportGET/POST/HEAD supportreponse postprocessingbasic support for AUTH and Cookiesdifferent report modules (different presentation of
results)
Flood has a homepage available:
http://httpd.apache.org/test/flood. Be sure to check it often
for news and releases. If you have a burning question, or would like
to report a problem (patches welcome), then please subscribe to flood
developemnt mailing list: test-dev@httpd.apache.org, by sending an
empty e-mail to adress:
test-dev-subscribe@httpd.apache.org
and following instructions in response.
Getting FloodThere are three ways in which you can get flood software.Source Tarball
This is the official way of obtaining flood source. Current flood
release can be found here:
http://www.apache.org/dist/httpd/flood/
Alternativelly, you can pick up a nearby mirror using this list:
http://www.apache.org/dyn/closer.cgi
Every official release schould be acompanied by everything you need
to build and use flood. That includes all external libraries
(except for SSL which is due to cryptography export restrictions)
and documentation.
CVS Repository
You can get flood source code directly from CVS repository. All you
have to do (besides getting CVS software itself) is to issue
following commands:
$ cvs -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic login
$ cvs -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic co httpd-test/flood
Note, that with CVS repository you're basically on your own with
meeting build dependencies. So you'll have to get apr libraries
as well:
$ cd httpd-test/flood
$ cvs -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic co apr
$ cvs -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic co apr-util
Please note that source code from CVS might not compile under
certain circumstances, so don't expect it to be as stable as
official releases. On the other hand, you schould try CVS
repository before reporting a problem. There is a huge chance,
that your bug is already fixed.
Flood binaries
Additionally, when flood developers will have some free time,
binary versions of flood official releases may show up on main
flood site:
http://www.apache.org/dist/httpd/flood/
If you find binary release that matches your platform, please
consult file README.binary located in the
root directory of binary packages, for information how flood was
compiled for your platform, and what features are available.
Compiling and installing
There are two scenarios for compillation and installation of flood.
Unix
Flood is build around apr and apr-util, so most Unix platforms
will work out of the box. Currently flood is developed on Linux
and Solaris, but FreeBSD is also known to work.
Before compilation we have to configure source tree for your
particular platform and personal requirements. You can do that
using configure script which is located in the
root directory of the flood source distribution. If you have
obtained flood sources from CVS, then you'll need to run
buildconf first (note that this requires recent
versions of autoconf and
libtool installed).
configure script recognizes following options:
Specifies target directory, which will contain all
installed files. Defaults to
/usr/local/flood.
This switch will cause flood to be built with SSL support,
which is disabled by default because of cryptography
export restrictions.
Path to directory containing OpenSSL installation.
Path to a directory with c_rehash'd CA files used by
OpenSSL. Defaults to
/certs.
Prefix for installed APR, path to APR build tree, or the
full path to apr-config.
Prefix for installed APU, or path to APU build tree.
A typical example of relevant command line:
./configure --enable-ssl
When flood source tree is configured you can start compilation by
typing make at command prompt:
make
When compilation finishes, you'll have to perform last step
-- installation. This is done by typing make install at command
prompt:
make install
Please note, that you will need write permissions for install
directory. This may require you to obtain root privileges.
If you have followed those instructions carefully, but still
couldn't get flood sources to compile, then please send e-mail to
test-dev@httpd.apache.org mailing list, with
description of your problem.
Windows
No information available at this time.
Running Flood
Flood is a command line software. You can run it from Unix shell,
MS-DOS prompt or simmilar facility of your operating system. Flood
accepts only one argument, and that is a path to configuration file.
Example:
$ flood /home/jacekp/tests/simple-test.xml
If no argument is given, then flood uses standard input stream
(stdin), so you can use flood in pipe processing:
$ get_urls.py /usr/local/apache/logs/access.log | flood
Flood outputs results to standard output stream
(stdout) and errors (if any) to standard error
stream (stderr). You can save result to file, or
pipe it to flood processing script, like this:
$ flood /home/jacekp/tests/simple-test.xml > /home/jacekp/tests/simple-test.out
$ flood /home/jacekp/tests/simple-test.xml | analize-relative
If flood is used in shell scripts, then you can test $?
variable for flood return code. If the code is 0, then flood has
performed a successful test. Other value (usually greater than 0)
means, that there was error during test.
There's ongoing development of GUI application for flood. When usable,
such application will make working with flood much easier.
Flood Configuration FileThis sections discusses flood configuration file.About File format
Flood XML parsing is built around apr-util XML capabilities, which
in turn are based on tweaked version of James Clark's marvelous
expat library. Because of that flood understands XML v1.0, so you
schould start your configuration file with following processing
instruction (PI for short):
<?xml version="1.0"?>
It is not required by flood, but it's good practice and may be
useful if you decide to use other XML tools for additional
processing. Please note, that expat support for different
encodings is rather limited. In particular it understands only
UTF-8 and ISO-8859-1 encodings, so processing of your
configuration may fail if you specify different encoding in your
configuration file, like this:
<?xml version="1.0" encoding="iso-8859-2"?>
Please note, that you can get around that limitation, by converting
you national characters to UTF-8. Flood itself only checks XML file
for well-formdess, which means checking if it is formed after basic
XML rules. If you want to do full validation of flood configuration
file, you may want to include following line right next to XML
processing instruction:
<!DOCTYPE configuration SYSTEM "flood.dtd">
With this declaration you can use software with validating XML
parser (like ASF's Xalan) and validate your configuration before
actually processing it with flood. That way you can ensure, that
flood.xml file is valid and obeys flood configuration rules. You
can find flood.dtd in
examples/directory relative
to package root directory.
Please note that all configure elements in flood aren't using
their own dedicated XML namespace, so if you (for some reason)
intend to mix flood configuration file with other XML, you may
experience tag name collision.
XML Survival Guide
Preparing flood configuration file can be a little troublesome.
At the time of writting this manual there are no external tools,
that could aid in this process (although there are plans to
develop those -- see next section). So you have to pick up editor
that has some sort of XML support. For various Unix systems
vi and
emacs seems to be reasonable choices.
You'll get colored syntax, possibly run-time (that is -- as you
type) DTD checking and maybe other switches to help you work with
XML files.
There are characters that are used by XML markup itself, and you
need a bit of magic when you want to use them as plain text.
Everytime you need such a character, you'll have to reference it
by an XML entity. XML entities start with ampersand (character
'&') and end with semicolon (character ';'). Text between
those character must refer to valid XML entity. Below is a table
that lists all 'reserved' XML characters and their coresponding
XML entities.
character namecharacter glyphentityampersand&&less than<<greater than>>apostrophe''quotation mark""
If you intend to use international characters in your flood
configuration file, and you don't have Unicode enabled editor,
you'll have to encode those characters. XML has special entity
for including just about any UTF-8 character. Basically it is
&#XXX; for decimal and &#xXXX for hex where XXX is a
Unicode character position. For example, polish letter "small
letter a with ogonek" can be referenced as ą (decimal)
or ą (hex) while german umlaut (latin small letter u
with diaeresis) schould be entered as ü (decimal) or
ü (hex). If you intend to use a lot of international
characters, you'd better get Unicode enabled text editor, since
number of XML entities will make your flood configuration file
less readable (not to mention typing hassle).
Mass URLs retrieval
Basic configuration examples are pretty easy to write, mostly
because they contain as little as 5, maybe 10 URLs. When you intend
to do serious tests of a large site or web application, you may be
unwilling to hand-type, say 600 URLs by hand. Flood is going to
help you with several applications under development:
mozilla applicationsmall proxyapache modulelog file analizerFlood Configuration Elements
This section describes flood configuration elements an their meaning.
All elements are presented here as a flat list, so please start
browsing with root element (which is:
<flood>) to get idea what elements
are available.
floodfloodroot elementsynopsis<flood> ... </flood>parent elementsnone (this is the root element).children elements
<urllist>+
<profile>+
<farmer>+
<farm>+
[ <seed> ]
attributesnametypedescriptiondefault valueconfigversionINTEGER
This attribute specifies config file version. It
is used by various flood config file generators
(mostly in development right now), to specify
which flood versions will be suitable for
performing certain test. Flood will output a
warning if config file and flood configversions
doesn't match.
0character dataignored.description
This is the top level element for flood configuration file.
It schould contain at least: one urllist, one profile, one
farmer and one farm.
examplesbellow is a smallest usable flood configuration file
<?xml version="1.0"?>
<flood>
<urllist>
<name>single url</name>
<url>http://www.example.com/</url>
</urllist>
<profile>
<name>simple profile</name>
<useurllist>single url</useurllist>
<profiletype>round_robin</profiletype>
<report>relative_times</report>
<verify_resp>verify_200</verify_resp>
</profile>
<farmer>
<name>john</name>
<useprofile>simple profile</useprofile>
</farmer>
<farm>
<!-- please note that right now faram *MUST* be called "Bingo" -->
<name>Bingo</name>
<usefarmer>john</usefarmer>
</farm>
</flood>
urllisturllistgroups several url adressesssynopsis<urllist> ... </urllist>parent elements
<flood>
children elements
<name>
[ <description> ]
[ <baseurl> ]
(<url> |
<sequence>)+
attributesnone.character dataignored.description
This element is a container for <url> elements, which
are basic data unit of flood. Every flood configuration file
must have at least one urllist element. Every urllist must be
asociated with name (using <name> children element) and
optionally with description.
When combined with proper profile type, this element allows a
creation of url paths. That means a sequence of url addresses,
that simulate a real human behaviour, such as: logon to
service, retrieve random (or specified) article, logoff.
examples
<urllist>
<name>single url</name>
<url>http://www.example.com/</url>
</urllist>
namenamenames object referenced by parent elementsynopsis<name>STRING</name>parent elements
( <urllist> |
<profile> |
<farmer> |
<farm> )
children elementsnone.attributesnone.character dataspecifies name of object referenced by parent element.description
This element sets name of object identified by parent element.
Most of the time, this element is mandatory, as it allows to
reference objects by their names.
examples
<urllist>
<name>single url</name>
<!-- ... -->
</urllist>
descriptiondescriptiondescription of object identified by parent elementsynopsis<description>STRING</description>parent elements
( <urllist> |
<profile> |
<farmer> |
<farm> )
children elementsnone.attributesnone.character datadescription of object identified by parent elementdescription
This element allows asociation of description with object
identified by parent element. It is usually optional and is
useful only in conjunction with GUI frontend. It isn't used
by flood at all.
examples
<urllist>
<name>my urllist</name>
<description>a bunch of random pages</desccription>
<!-- ... -->
</urllist>
baseurlbaseurlbase URL for all url elementssynopsis<baseurl>STRING</baseurl>parent elements
<urllist>
children elementsnone.attributesnone.character database url.description
This element sets base URL for all url elements in current
urllist. Every URL is created by concatenating base url and
url itself. This allows to abstract urllist and move it around
easy.
examples
<urllist>
<!-- ... -->
<baseurl>http://www.example.com</baseurl>
<url>/foo.html</url>
<url>/bar.html</url>
</urllist>
urlurlspecifies URL address for retrievalsynopsis
<url
[ method="{ GET | POST | HEAD }" ]
[ payload="STRING" ]
[ payloadparam="STRING" ]
[ payloadparamcount="INTEGER" ]
[ payloadtemplate="STRING" ]
[ responsename="STRING" ]
[ responsetemplate="STRING" ]
[ responsescript="STRING" ]
[ requesttemplate="STRING" ]
[ requestparamcount="INTEGER" ]
[ predelay="INTEGER" ]
[ predelayprecision="INTEGER" ]
[ postdelay="INTEGER" ]
[ postdelayprecision="INTEGER" ]
[ user="STRING" ]
[ password="STRING" ]
[ Content-Type="STRING" ]> [ STRING</url> ]
parent elements
<urllist>
children elementsnone.attributesnametypedescriptiondefault valuemethodSTRING
This attribute specifies HTTP method used in
request. Valid values include: GET, POST, HEAD.
GETpayloadSTRING
This attribute specifies POST request body.
Contents of this attribute are sent raw as POST
request body, so they need to be properly encoded.
(empty)payloadparamSTRING
No information available at this time
(empty)payloadparamcountINTEGER
No information available at this time
0payloadtemplateSTRING
This attribute holds POST request body, that will
be searched for variables. Every variable found
will be expanded to it's value.
(empty)responsenameSTRING
This attribute sets the name of variable, that
will be created upon successful match of
responsetemplate.
(empty)responsetemplateSTRING
This attribute holds regular expression that will
be matched against response. If successful,
resulting string will be placed in newly created
variable, whose name is determined by responsename
attribute.
(empty)responsescriptSTRING
This attribute holds path to executable script,
that will be used in response postprocessing. Such
script schould read standard input, which will
contain server response (both HTTP headers and
response body). Script exit value of 0 means
success, while any other value means error.
(empty)requesttemplateSTRING
This attribute holds the URL address, that will be
searched for variables. Every variable found will
be expanded to it's value. Presence of this
attribute makes url an empty element, becasue
content of this attribute is used as request URL.
(empty)requestparamcountINTEGER
No information available at this time
0predelayINTEGER
No information available at this time
0predelayprecisionINTEGER
No information available at this time
0postdelayINTEGER
No information available at this time
0postdelayprecisionINTEGER
No information available at this time
0userSTRING
This attribute holds username for auth (401
header).
(empty)passwordSTRING
This attribute holds password for auth (401
header).
(empty)Content-TypeSTRING
This attribute (when present) overrides default
Content-Type header which is 'application/x-www-form-urlencoded'.
(empty)character datanone | specifies URL address for retrieval.description
This element is the basic data unit of flood. It contains
detailed description of url address scheduled for retrieval.
You can specify how to perform request (method, optional
payload) and how to postprocess response.
When using responsescript attribute, please keep in mind, that
the execution time of script might distort overall results. In
addition, please add following element to your profile:
<recv_resp>generic_fullresp_recv_resp</recv_resp>
This wil cause flood to fetch whole response every time.
Without such element flood could sometimes retrieve only
first chunk of data, which just isn't enough for valid
postprocessing.
examples
<urllist>
<name>check release</name>
<url responsename="filename" responsetemplate="href=(.*)">http://www.example.com/downloads/</url>
<url requesttemplate="http://www.example.com/files/${filename}" requestparamcount="1"/>
<!-- ... -->
</urllist>
sequencesequencegenerates sequence of urlssynopsis
<sequence
sequencename="STRING"
sequencelist="INTEGER"> ... </sequence>
parent elements
<urllist>
children elements
<url>*
<sequence>*
attributesnametypedescriptiondefault valuesequencenameSTRING
This attribute specifies the name of current
sequence.
(empty)sequencelistSTRING
This attribute holds coma separated list of
sequence iteration values.
(empty)character dataignored.description
This element allows a creation of simple sequence of URLs.
This allows certain URL patterns to be simplified.
examples
<urllist>
<sequence sequencename="myseq" sequencelist="bob, jane, joe">
<url requesttemplate="http://www.you.com/~${myseq}/" />
<url requesttemplate="http://www.you.com/~${myseq}/index.html" />
</sequence>
</urllist>
profileprofiledescribes farmer profilesynopsis<profile> ... </profile>parent elements
<flood>
children elements
<name>
[ <description> ]
<useurllist>
<profiletype>
[ <socket> ]
<verify_resp>
[ <report> ]
attributesnone.character dataignored.description
This element describes profile of a farmer. Profile describes
farmer behaviour in detail. What farmer does and how he does
it.
examples
<profile>
<name>my profile</name>
<profiletype>round_robin</profiletype>
<useurllist>my urls</useurllist>
<verify_resp>verify_200</verify_resp>
<report>simple</report>
</profile>
useurllistuseurllisturllist to use in profilesynopsis<useurllist>STRING</useurllist>parent elements
<profile>
children elementsnone.attributesnone.character dataname of urllist to use.description
This element references an exisiting urllist, which will be
used in current profile. If there's no urllist by that name,
flood will report error and exit.
examples
<urllist>
<name>my urls</name>
<-- ... -->
</urllist>
<-- ... -->
<profile>
<-- ... -->
<useurllist>my urls</useurllist>
<-- ... -->
</profile>
profiletypeprofiletypespecifies type of profilesynopsis<profiletype>STRING</profiletype>parent elements<profile>children elementsnone.attributesnone.character dataname of profile type.description
This element specifies type of profile. Actually there is only
one valid profile type -- round_robin.
examples
<profile>
<-- ... -->
<profiletype>round_robin</profiletype>
<-- ... -->
</profile>
socketsocketspecifies profile socket typesynopsis<socket>STRING</socket>parent elements
<profile>
children elementsnone.attributesnone.character dataprofile socket type.description
This element specifies socket type to be used in conjunction
with current profile. Valid examples are
generic, keepalive and
ssl. In practice generic socket
allows for one request per connection (connection is
terminated right after receiving response), while
keepalive allows for more than one request to
be transmitted over active connection. This difference can
have significant impact on test results.
examples
<profile>
<-- ... -->
<socket>generic</socket>
<-- ... -->
</profile>
verify_respverify_respspecifies response verification typesynopsis<verify_resp>STRING</verify_resp>parent elements
<profile>
children elementsnone.attributesnone.character dataresponse verification type.description
This element specifies type of response verification. Valid
examples are verify_200 and
verify_http_code. Both types assume 3xx and 2xx
responses are valid, and others are errors. In practice,
verify_200 is very fast, while
verify_http_code is safe with carefull
response parsing. Please use verify_http_code
if you experience problems.
Flood recognizes 3xx type responses (so called redirects) as
valid responses, but do not follows them. Following 3xx type
responses will probably become optional in near future.
examples
<profile>
<-- ... -->
<verify_resp>verify_200</verify_resp>
<-- ... -->
</profile>
reportreportspecifies reporting typesynopsis<report>STRING</report>parent elements
<profile>
children elementsnone.attributesnone.character datareport type.description
This element specifies report type to be used in conjunction
with current profile. Valid examples are simple,
easy and relative_times.
examples
<profile>
<-- ... -->
<socket>generic</socket>
<-- ... -->
</profile>
farmerfarmerdefines a virtual working humansynopsis<farmer> ... </farmer>parent elements
<flood>
children elements
<name>
[ <description> ]
[ ( <count> |
<time> ) ]
<useprofile>+
attributesnone.character dataignored.description
This element defines basic flood work unit. A farmer can be
(depending on compile-time configuration) a separate process,
or a threead withing main flood process.
examples
<farmer>
<name>john</name>
<useprofile>my profile</useprofile>
</farmer>
useprofileuseprofileprofile to be used by farmersynopsis<useprofile>STRING</useprofile>parent elements
<farmer>
children elementsnone.attributesnone.character dataname of profile to be used by farmer.description
This element references an exisiting profile, which will be
used by current farmer. If there's no profile by that name,
flood will report error and exit.
examples
<profile>
<name>my profile</name>
<!-- ... -->
</profile>
<!-- ... -->
<farmer>
<name>john</name>
<useprofile>my profile</useprofile>
</farmer>
countcountamount of farmer work loopssynopsis<count>INTEGER<count>parent elements
<farmer>
children elementsnone.attributesnone.character dataamount of full work loops.description
This element specifies amount of loops that schould be
performed by farmer. The amount of work in loop depends on
profile, but it usually means hitting all url's from urllist.
examples
<farmer>
<name>john</name>
<count>5</count>
<useprofile>my profile</useprofile>
</farmer>
timetimeduration of farmer worksynopsis<time>INTEGER<time>parent elements
<farmer>
children elementsnone.attributesnone.character dataduration of farmer work in seconds.description
This element specifies duration of farmer work in seconds. If
specified time is greater than time needed to perform full work
loop, then farmer starts another work loop.
examples
<farmer>
<name>john</name>
<time>20</time>
<useprofile>my profile</useprofile>
</farmer>
farmfarma group of farmerssynopsis<farm> ... </farm>parent elements
<flood>
children elements
<name>
[ <description> ]
<usefarmer>+
attributesnone.character dataignored.description
This element defines a unit, that is a container for flood
basic work unit -- farmer. With farm you can easily clone
farmers and therefore generate massive loads.
Currently flood *requires* only one farm with name "Bingo" (case sensitive).
examples
<farm>
<name>Bingo</name>
<usefarmer>john</usefarmer>
</farm>
usefarmerusefarmerspecifies farmer to use in farmsynopsis
<usefarmer
[ count="INTEGER" ]
[ startcount="INTEGER" ]
[ startdelay="INTEGER" ]>STRING</usefarmer>
parent elements
<farm>
children elementsnone.attributesnametypedescriptiondefault valuecountINTEGER
This attribute specifies amount of farmers to
"clone".
0startcountINTEGER
This attribute specifies amount of farmers to
start each time.
0startdelayINTEGER
This attribute specifies amount of time (in
seconds) between each farmer start.
0character dataname of farmer to usedescription
This element asociates farmer with farm. In addition, it is
possible to "clone" farmer -- that is -- to have several
separate parallel farmers with identical behaviour (profile,
urls).
examples
<farm>
<name>Bingo</name>
<usefarmer count="2">john</usefarmer>
</farm>
seedseedspecifies fixed seed for PRNGsynopsis<seed>INTEGER</seed>parent elements
<flood>
children elementsnone.attributesnone.character datafixed seed for PRNG.description
During tests flood sometimes uses system PRNG (Pseudo Random
Number Generator). For example, this happens when you use
${=name} construct. Since numbers returned by
PRNG are different for every test, you can have situation in
which the same set of URLS has very different results.
Consider retrieval of article from database with such url:
<url requesttemplate="http://www.example.com/show_art.php?id=${=aid}"/>
When you run the test for the first time, aid
evaluates to 6, and results in a 20 KB article. However at the
second run aid evaluates to 216 which can
trigger a more complex query, or return a larger article (say
108 KB). You used the same config, but results are different.
If you use <seed> element with a fixed value (like 23,
48 or 84) you can expect PRNG to generate the same number with
every run.
examples
<seed>23</seed>
Analyzing Flood outputNo information available at this time.