Java Olio is a macro-level benchmark consisting of the following components:
The web application
The main database
Distributed storage servers (NFS or MogileFS)
Storage metadata database (for MogileFS)
If your primary interest is in setting up the application alone, you need items 1-3 above and they can all be setup on a single system. If on the other hand, you would like to drive load against the application, you will need at least 2 systems. At higher loads, you may need multiple systems. At a minimum, we need to separate the SUT (System Under Test) components and the non-SUT components to get valid results. The non-SUT components are the Geocoder emulator and the workload driver. It is best to connect the driver machine to the SUT machine on a local private network. This ensures that latencies measured do not include arbitrary delays.
For a horizontally scaled workload, or to measure the
performance of the individual components, you can deploy the SUT
components on separate physical or virtual machines. Keep in
mind though that the
bulk of cpu is spent in the application server component.
In the following sections, we'll go over the steps needed to configure each component :
Setting up the Driver
Setting up the Database
Loading the Database
Setting up Caching
Setting up the Geocoder Emulator
Setting up the
Building the Web Application
Testing the Web Application
Starting a benchmark run
# ls olio-java-0.2*
LICENSE OlioDriver.jar RELEASE-NOTES-java-0.2.txt geocoder.war java_setup_kit.html
NOTICE webapp.war release
RELEASE-NOTES-java-0.2.txt contains the release notes for this release.
See http://faban.sunsource.net/docs/guide/harness/install.html for Faban installation instructions. Note that faban needs to be installed on all the machines used for the benchmark. Please also read the Getting Started Guide to get a high-level understanding of Faban terminology and how it works. From now on, we will refer to the faban install directory as $FABAN_HOME.
cp samples/services/GlassfishService/build/GlassfishService.jar services (if using Glassfish)
cp samples/services/MysqlService/build/MysqlService.jar services
cp samles/services/MemcachedService/build/MemcachedService.jar services (if using Memcached)
Test that you can connect to the master by pointing your browser at http://<driver_machine>:9980. You should see the Faban welcome page or if you have already deployed the driver, a page with the text "Welcome to Java Olio Workload"
Download MySQL Connector/J (JDBC Driver for MySQL) and install the jar in $FABAN_HOME/benchmarks/OlioDriver/lib.
Setup the mysql user/group and permissions for its
# groupadd mysql
# useradd -d $MYSQL_HOME -g mysql -s /usr/bin/bash mysql
# chown -R mysql:mysql $MYSQL_HOME
Create the database :
# su - mysql
$ cd bin
Start the mysql server. Substitute your own password for pwd
typically use adminadmin)
$ ./mysqld_safe --defaults-file=/etc/my.cnf &
$ ./mysqladmin -u root password <your-password>
$ ./mysqladmin -u root -h <machine-name> password <your-password>
Create the olio user and grant privileges :
$ ./mysql -uroot -ppwd
mysql> create user 'olio'@'%' identified by 'olio';
mysql> grant all privileges on *.* to 'olio'@'%' identified by 'olio' with grant option;
In some cases the wildcard '%' does not work reliably as a substitution for all hosts. You need to grant the privileges to 'olio'@'<hostname>' individually, where hostname are the names of the driver and apache systems.
Eg. mysql> grant all privileges on *.* to 'olio'@'localhost' identified by 'olio' with grant option;
It is best to load the database manually the first time so that we can test the web application. However, while doing benchmark runs, the benchmark driver can be configured to automatically re-load the database before the run.
Login to the machine running the Faban master driver. Only this machine has the loader at this time.
The Geocoder Emulator is a simple Java EE application that you can deploy on Glassfish or Tomcat. It is typically run on a driver machine. The following steps describe how to install it.
Download and install Tomcat (http://tomcat.apache.org) on the driver machine. The install directory doesn't matter – we will refer to it as $TOMCAT_HOME.
Start Tomcat using $TOMCAT_HOME/bin/startup.sh.
Install Glassfish application server V2.1.1 or GlassFish v3 from
The directory where Glassfish has been installed will now be referred to as $GLASSFISH_HOME.
Download the JDBC driver for mysql from http://dev.mysql.com/downloads/connector/j/5.1.html. Extract the mysql-connector-java-5.1.X-bin.jar and drop this is $GLASSFISH_HOME/domains/domain1/lib/ext directory (or whatever you have named your domain).
Create JDBC connection pool and resource. Please
for details. MM MySQL Type 4 Driver (Non-XA)
The JAR file for the MySQL driver is mysql-connector-java-version-bin.jar, for example, mysql-connector-java-5.1.6-bin.jar. Configure the connection pool using the following settings:
Ensure that you have the JVM option "-server" and the heap size has been adequately sized. For benchmarking purposes, we have sized Xmx and Xms to 3G.
Start Glassfish. Check that you can connect to it from your
browser (http://host:8080), but don't try to access
any of the application pages yet ! To do
MapCache. This is the default and doesn't require any additional setting. It does a good job of off-loading the database. It is activated by the
-DmemcachedInstances=<host>:<port>where <host>:<port> point to the memcached instance. <port> is 11211 if you use the default port when starting memcached.
Start the memcached
server thus :
# nohup /opt/coolstack/bin/memcached -m 256 -u mysql &
Olio for Java EE can be configured to use either a local filesystem (typically NFS-mounted off a file server) or MogileFS for the object data. Our initial testing with MogileFS found some severe performance issues, so for now we advice using a normal filesystem. You will need about 50GB of space for the data, as the data does grow over runs. Using a single spindle does work but may create performance bottlenecks. We recommend striping the filesystem across at least 3 spindles to avoid such bottlenecks. A local file system needs to be setup on the same machine as the web application. A network file system can reside on a separate server but needs to be exported and mounted on the system running the web application.
Create a directory (or mount a
filesystem) designated for storing the image and binary files. This
directory is referred to as <filestore>. Any valid name
for the OS should be fine. Ensure that everyone has read and write
# mkdir -p <filestore>
# chmod a+rwx <filestore>
Now load the filestore.
You should have the $FABAN_HOME/benchmarks/OlioDriver.jar
file on this system.
# cd $FABAN_HOME/benchmarks; mkdir olio
# cd olio; jar xvf ../OlioDriver.jar; chmod a+x bin/*
# $FABAN_HOME/benchmarks/olio/bin/fileloader.sh <load_scale> $FILESTORE
This loads files for use for up to load_scale number of active users.
Eg. To load the filestore for 1000 users, you would do the following:
# cd /filestore/artifacts
# /opt/faban/benchmarks/OlioDriver/bin/fileloader.sh 1000 .
This loads files for use for up to load_scale number of concurrent users.
Check the home page (HomePage) http://<web_server>:8080/webapp/index.jsp. If there are no error messages and all images get loaded, that's a great start !
Click on an event (EventDetail). Make sure the whole page looks OK.
Click on an attendee (PersonDetail) to see a person's profile.
Go back to the home page and click on a tag in the tag cloud. Choose a big tag and check that we have good results and images get loaded OK.
Click on the Register button on the top left of
the home page. Fill
in the form and create a user. Make sure you find some jpeg images to
upload. If not, take them from
Submit the form. Make sure the form goes through. This completes the AddPerson transaction.
Login using your new login name you just created. The top right of the screen should show that you're logged on.
Select an event, go back to
the EventDetail page but this time as a logged on user.
Add yourself as an attendee. This is the EventDetail transaction with attendee added (about 8-9% of all EventDetail views).
Click on the add event tab and
add an event. Make sure to include an image and some literature. You
can also use the files from
$FABAN_HOME/benchmarks/OlioDriver/resources. Fill in the form and submit. This is the AddEvent transaction.
Now that we know that the web application is running and the faban harness is up, it is time to kick off a benchmark run.
Point your browser at http://<driver_machine>:9980
Click on the Schedule Run link.
Under the JAVA tab, set the JAVA_HOME. You can accept the default value for JVM options. DO NOT click on the OK button yet !
Select the Driver tab.
Enter a Description for the run (say 'First test run' for this case). In general, the Description field is very useful to get a quick idea of what a particular run is testing.
Enter the name of your driver(s) machine for Host (when using more than one machine, simply separate them by a space).
Enter 10 for 'Concurrent Users' (we want to start small).
Enter 'vmstat 10' for Tools. This indicates the measurement tools that will be run on the driver machine. It's a good idea to keep an eye on the driver cpu utilization.
Now enter 30, 30, 10 for the Ramp up, Steady State and Ramp down times. This is a very short test run. For normal runs, you may need a ramp up of 200 seconds and a steady state of at least 600 seconds during which measurements are made.
For current systems,
the time between client startup of 200 milliseconds is good enough.
Some servers or slower systems may not be able to accept
connections very frequently. In that case we may want to increase this
value to 1000 milliseconds.
The number of Agents needs to be set based on the number of concurrent
users - we start with 1 and add an agent for every 500 users.
Select the Web Server tab.
The Host:Port Pairs field takes the host port pairs where the web applications are running. The host and port is separated by a colon. Each pair is separated by space. Note that you cannot use the string "localhost" - you must specify a valid hostname or IP address.
In the tools box, enter the tools you want to run. Here are the tools we typically run : vmstat 10; mpstat 10; nicstat 10; iostat -x 10.
For Application Server, select "GlassFish" or leave the field blank if you're using some other server. Only the GlassFish server is supported at this time - for other servers, you will have to manually start/stop the servers before and after a run. Then provide the application server's bin, log, and config directories. For Glassfish, faban will automatically capture the server config and log file and make it available via the 'Statistics' tab of a run.
The "restart" field indicates whether you want the Faban harness to automatically start and stop the GlassFish server before and after each run. By default, it is set to true which indicates that the Faban harness will restart the GlassFish server, assuming that the path information for the server is entered correctly.
Select the Data Servers tab.
For the database
server, enter the Host name. Edit the hostname part in the JDBC
Connection URL. Note that you cannot use the string
"localhost" - you must specify a valid hostname or IP number.
This is used by the loader program to reload the database before a run. Set the 'Loader for Concurrent Users' to 25 (this is the minimum number of users we can load for and is good for up to 25 concurrent users). You can set the loader to run a larger number so that you don't have to edit this field every time. It is not absolutely necessary to reload the database and files every time, but you should do so for all performance runs. In that case, set the reload fields to true every time.
Set the Data Storage server. For local storage this is the same host as the application server.
Set the memcached
server instances to the servers you've configured in web.xml of the web
application. The driver harness will start the
memcached server instances accordingly. Note that the memcached server
instances are given as host:port pairs, separated by space. If a port
is not given, the default memcached port of 11211 is assumed.
That's it. Click OK and the run should be scheduled. You can click on the View Results link on the left to monitor the run.