Home > Documentation > Apache Geronimo v1.0 - User's Guide > Sample applications > SPECjAppServer2004
Apache Geronimo v1.0
SPECjAppServer2004
Edit Page   Browse Space   Add Page   Add News

Running SPECjAppServer2004 benchmark on Geronimo

Still not a success, your help needed!

This article shows how to measure the performance of the Geronimo application server using the industry standard SPECjAppServer2004 benchmark.

Issues still exist that prevent SPECjAppServer2004 from running on Geronimo.

This article is not a success story, but a collection of notes on the progress that has been made in this direction.

Current issue that requires your help is: #Running the benchmark

Disclaimer: This article is created to write down my experience and make it reproducible. It is not targeted to be a comprehensive guide on either product or on merging them together. It's not also a replacement to the products' documentation, but just a step-by-step guide on how to make things work in a simple configuration, as it worked for me. Make sure you at least look through the documentation on both products before you proceed.

SPECjAppServer is a trademark of the Standard Performance Evaluation Corp. (SPEC). The SPECjAppServer2004 results or findings in this publication have not been reviewed or accepted by SPEC, therefore no comparison nor performance inference can be made against any published SPEC result. The official web site for SPECjAppServer2004 is located at http://www.spec.org/jAppServer2004/.

This article is written for SPECjAppServer2004 v1.05 and Geronimo v1.0. For other versions some stages may be different.

The described configuration uses as many Geronimo components as possible, including the built-in Derby database and the built-in Jetty or Tomcat servlet container. In fact, the configuration only uses Java, Geronimo, an external servlet container (e. g. Tomcat) and SPECjAppServer2004. To plug external components (most probably, a database), you have to change your configuration accordingly.

This configuration also assumes that all the components (except, possibly, the SPECjAppServer2004 Driver and the SPECjAppServer2004 Supplier Emulator and its servlet container) are run on the same machine. If you want to run the Distributed workload, your configuration will be different.

This configuration uses the Microsoft Windows XP Professional Service Pack 2 operating system, Cygwin shell, Sun Java SE 1.4.2 and Tomcat v5.0.30 to write this article. If you use another OS, Java or servlet container, some stages may be different.

This article uses forward slashes ( / ) for command lines, adjust to backslashes ( \ ) accordingly if you use Windows command prompt.

This article has the following structure:

General information

About Geronimo

Geronimo is the Apache Software Foundation Java EE application server. It is developed under Apache License and can be downloaded freely.

Apache site: http://apache.org

Product site: http://geronimo.apache.org

Documentation page: http://geronimo.apache.org/documentation.html

The best document available is "Apache Geronimo Development and Deployment" book by Aaron Mulder: http://chariotsolutions.com/geronimo/geronimo-html-one-page.html

The second necessary doc is the Wiki page: http://wiki.apache.org/geronimo/

About SPECjAppServer2004

SPECjAppServer2004 is a commercial benchmark for measuring the performance of Java EE application servers.

SPEC site: http://www.spec.org

Product site: http://www.spec.org/jAppServer2004/

FAQ: http://www.spec.org/jAppServer2004/docs/FAQ.html

User's Guide: http://www.spec.org/jAppServer2004/docs/UserGuide.html

Run and Reporting Rules: http://www.spec.org/jAppServer2004/docs/RunRules.html

Back to Top

Obtaining products

Obtaining Geronimo

The latest Geronimo version for now is 1.0.

General download page: http://geronimo.apache.org/downloads.html

Two builds of Geronimo exist, with Jetty or Tomcat servlet container enabled by default. You can download either one at http://geronimo.apache.org/downloads.html, they are slightly larger than 60 MB in size. This document was written primarily using Jetty version, but Tomcat version works fine also.

Previous versions additionally included a GUI installer, which is not available on the download page now. However, you can build the installer from sources. You will get a geronimo-installer-1.0.jar file, 56 MB in size.

Obtaining SPECjAppServer2004

SPECjAppServer2004 costs $2000, you can order it online. See FAQ for details.

The latest version is 1.05 coming as the SPECjAppServer2004-Kit-v1.05-050425.jar file, 12 MB in size.

Back to Top

Conventions and environment

This section contains important notions that mark the important hosts and directories.

Hosts

This article is written in terms of the following machines:

  • driver.host – the machine where the SPECjAppServer2004 Driver is run. If you use configuration with multiple Drivers, you have to repeat all the operations for this host on all Driver hosts.
  • master.host – the main, Master driver.host in configurations with multiple Drivers.

The emulator.host and the driver.host may be the same machine.

The geronimo.host and the emulator.host may be the same machine, moreover, the SPECjAppServer2004 Supplier Emulator may be deployed into a Geronimo built-in servlet container (Jetty or Tomcat).

The geronimo.host and the driver.host may be the same machine, but you have to adjust the Geronimo configuration, as both Geronimo and the SPECjAppServer2004 Driver create RMI Registry on the default port (1099) and would conflict on that.

Sharing geronimo.host with emulator.host or driver.host is contradicting with the SPECjAppServer2004 documentation and would impact the performance severely and invalidate the benchmark results. However technically this is possible.

Directories

This section lists important directories that are futher addressed in this article. They can be chosen arbitrary, but should not overlap.

  • <GERONIMO> – directory at the geronimo.host where Geronimo is installed.
  • <KIT> – directory at the geronimo.host containing the files attached to this article.
  • <TOMCAT> – directory at the emulator.host where Tomcat is installed.
  • <DRIVER> – directory at the driver.host that is a copy of the <SPEC> directory.
  • <DRIVER_GERONIMO> – directory at the driver.host that is a copy of the <GERONIMO> directory.
  • <JAVA_HOME> – JAVA_HOME location at the driver.host.
  • <OUTPUT> – directory at the driver.host where the SPECjAppServer2004 Driver will store its output.
  • <DUMP> – directory at the driver.host where the SPECjAppServer2004 Driver will store its temporal files.

It was reported that some components may work incorrectly if working paths are too long or contain spaces. So it's recommended that you avoid long paths and spaces in them.

Back to Top

Installing products

First, save the files attached to this article to a local directory. This will be your <KIT> directory.

Installing Geronimo

You can install Geronimo using one of the following – the .zip archive, the .jar GUI installer or an installation script. Installation script is created by the GUI installer at the previous installation.

Installing from a .zip file

Extract the downloaded archive to a local directory. The geronimo-1.0 directory is created, that is your <GERONIMO> directory.

Installing with the .jar installer

Run:

java -jar geronimo-installer-1.0.jar

Click Next, read and accept the license agreement, and type in the directory you want Geronimo installed to. This directory will be your <GERONIMO> directory.

In the components list, deselect either Tomcat Web Container or Jetty Web Container to save yourself configuration effort, as you only need one servlet container to run SPECjAppServer2004.

Leave other settings intact. Click Next.

On the following tabs, leave the default values for the user name (system), password (manager) and port numbers, just click Next.

If you did not turn off one of the servlet containers earlier, make sure you provide different port numbers for them, otherwise they would conflict on that.

Wait until the installation completes, then click Next, and again Next.

At this stage, you may create an installation script to re-install Geronimo later.

Click Done.

Installing with an installation script

To re-install Geronimo (possibly, on a different machine) using the installation script created by GUI installer earlier, run the installer as follows:

java -jar geronimo-installer-1.0.jar <installation script>

The script doest not start GUI and asks for no additional settings, but just follows the pre-set installation scenario.

The installation directory is the same as per the initial installation. If you wish to install to another location, edit the installation script beforehand: change the <installpath> tag value to the desired <GERONIMO> location.

Installing SPECjAppServer2004

Run:

java -jar SPECjAppServer2004-Kit-v1.05-050425.jar

Click Next, read and accept the license agreement, and type in the directory you want the SPECjAppServer2004 to be installed to. This directory will be your <SPEC> directory.

Click Install.

Wait until the installation completes, then click Ready.

Back to Top

Configuring Geronimo

Adjusting configuration

Adjust the EJB configuration by adding the allowHosts attribute to the <GERONIMO>/var/config/config.xml file, as shown below.

This is necessary to allow the SPECjAppServer2004 Driver to remotely access the EJBs deployed in Geronimo.

If your geronimo.host and your driver.host are the same machine, you have to adjust the port number of the Geronimo RMI Registry (to e. g. 10099), otherwise it would conflict with the SPECjAppServer2004 Driver that uses the default port of 1099:

Due to a bug in Geronimo 1.0 changing RMI Registry port prevents JMXConnector module starting, and that in turns blocks Deployer operation. So until this bug is fixed, you should leave the port at default value until you deploy all the modules you need, and after that you may set the port number to whatever you want and restart Geronimo.

Starting Geronimo

Go to your <GERONIMO> directory.

Start Geronimo by typing:

java -jar bin/server.jar

It will take some time to start. After that, you will see:

Booting Geronimo Kernel (in Java 1.4.2)...
Starting Geronimo Application Server
[*********************] 100%  40s Startup complete
  Listening on Ports:
    1099 0.0.0.0   RMI Naming
    1389 0.0.0.0   Apache Directory LDAP
    1527 0.0.0.0   Derby Connector
    4201 0.0.0.0   ActiveIO Connector EJB
    4242 0.0.0.0   Remote Login Listener
    8019 127.0.0.1 Jetty Connector AJP13
    8080 0.0.0.0   Jetty Connector HTTP
    8443 0.0.0.0   Jetty Connector HTTPS
   61616 0.0.0.0   ActiveMQ Message Broker Connector

  Started Application Modules:
    EAR: geronimo/uddi-jetty/1.0/car
    EAR: geronimo/webconsole-jetty/1.0/car
    RAR: geronimo/activemq/1.0/car
    RAR: geronimo/system-database/1.0/car
    WAR: geronimo/jmxdebug-jetty/1.0/car
    WAR: geronimo/jsp-examples-jetty/1.0/car
    WAR: geronimo/remote-deploy-jetty/1.0/car
    WAR: geronimo/servlets-examples-jetty/1.0/car
    WAR: geronimo/welcome-jetty/1.0/car

  Web Applications:
    http://geronimo.host:8080/
    http://geronimo.host:8080/console
    http://geronimo.host:8080/console-standard
    http://geronimo.host:8080/debug-tool
    http://geronimo.host:8080/jsp-examples
    http://geronimo.host:8080/juddi
    http://geronimo.host:8080/remote-deploy
    http://geronimo.host:8080/servlets-examples

Geronimo Application Server started

If you get another result, particularly, if network errors show up, then something has gone wrong.

Sometimes, the startup fails because some local network addresses are inaccessible. This could happen, for example, if you have used a VPN interface that is disconnected now. By default, Geronimo uses the first local address it comes across to access its components, and may try to use a stale address, causing startup errors.

You may try disabling and then re-enabling the unused network interfaces to resolve such issues.

Accessing the console

Open your web browser and connect to the Geronimo Console at http://geronimo.host:8080/console/

Log in using the user name and password you provided during the installation (system and manager by default).

Now you may investigate the console if you wish.

Creating the database

Go to the downmost link in the console, DB Manager.

Create the benchmark database by typing its name (SPECDB) in the Create DB field and clicking Create.

Locating the SQL files

To create database tables, you can use the default SQL scripts provided in the <SPEC>/schema/sql directory. However, the directory includes five scripts, and they contain DROP TABLE commands that would fail if you try to execute them in the console when tables are not yet created.

Instead, it is recommended that you use the allTablesNoDrop.sql file, if you are creating the tables from scratch, or allTables.sql file if you want to drop and recreate the tables. Both files were created from the <SPEC>/schema/sql files by simple concatenation, allTablesNoDrop.sql also has DROP TABLE commands removed.

Creating the tables

Make sure SPECDB is selected in Use DB field and then copy-paste the SQL code to SQL Command/s frame. Click Run SQL button above it.

After a short delay, the frame will clear and the Result field below it will say SQL command/s successful. If not – check what you did wrong and try again.

If you use multiple SQL scripts, repeat the operations above for each of them.

Back to Top

Configuring SPECjAppServer2004

Basic configuration

deploy directory

Go to the <SPEC>/src/deploy directory and copy the reference subdirectory with its contents with the name geronimo.

geronimo.env file

Go to the <SPEC>/config directory.

Put the attached geronimo.env template file there. Edit it, make sure you set the values for the following variables:

JAS_HOME=<SPEC>
JAVA_HOME=<JAVA_HOME>
J2EE_HOME=<GERONIMO>
JAS_HOST=geronimo.host
EMULATOR_HOST=emulator.host

Use forward slashes ( / ) as directory separators!

You may leave the other variables intact.

appsserver file

Edit the <SPEC>/config/appsserver file – replace the word default there with the word geronimo.

run.properties file

Edit the <SPEC>/config/run.properties file. Note that it will be used on the driver.host and make sure the following variables have correct values:

Url = http://geronimo.host:8080/SPECjAppServer/app?
outDir = <OUTPUT>
dumpDir = <DUMP>

setenv.bat file

Edit the <SPEC>/bin/setenv.bat file, make sure you set the values for the following variables:

JAVA_HOME=<JAVA_HOME>
JAS_HOME=<DRIVER>
APPSSERVER=geronimo

jndi.properties file

jndi.properties is important for OpenEJB configuration and must be available in the Geronimo Driver classpath. Put it to the <SPEC>/config directory.

Requirement for this file is a bug in OpenEJB build used in Geronimo version 1.0, it is going to be fixed in version 1.1.

Building the application

Go to the <SPEC> directory.

Clean-up your installation:

ant/bin/ant clean

Build the application and configure it for Geronimo:

ant/bin/ant -Dappserver=geronimo

You will get the BUILD SUCCESSFUL diagnostic.

Make sure the files SPECjAppServer.ear and Emulator.war are created in the <SPEC>/jars directory.

Rename emulator.war to Emulator.war.

Preparing database configuration

In the described configuration, the same database is used for all tables.

Go to the <SPEC>/config directory. Replace the content of each of the *db.properties files you find there with the contents of the attached db.properties template file. Make sure the pipeDir variable there points to a valid temporary directory, adjust if necessary.

Loading the tables

Run:

ant/bin/ant -Dappserver=geronimo loaddb

After some time, you will get the BUILD SUCCESSFUL diagnostic.

Back to Top

Deploying components

At this stage you need to deploy the configured components to Geronimo.

Deploying database connector

To deploy a connector to the Derby SPECDB database you created earlier, go to the <GERONIMO> directory and run:

java -jar bin/deployer.jar --user system --password manager deploy repository/tranql/rars/tranql-connector-derby-embed-xa-1.1.rar <KIT>/specdb.xml

You will get the Deployed SPECjAppServerDB diagnostic.

Deploying JMS connector

To deploy an ActiveMQ JMS connector for SPECjAppServer2004, go to the <GERONIMO> directory and run:

java -jar bin/deployer.jar --user system --password manager deploy repository/activemq/rars/activemq-ra-3.2.1.rar <KIT>/specjms.xml

You will get the Deployed SPECjAppServerJMS diagnostic.

Deploying the main application

To deploy SPECjAppServer2004 on Geronimo, this configuration uses the slightly modified deployment plan that was originally found in Geronimo sources at http://svn.apache.org/viewcvs.cgi/geronimo/sandbox/specjappserver2004/specj2004-deployment-plan.xml.

Go to the <GERONIMO> directory and run:

java -jar bin/deployer.jar --user system --password manager deploy <SPEC>/jars/SPECjAppServer.ear <KIT>/specj2004-deployment-plan.xml

You will get the Deployed SPECjAppServer diagnostic.

Note an important issue about this deployment plan: http://www.ibm.com/support/docview.wss?rs=2327&uid=swg21229335

Verifying the deployment

At this stage you may check that the deployment has been done correctly and that SPECjAppServer2004 is operational.

Atomicity tests

Open the deployed SPECjAppServer2004 page: http://geronimo.host:8080/SPECjAppServer/

In the left-hand menu, click the Atomicity Tests link.

You will see the results of three atomicity tests' runs. If all three of them are marked as PASSED, your deployment is correct.

Manual transactions

Open the deployed SPECjAppServer2004 page: http://geronimo.host:8080/SPECjAppServer/

In the left-hand menu, click the Go Trade Autos! or Go Build Cars! link.

Log in with the default credentials (1) by clicking Log in.

You should see the program interface and be able to perform transactions.

Back to Top

Deploying the Supplier Emulator

To deploy the SPECjAppServer2004 Supplier Emulator at the emulator.host, use one of the following:

The Geronimo built-in servlet container (in case the emulator.host and the geronimo.host are the same machine); or a stand-alone servlet container on the emulator.host.

Note that SPECjAppServer2004 documentation requires that the Supplier Emulator servlet container should have the keep-alive option turned off. You can ignore this requirement, but that would impact the performance severely.

Using the Geronimo servlet container

Go to the <GERONIMO> directory and run:

java -jar bin/deployer.jar --user system --password manager deploy <SPEC>/jars/Emulator.war

You will get the Deployed Emulator @ http://geronimo.host:8080/Emulator diagnostic.

Using a stand-alone servlet container

This configuration assumes that your stand-alone servlet container on the emulator.host is Tomcat running on the default port (8080).

Install Tomcat to your <TOMCAT> directory on the emulator.host.

Do not bother editing <SPEC>/config/tomcat.env file or running ant/bin/ant -f tomcat.xml.

Both files are obsolete, they generate the Emulator.war file, which has already been created at #Building the application phase.

Copy the <SPEC>/jars/Emulator.war file to the <TOMCAT>/webapps directory and remove the <TOMCAT>/webapps/Emulator directory if it exists.

Go to the <TOMCAT> directory on the emulator.host and start Tomcat:

bin/catalina run

Veryfying the deployment

Go to the page http://emulator.host:8080/Emulator/. It should load normally and contain a single directory, dtd, with two files inside, delivery.dtd and po.dtd.

Go to the page http://emulator.host:8080/Emulator/EmulatorServlet. You should see a page with text like this:

Emulator Servlet seems to work OK
JAS_HOST : emulator.host
JAS_PORT : 8080
Servlet URL : Supplier/DeliveryServlet

Number of Transactions : 0
Servlet invoked without command specified

Back to Top

Running the benchmark

Copy the <GERONIMO> directory to the driver.host, the copy will be your <DRIVER_GERONIMO> directory (in fact you only need some jars from it).

Copy the <SPEC> directory to the driver.host, the copy will be your <DRIVER> directory.

In the <DRIVER>/config/geronimo.env file adjust the JAS_HOME variable to the <DRIVER> directory and J2EE_HOME variable to the <DRIVER_GERONIMO> directory.

Go to the <DRIVER> directory on the driver.host and run:

bin/setenv.bat

This configures the environment to run the Driver.

To start the Driver itself, run:

bin/driver.bat

If you wish to run a distributed load with multiple Drivers, then after the Driver is started on the first host (the master.host), start the Driver on other driver hosts like this:

bin/driver.bat master.host

After starting the Driver, you should see the output like this:

The following environment settings are in effect for SPECjAppServer2004
* ========================= *
   JAVA_HOME=<JAVA_HOME>
    JAS_HOME=<DRIVER>
  CONFIG_DIR=<DRIVER>\config
  APPSSERVER=geronimo
     ENVFILE=<DRIVER>\config\geronimo.env
* ========================= *
Driver Host: <driver.host>
Binding controller to //<driver.host>/Controller
DriverDebug: DealerAgent <propsFile> <agentName> <masterMachine>
Controller: Registering M1 on machine <driver.host IP address>
Controller: Registering O1 on machine <driver.host IP address>
Controller: Registering L1 on machine <driver.host IP address>
Calling switchLog as master
RunID for this run is : 75
Output directory for this run is : <OUTPUT>\75
TTF1 = 93
ttf = 93
Configuring 1 DealerAgent(s)...
DealerAgent O1, Thread 0 started
DealerAgent O1, Thread 1 started
DealerAgent O1, Thread 2 started
DealerAgent O1, Thread 3 started
DealerAgent O1, Thread 4 started
DealerAgent O1, Thread 5 started
DealerAgent O1, Thread 6 started
DealerAgent O1, Thread 7 started
DealerAgent O1, Thread 8 started
DealerAgent O1, Thread 9 started
Configuring 1 MfgAgent(s)...
MfgAgent M1, Thread 0 started
MfgAgent M1, Thread 1 started
MfgAgent M1, Thread 2 started
Configuring 1 LargeOLAgent(s)...


MfgAgent L1, Thread 0 started
Rampup      = Fri May 12 20:49:51 MSD 2006
SteadyState = Fri May 12 20:59:51 MSD 2006
Rampdown    = Fri May 12 21:59:51 MSD 2006
Finish      = Fri May 12 22:04:51 MSD 2006

sleeptime is 28417 note this is time in excess needed for trigger
Starting Ramp Up...

This means the Driver started normally.

Note the times for Rampup, SteadyState, Rampdown and Finish to figure out the time needed for the benchmark to complete.

You can interrupt the run at any point with Ctrl-C.

Sometimes binding exceptions or other problems may occur at the Driver startup. In such a case, interrupt the test run with Ctrl-C and rerun it again. Sometimes it helps.

In some cases diagnostics like these may occur:

java.rmi.RemoteException: Failure in calling validateInitialValues() java.rmi.RemoteException: Invalid initial Order DB State
        at org.spec.jappserver.driver.Auditor.validateInitialValues(Auditor.java:201)
        at org.spec.jappserver.driver.Driver.configure(Driver.java:330)
        at org.spec.jappserver.driver.Driver.<init>(Driver.java:160)
        at org.spec.jappserver.driver.Driver.main(Driver.java:1137)

or

java.rmi.RemoteException: Failure in calling validateInitialValues() javax.transaction.TransactionRolledbackException: java.sql.SQLException
        at org.spec.jappserver.driver.Auditor.validateInitialValues(Auditor.java:201)
        at org.spec.jappserver.driver.Driver.configure(Driver.java:330)
        at org.spec.jappserver.driver.Driver.<init>(Driver.java:160)
        at org.spec.jappserver.driver.Driver.main(Driver.java:1137)

To overcome such problems, you may clear the contents of M_LARGEORDER table using the following SQL statement:

DELETE FROM M_LARGEORDER

or you may delete the SPECDB database, recreate the database and the tables and then reload them.

During the run, the following diagnostics may appear in the Driver window:

Error in html org.spec.jappserver.driver.http.SJASHttpException: error while checking html
org.spec.jappserver.driver.http.SJASHttpException: error logging in customer 7308 org.spec.jappserver.driver.http.SJASHttpException: error while checking html
        at org.spec.jappserver.driver.http.HttpDealer.login(HttpDealer.java:621)
        at org.spec.jappserver.driver.DealerEntry.logIn(DealerEntry.java:935)
        at org.spec.jappserver.driver.DealerEntry.loginCtl(DealerEntry.java:880)
        at org.spec.jappserver.driver.DealerEntry.doBrowse(DealerEntry.java:790)
        at org.spec.jappserver.driver.DealerEntry.run(DealerEntry.java:247)
AbstractSJASLoad> Application error has already been cancelled
AbstractSJASLoad> Application error has already been cancelled
AbstractSJASLoad> Application error has already been cancelled

and in the same time, various TransactionRolledback and other exceptions of the same kind are being printed in the Geronimo shell.

These diagnostics may be normal, but may be errors that invalidate the benchmark results. This still has to be investigated.

After the run has completed successfuly, you will see the output like this:

Gathering DealerStats ...
Gathering MfgStats ...
summary file is <OUTPUT>\75\SPECjAppServer.summary
SPECjAppServer2004 v1.05 Results
JOPS: ***
Dealer Response Times
     Purchase...0.4
     Manage.....1.5
     Browse.....0.4
Manufacturing Response Times
     Mfg........0.0
Calling getLog as master

The number of JOPS is a final benchmark metric.

For now these values for Geronimo are terribly low (around 1), and don't depend on the hardware being used. Probably this is due to some configuration issues that still exist.

Back to Top

Processing results

When driver is run, a subdirectory with a numerical name is created in the <OUTPUT> directory. The subdirectory with the largest number corresponds to the latest run. After the Driver run is complete, the result.props file is created there, it contains the raw results from the benchmark.

Go to the <DRIVER>/reporter directory, copy the file Sample_Submission.txt under arbitraty name (e. g. Your_Submission.txt) and edit the copy, as described in SPECjAppServer2004 User's Guide :: Section 5 – Results, add the raw data from the result.props file.

Run the following command to generate an HTML benchmark report, it would be named Your_Submission.report.html:

java -cp reporter.jar reporter Your_Submission.txt

Run the following command to generate a text-only benchmark report, it would be named Your_Submission.report.txt:

java -cp reporter.jar reporter -a Your_Submission.txt

For the details on how yo submit your results, see SPECjAppServer2004 User's Guide :: Section 5.3 – Submitting the Results

Back to Top

   Delicious Bookmark this on Delicious    Digg! Digg this    Privacy Policy  -   Copyright © 2003-2009, The Apache Software Foundation, Licensed under ASL 2.0.