The Apache Software Foundation^® AriesTrader Performance Benchmark Sample provides a suite of workloads for characterizing performance of Apache Aries enabled JavaEE Application Servers. The workloads consist of an end-to-end Web application and a full set of Web primitives. The applications are a collection of Java^TM classes, Java servlets, Java ServerPages^TM (JSP^TM) files and Enterprise JavaBeans^TM using JPA for the Java 2 Platform, Enterprise Edition (J2EE^TM) APIs. Together, the Trade application and Web primitives provide versatile and portable test cases that are designed to measure aspects of scalability and performance.

Application Design

What is AriesTrader?

AriesTrader is an end-to-end Web application that is modeled after an on-line stock brokerage. AriesTrader leverages J2EE components such as servlets, JSP files, enterprise beans, message-driven beans (MDBs) and Java database connectivity (JDBC^TM) to provide a set of user services such as login/logout, stock quotes, buy, sell, account details, and so on through standards-based HTTP and Web services protocols.

What are Web Primitives?

The Web primitives leverage the AriesTrader infrastructure to test specific features of the Application Servers implementing the J2EE 1.4 programming model. A description of each of the Web primitives is provided on the main web primitive page.

What software is required to run AriesTrader?

• Any J2EE 1.4 Compliant Application Server
• A database that has a suitable JDBC driver for both XA and non-XA connectivity.

What are the most common configuration scenarios?

• Single server with a remote database - The AriesTrader application runs on a stand alone Application Server instance. The required database software and the associated Trade database are located on a different system from the Application Server. The Application Server system must have the necessary database client software to connect to the remote database.
• Single server with a local database - Same as the previous scenario; however, the required database software and the associated AriesTrader database are located on the same system as the Application Server.

Run-time Configuration

What does the ResetAriesTrader link do?

The ResetAriesTrader link on the configuration page must be clicked between AriesTrader runs. This link sets the database to a consistent size by removing all the newly registered users created during a AriesTrader run. The reset also sets all outstanding orders to a consistent state. Resetting the database to a consistent size ensures repeatable throughput on subsequent AriesTrader runs.

How are the AriesTrader configuration parameters modified?

The Trade configuration page provides a dynamic mechanism to set the run-time configuration for a AriesTrader run. These settings control the application run-time characteristics such as the run-time mode, the order processing mode, and other run-time variations supported in AriesTrader. All settings are reset to defaults when the AriesTrader application server is restarted.

Can you make configuration changes permanent?

Yes. Normally, Trade configuration parameters return to defaults whenever the Trade application server is restarted. Settings can be made permanent by setting the configuration values in the servlet init parameters of the TradeApp servlet and the TradeScenario servlet. Modify the servlet init parameters in the web.xml file of the Trade Web application to change these parameters.

What are the run-time modes?

AriesTrader provides three server implementations of the emulated AriesTrader brokerage services.

• JDBC - Database access uses JDBC technology to drive transactional trading operations.
• JPA App Managed - Database access uses Direct JPA technology with application managed entity managers and transactions to drive trading operations.
• JPA Container Managed - Database access uses Direct JPA technology with container managed entity managers and declarative transactions to drive rading operations.

What is the Primitive Iteration setting?

By default, the AriesTrader primitives run one operation per Web request. Setting this value alters the number of operations performed per client request. This is useful for reducing the amount of work that is performed by the Web Container and for stressing other components within the application server.

Benchmarking

What is the TradeScenario servlet?

The TradeScenario servlet provides a simple mechanism to drive the AriesTrader application. The Trade database is initially populated with a set of fictitious users with names ranging from uid:0 to uid:49 and a set of stocks ranging from s:0 to s:99. The TradeScenario servlet emulates a population of Web users by generating a specific AriesTrader operation for a randomly chosen user on each access to the URL. To run the TradeScenario servlet use the single TradeScenario URL (http://hostname/ariestrader/scenario) with a load generation tool.

Although TradeScenario servlet provides a simple mechanism for driving the AriesTrader application, there is a drawback to using this method versus using a series of load generation scripts that drive the operations directly. This servlet consumes processing resources on the server to manage incoming clients and dispatch these simple client requests to complex Trade actions. This action artificially decreases server throughput because the server is emulating tasks that are normally performed by a standard client or a more complex load generation tool.

What is the typical procedure for collecting performance measurements with AriesTrader?

When AriesTrader is successfully installed on the application server and the supporting database is populated, you can us the AriesTrader application to collect performance measurements. The following list provides the typical process for gathering performance measurements with AriesTrader.

1. Select the AriesTrader run-time configuration parameters from the configuration page (JDBC, JPA Application Managed, JPA Container Managed, and so on).
2. Reset the AriesTrader run-time using the Reset AriesTrader link.
3. Warm-up the application server JVM^TM by applying load for a short period of time. The load generation tool may access the TradeScenario servlet, web primitives, or use custom scripts to drive the various operations of TradeApp servlet. To warm-up the JVM, each code path within AriesTrader must be processed enough times to esnure that the JIT compiler has compiled and optimzed the application and server paths; generally, about 3000 iterations should do the trick. Remember that the same code path is not necessarily run on each request unless primitives are being run. Therefore, perform an adequate number of requests to stabilize the performance results.
4. Stop the load generation tool.
5. Reset the Trade run-time again
6. Restart the load generation tool and record measurements after the driver completes the requests.
7. Repeat steps 5 and 6 to obtain additional measurements.

Where did AriesTrader come from?

AriesTrader was originally an application designed by IBM to test their commercial Application Server. The application was designed around common development patterns as well as to use the majority of the J2EE programming model. The original author was Stan Cox where he developed Trade (the original name) for J2EE 1.3. Since then Stan has evolved Trade and several other individuals have contributed to the project. Christopher Blythe has been instrumental in stabilizing the long running capability of the benchmark and Andrew Spyker introduced the Application Clients. The Application Clients (Streamer and WSAppClient) provide remote capability to validate remote J2EE functionality and database consistency as well as provide a remote WebServices client. Matt Hogstrom has used Trade extensively for performance analysis and brought Trade to the Apache Software Foundation Geronimo Project. He has removed (hopefully) all WebSphere specific items in the application and introduced additional functionality for gathering server compliance information and low-level diagnostic information.

Where is it going?

Version 1.1 of AriesTrader represents the first presentation of the application as an Open Source version for performance and benchmarking. Currently it is focused on J2EE 1.4 but needs to be upgraded to J2EE 1.5. Also, as there is a large number of developers that are not interested in the full J2EE stack, AriesTrader needs to be updated to be modular such that only the interesting pieces need to be deployed.

Also, AriesTrader needs to incorporate some additional technology such as Spring and Hibernate for performance testing such that comparisons can be made against competing technologies.