<%@ page import="org.apache.geronimo.samples.daytrader.*" session="false" isThreadSafe="true" isErrorPage="false" %> <% double loginPercentage = (double) ((TradeConfig.getScenarioMixes())[TradeConfig.workloadMix][TradeConfig.LOGOUT_OP]) / 100.0; double logoutPercentage = (double) ((TradeConfig.getScenarioMixes())[TradeConfig.workloadMix][TradeConfig.LOGOUT_OP]) / 100.0; double buyOrderPercentage = (double) ((TradeConfig.getScenarioMixes())[TradeConfig.workloadMix][TradeConfig.BUY_OP]) / 100.0; double sellOrderPercentage = (double) ((TradeConfig.getScenarioMixes())[TradeConfig.workloadMix][TradeConfig.SELL_OP]) / 100.0; double orderPercentage = buyOrderPercentage + sellOrderPercentage; double registerPercentage = (double) ((TradeConfig.getScenarioMixes())[TradeConfig.workloadMix][TradeConfig.REGISTER_OP]) / 100.0; int logins = runStatsData.getSumLoginCount()-runStatsData.getTradeUserCount(); //account for each user being logged in up front if (logins < 0 ) logins = 0; //no requests before reset //double expectedRequests = ((double) logins) / loginPercentage; double expectedRequests = (double)TradeConfig.getScenarioCount(); TradeConfig.setScenarioCount(0); int verifyPercent = TradeConfig.verifyPercent; %> <%!// verifies 2 values are w/in tradeConfig.verifyPercent percent String verify(double expected, double actual, int verifyPercent) { String retVal = ""; if ( (expected==0.0) || (actual == 0.0) ) return "N/A"; double check = (actual / expected) * 100 - 100; //PASS retVal += check +"% "; if ( (check>=(-1.0*verifyPercent)) && (check<=verifyPercent) ) retVal += " Pass"; else retVal += " Fail4"; if (check > 0.0) retVal = "+" + retVal; //System.out.println("verify --- expected="+expected+" actual="+actual+ " check="+check); return retVal; } String verify(int expected, int actual, int verifyPercent) { return verify((double)expected, (double)actual, verifyPercent); } %>
Trade Scenario Runtime Statistics Trade
<% String status; status = (String) request.getAttribute("status"); if ( status != null )out.print(status); %> Modify runtime configuration
Benchmark scenario statistics
Benchmark runtime configuration summary Value
Run-Time Mode <%= (TradeConfig.getRunTimeModeNames())[TradeConfig.runTimeMode] %>
Order-Processing Mode <%= (TradeConfig.getOrderProcessingModeNames())[TradeConfig.orderProcessingMode]%>
Scenario Workload Mix <%= (TradeConfig.getWorkloadMixNames())[TradeConfig.workloadMix]%>
Web Interface <%= (TradeConfig.getWebInterfaceNames())[TradeConfig.webInterface]%>
Active Traders / Trade User population <%= runStatsData.getTradeUserCount() %> / <%= TradeConfig.getMAX_USERS() %>
Active Stocks / Trade Stock population <%= TradeConfig.getMAX_QUOTES() %> / <%= runStatsData.getTradeStockCount() %>
Benchmark scenario verification
Run Statistic Scenario verification test Expected Value Actual Value Pass/Fail
Active Stocks Active stocks should generally equal the db population of stocks <%= runStatsData.getTradeStockCount() %> <%= TradeConfig.getMAX_QUOTES() %> <%= ( runStatsData.getTradeStockCount() == TradeConfig.getMAX_QUOTES() ) ? "Pass":"Warn" %>
Active Traders Active traders should generally equal the db population of traders <%= runStatsData.getTradeUserCount() %> <%= TradeConfig.getMAX_USERS() %> <%= ( runStatsData.getTradeUserCount() == TradeConfig.getMAX_USERS() ) ? "Pass":"Warn" %>
Estimated total requests Actual benchmark scenario requests should be within +/- 2% of the estimated number of requests in the last benchmark run to pass. <%= expectedRequests %> see2 see2
New Users Registered <%= registerPercentage * 100 %>% of expected requests (<%= registerPercentage%> * <%= expectedRequests %> ) <%= registerPercentage * expectedRequests %> <%= runStatsData.getNewUserCount() %> <%= verify(registerPercentage * expectedRequests, (double)runStatsData.getNewUserCount(), verifyPercent) %>
Logins <%= loginPercentage * 100 %>% of expected requests (<%= loginPercentage%> * <%= expectedRequests %> ) + initial login <%= loginPercentage * expectedRequests + runStatsData.getTradeUserCount() %> <%= runStatsData.getSumLoginCount() + runStatsData.getTradeUserCount() %> <%= verify((double)loginPercentage * expectedRequests, (double)runStatsData.getSumLoginCount(), verifyPercent)%>
Logouts #logouts must be >= #logins-active traders ( <%= runStatsData.getSumLoginCount() %> - <%= TradeConfig.getMAX_USERS() %> ) <%= runStatsData.getSumLoginCount()- TradeConfig.getMAX_USERS() %> <%= runStatsData.getSumLogoutCount() %> <%= (runStatsData.getSumLogoutCount() >= (runStatsData.getSumLoginCount()- TradeConfig.getMAX_USERS())) ? "Pass" : "Fail4" %>
User Holdings Trade users own an average of 5 holdings, 5* total Users = ( 5 * <%= runStatsData.getTradeUserCount() %>) <%= 5 * runStatsData.getTradeUserCount() %> <%= runStatsData.getHoldingCount() %> <%= verify( 5 * runStatsData.getTradeUserCount(), runStatsData.getHoldingCount(), verifyPercent ) %>
Buy Order Count <%= buyOrderPercentage * 100 %>% of expected requests (<%= buyOrderPercentage%> * <%= expectedRequests %> ) + current holdings count <%= buyOrderPercentage * expectedRequests + runStatsData.getHoldingCount() %> <%= runStatsData.getBuyOrderCount() %> <%= verify(buyOrderPercentage * expectedRequests + runStatsData.getHoldingCount() , (double)runStatsData.getBuyOrderCount(), verifyPercent)%>
Sell Order Count <%= sellOrderPercentage * 100 %>% of expected requests (<%= sellOrderPercentage%> * <%= expectedRequests %> ) <%= sellOrderPercentage * expectedRequests %> <%= runStatsData.getSellOrderCount() %> <%= verify(sellOrderPercentage * expectedRequests, (double)runStatsData.getSellOrderCount(), verifyPercent)%>
Total Order Count <%= orderPercentage * 100 %>% of expected requests (<%= orderPercentage%> * <%= expectedRequests %> ) + current holdings count <%= orderPercentage * expectedRequests + runStatsData.getHoldingCount() %> <%= runStatsData.getOrderCount() %> <%= verify(orderPercentage * expectedRequests + runStatsData.getHoldingCount(), (double)runStatsData.getOrderCount(), verifyPercent)%>
Open Orders All orders should be completed before reset3 0 <%= runStatsData.getOpenOrderCount() %> <%= (runStatsData.getOpenOrderCount() > 0) ? "Fail4" : "Pass" %>
Cancelled Orders Orders are cancelled if an error is encountered during order processing. 0 <%= runStatsData.getCancelledOrderCount() %> <%= (runStatsData.getCancelledOrderCount() > 0) ? "Fail4" : "Pass" %>
Orders remaining after reset After Trade reset, each user should carry an average of 5 orders in the database. 5* total Users = (5 * <%= runStatsData.getTradeUserCount() %>) <%= 5 * runStatsData.getTradeUserCount() %> <%= runStatsData.getOrderCount()-runStatsData.getDeletedOrderCount() %> <%= verify( 5 * runStatsData.getTradeUserCount(), runStatsData.getOrderCount()-runStatsData.getDeletedOrderCount(), verifyPercent ) %>
  1. Benchmark verification tests require a Trade Reset between each benchmark run.
  2. The expected value of benchmark requests is computed based on the the count from the Web application since the last Trade reset.The actual value of benchmark request requires user verification and may be incorrect for a cluster.
  3. Orders are processed asynchronously in Trade. Therefore, processing may continue beyond the end of a benchmark run. Trade Reset should not be invoked until processing is completed.
  4. Actual values must be within <%= TradeConfig.verifyPercent %>% of corresponding estimated values to pass verification.
Trade Scenario Runtime Statistics Trade
Apache Geronimo Performance Benchmark Sample DayTrader
Copyright 2005, Apache Software Foundation