Changes

Version 3.1

Summary

• New and Noteworthy
• Incompatible changes
• Bug fixes
• Improvements
• Non-functional changes
• Known problems and workarounds
• Thanks

New and Noteworthy

Improve Report/Dashboard

The Dashboard has been improved with 3 new graphs and 1 summary table:

• Connect Time over Time graph :

  

• Response Time Percentiles Over Time (successful responses) graph :

  

• Response Time Overview graph :

  

• Top 5 errors by Sampler table :

  

• More details on errors in Errors table
• Average response time added to Statistics table :

  

• Active Threads table now stacks threads :

  

New Metrics

A new sent_bytes metric has been introduced which reports the bytes sent to server.
Another metric connect_time has been enabled by default in this version

Handling Big responses

JMeter is now able to handle in terms of metrics responses bigger than 2GB, limit has been increased to 9223372 TB.
To handle such big responses, it can also now truncate part of the response to avoid overflooding memory. See httpsampler.max_bytes_to_store_per_request property.

New __groovy function

Introduce a new function __groovy that enables Groovy functions. This can be handy, as JavaScript can be quite slow (same for BeanShell), when used in highly concurrent test plans.

Use Groovy as default for JSR-223 elements

Groovy is now set as the default language for JSR-223 elements. If you want to use another of the supported language, you have to make an explicit choice.

Formatted HTML source view in Results Tree View

The HTML source code in the Results Tree View can now be viewed formatted. This is extremely useful, if the code of the webpage has been stripped of all superfluous whitespace.

Ability to update all timers in Test plan with a new property

A new property timer.factor=1.0f has been introduced which allows you to multiply pause times computed by Gaussian, Uniform and Poisson Timers by it. This allows you to update Think Times from one place and let you gain productivity.

Core improvements

• Various GUI and UX fixes
• Memory usage improvements
• JDBC Request is now able to return Blob/Clob and computes latency and connect time
• CSS Parsing introduced in 3.0 has been optimized by introduction of a parsing cache
• HTTP Request is now able to handle body in GET request, this is useful for Elastic Search requests for example.

Documentation improvements

• Documentation review and improvements for easier startup
• New properties reference documentation section

Incompatible changes

• A cache for CSS Parsing of URLs has been introduced in this version, it is enabled by default. It is controlled by property css.parser.cache.size. It can be disabled by setting its value to 0. See Bug 59885
• ThroughputController defaults have changed. Now defaults are Percent Executions which is global and no more per user. See Bug 60023
• Since version 3.1, HTML report ignores empty Transaction Controller (possibly generated by If Controller or Throughput Controller) when computing metrics. This provides more accurate metrics
• Since version 3.1, Summariser ignores SampleResults generated by Transaction Controller when computing the live statistics, see Bug 60109
• Since version 3.1, when using Stripped modes (by default StrippedBatch is used), response will be stripped also for failing SampleResults, you can revert this to previous behaviour by setting sample_sender_strip_also_on_error=false in user.properties, see Bug 60137
• Since version 3.1, jmeter.save.saveservice.connect_time property value is true, meaning CSV file for results will contain an additional column containing connection time, see Bug 60106
• Since version 3.1, Random Timer subclasses (Gaussian Random Timer, Uniform Random Timer and Poisson Random Timer) implement interface org.apache.jmeter.timers.ModifiableTimer
• Since version 3.1, if you don't select any language in JSR223 Test Elements, Apache Groovy language will be used. See Bug 59945
• Since version 3.1, CSV DataSet now trims variable names to avoid issues due to spaces between variables names when configuring CSV DataSet. This should not have any impact for you unless you use space at the begining or end of your variable names. See Bug 60221
• Since version 3.1, HTTP Request is able when using HttpClient4 (default) implementation to handle responses bigger than 2147483647 Bytes, that is 2GB. To allow this two properties have been introduced:
  • httpsampler.max_bytes_to_store_per_request (defaults to 10MB) will control what is held in memory. By default JMeter will only keep in memory the first 10MB of a response. If you have responses larger than this value and use assertions that are after the first 10MB, then you must increase this value
  • httpsampler.max_buffer_size will control the buffer used to read the data. Previously JMeter used a buffer equal to Content-Length header which could lead to failures and make JMeter less resistant to faulty applications, but note this may impact response times and give slightly different results than previous versions if your application returned a Content-Length header higher than current default value (65KB)
  See Bug 53039

Deprecated and removed elements or functions

• Bug 60222 - Remove deprecated elements Distribution Graph, Spline Visualizer
• Bug 60224 - Deprecate Monitor Results listener. It will be dropped in next version.
• Bug 60323 - Deprecate BSF Elements (Use JSR223 Elements instead). They will probably be dropped in N+2 version. The following elements are deprecated:
  • BSF Sampler
  • BSF Listener
  • BSF Assertion
  • BSF Timer
  • BSF PreProcessor
  • BSF PostProcessor
• Bug 60225 - Drop deprecated __jexl function, jexl support in BSF and dependency on commons-jexl-1.1.jar. This function can be easily replaced with __jexl3 function
• Bug 60268 - Drop org.apache.jmeter.gui.action.Analyze and deprecate org.apache.jmeter.reporters.FileReporter (will be removed in next version)

Improvements

HTTP Samplers and Test Script Recorder

• Bug 59882 - Reduce memory allocations for better throughput. Contributed by Benoit Wiart (b.wiart at ubik-ingenierie.com) through Pull request #217 - and Pull request #228 -
• Bug 59885 - Optimize css parsing for embedded resources download by introducing a cache. Contributed by Benoit Wiart (b.wiart at ubik-ingenierie.com) through Pull request #219 -
• Bug 60092 - View Result Tree: Add shortened version of the PUT body to sampler result.
• Bug 60229 - Add a new metric : sent_bytes. Implemented by Philippe Mouawad (p.mouawad at ubik-ingenierie.com) and contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 53039 - HTTP Request : Be able to handle responses which size exceeds 2147483647 bytes (that is 2GB)
• Bug 60265 - HTTP Request : In Files Upload Tab you cannot resize columns
• Bug 60318 - Ignore CSS warnings when parsing with ph-css library.
• Bug 60358 - Http Request : Allow sending Body Data for HTTP GET request

Other samplers

• Pull request #211 - Differentiate the timing for JDBC Sampler. Use latency and connect time. Contributed by Thomas Peyrard (thomas.peyrard at murex.com)
• Bug 59620 - Fix button action in "JMS Publisher → Random File from folder specified below" to allow to select a directory
• Bug 60066 - Handle CLOBs and BLOBs and limit them if necessary when storing them in result sampler.

Controllers

• Bug 59351 - Improve log/error/message for IncludeController. Partly contributed by Antonio Gomes Rodrigues (ra0077 at gmail.com)
• Bug 60023 - ThroughputController : Make "Percent Executions" and global the default values. Contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 60082 - Validation mode : Be able to force Throughput Controller to run as if it was set to 100%
• Bug 59349 - Trim spaces in input filename in IncludeController.
• Bug 60081 - Interleave Controller : Add an option to alternate across threads

Listeners

• Bug 59953 - GraphiteBackendListener : Add Average metric. Partly contributed by Maxime Chassagneux (maxime.chassagneux at gmail.com)
• Bug 59975 - View Results Tree : Text renderer annoyingly scrolls down when content is bulky. Contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 60109 - Summariser : Make it ignore TC generated SampleResult in its summary computations
• Bug 59948 - Add a formatted and sane HTML source code render to View Results Tree
• Bug 60252 - Add sent kbytes/s to Aggregate Report and Summary report
• Bug 60267 - UX : In View Results Tree it should be possible to close the Configure popup by typing escape. Contributed by Ubik Load Pack (support at ubikloadpack.com)

Timers, Assertions, Config, Pre- & Post-Processors

• Bug 59845 - Log messages about JSON Path mismatches at debug level instead of error.
• Pull request #212 - Allow multiple selection and delete in HTTP Authorization Manager. Based on a patch by Benoit Wiart (b.wiart at ubik-ingenierie.com)
• Bug 59816 - Pull request #213 - Allow multiple selection and delete in HTTP Header Manager. Based on a patch by Benoit Wiart (b.wiart at ubik-ingenierie.com)
• Bug 59967 - CSS/JQuery Extractor : Allow empty default value. Contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 59974 - Response Assertion : Add button "Add from clipboard". Contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 60050 - CSV Data Set : Make it clear in the logs when a thread will exit due to this configuration
• Bug 59962 - Cache Manager does not update expires date when response code is 304.
• Bug 60018 - Timer : Add a factor to apply on pauses. Partly based on a patch by Ubik Load Pack (support at ubikloadpack.com)
• Bug 60203 - Use more available space for textarea in XPath Assertion.
• Bug 60220 - Rename JSON Path Post Processor to JSON Extractor
• Bug 60221 - CSV DataSet : trim variable names
• Bug 59329 - Trim spaces in input filename in CSVDataSet.

Functions

• Bug 59963 - New function __RandomFromMultipleVars: Ability to compute a random value from values of one or more variables. Contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 59991 - New function __groovy to evaluate Groovy Script. Contributed by Ubik Load Pack (support at ubikloadpack.com)

I18N

• Pull request #214 - Add spanish translation for delayed starting of threads. Contributed by Asier Lostalé (asier.lostale at openbravo.com).
• Bug 60348 - Change chinese translation for Save as. Contributed by XMeter (support at xmeter.net).

Report / Dashboard

• Bug 59954 - Web Report/Dashboard : Add average metric
• Bug 59956 - Web Report / Dashboard : Add ability to generate a graph for a range of data
• Bug 60065 - Report / Dashboard : Improve Dashboard Error Summary by adding response message to "Type of error". Contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 60079 - Report / Dashboard : Add a new "Response Time Overview" graph
• Bug 60080 - Report / Dashboard : Add a new "Connect Time Over Time " graph. Contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 60091 - Report / Dashboard : Have a new report containing min/max and percentiles graphs.
• Bug 60108 - Report / Dashboard : In Requests Summary rounding is too aggressive
• Bug 60098 - Report / Dashboard : Reduce default value for "jmeter.reportgenerator.statistic_window" to reduce memory impact
• Bug 60115 - Add date format property for start/end date filter into Report generator
• Bug 60171 - Report / Dashboard : Active Threads Over Time should stack lines to give the total amount of threads running
• Bug 60250 - Report / Dashboard : Need to Add Sent KB/s in Statistics Report of HTML Dashboard
• Bug 60287 - Report / Dashboard : Have a new Top5 Errors by sampler table in Dashboard. Implemented by Philippe Mouawad (p.mouawad at ubik-ingenierie.com) and contributed by Ubik Load Pack (support at ubikloadpack.com)

General

• Bug 59803 - Use isValid() method from JDBC driver, if no validationQuery is given in JDBC Connection Configuration.
• Bug 57493 - Create a documentation page for properties
• Bug 59924 - The log level of XXX package is set to DEBUG if log_level.XXXX property value contains spaces, same for __log function
• Bug 59777 - Extract SLF4J binding into its own jar and make it a JMeter lib.
  If you get a warning about multiple SLF4J bindings on startup. Remove either the Apache JMeter provided binding lib/ApacheJMeter_slf4j_logkit.jar, or all of the other reported bindings. For more information you can have a look at SLF4Js own info page.
• Bug 60085 - Remove cache for prepared statements, as it didn't work with the current JDBC pool implementation and current JDBC drivers should support caching of prepared statements themselves.
• Bug 60137 - In Distributed testing when using StrippedXXXX modes strip response also on error
• Bug 60106 - Settings defaults : Switch "jmeter.save.saveservice.connect_time" to true (after 3.0)
• Pull request #229 - tiny memory allocation improvements. Contributed by Benoit Wiart (b.wiart at ubik-ingenierie.com)
• Bug 59945 - For all JSR223 elements, if script language has not been chosen on the UI, the script will be interpreted as a groovy script.
• Bug 60266 - Usability/ UX : It should not be possible to close/exit/Revert/Load/Load a recent project or create from template a JMeter plan or open a new one if a test is running
• Bug 57305 - Remove dependency of ProxyControl on GuiPackage. Based on patches by jarek102 (jarek102 at gmail.com) and Wyatt Epp (wyatt.epp at gmail.com)

Non-functional changes

• Updated to jsoup-1.10.1 (from 1.8.3)
• Updated to ph-css 4.1.6 (from 4.1.4)
• Updated to tika-core and tika-parsers 1.14 (from 1.12)
• Updated to commons-io 2.5 (from 2.4)
• Updated to commons-lang3 3.5 (from 3.4)
• Updated to commons-net 3.5 (from 3.4)
• Updated to groovy 2.4.7 (from 2.4.6)
• Updated to httpcore 4.4.5 (from 4.4.4)
• Updated to slf4j-api 1.7.21 (from 1.7.13)
• Updated to rsyntaxtextarea-2.6.0 (from 2.5.8)
• Updated to xstream 1.4.9 (from 1.4.8)
• Updated to jodd 3.7.1 (from 3.6.7.jar)
• Updated to xmlgraphics-commons 2.1 (from 2.0.1)
• Pull request #215 - Reduce duplicated code by using the newly added method GuiUtils#cancelEditing. Contributed by Benoit Wiart (b.wiart at ubik-ingenierie.com)
• Pull request #218 - Misc cleanup. Contributed by Benoit Wiart (b.wiart at ubik-ingenierie.com)
• Pull request #216 - Re-use pattern when possible. Contributed by Benoit Wiart (b.wiart at ubik-ingenierie.com)
• Bug 60364 - Document Test Coverage. Contributed by Thomas Schapitz (ts-nospam12 at online.de)

Bug fixes

HTTP Samplers and Test Script Recorder

• Bug 58888 - HTTP(S) Test Script Recorder (ProxyControl) does not add TestElement's returned by SamplerCreator#createChildren()
• Bug 59902 - Https handshake failure when setting httpclient.socket.https.cps property
• Bug 60084 - JMeter 3.0 embedded resource URL is silently encoded
• Bug 60376 - Http Test Script Recorder : If deflate is used by server then recording may break application

Other Samplers

• Bug 59113 - JDBC Connection Configuration : Transaction Isolation level not correctly set if constant used instead of numerical

Controllers

• Bug 60361 - ModuleController : If a Test plan contains a Module Controller which references an unexistant Controller, JMeter in GUI mode will not stop

Listeners

• Bug 59712 - Display original query in RequestView when decoding fails. Based on a patch by Teemu Vesala (teemu.vesala at qentinel.com)
• Bug 60278 - Since 2.13 (and Bug 57514), Aggregate Graph, Summary Report and Aggregate Report lost precision in the Error, Rate and Bandwidth values saved in the saved file csv
• Bug 60360 - View Result Tree : Request Tab does not show body of a DELETE request

Timers, Assertions, Config, Pre- & Post-Processors

• Bug 59964 - JSR223 Test Element : Cache compiled script if available is not correctly reset. Contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 59609 - Format extracted JSON Objects in JSON Post Processor correctly as JSON.
• Bug 60332 - View Results Tree : With Windows LAF, JSON Extractor does not show JSON Path Expression and Result panel

Functions

I18N

General

• Bug 59400 - Get rid of UnmarshalException on stopping when -X option is used.
• Bug 59607 - JMeter crashes when reading large test plan (greater than 2GB). Based on fix by Felix Draxler (felix.draxler at sap.com)
• Bug 59621 - Error count in report dashboard is one off.
• Bug 59657 - Only set font in JSyntaxTextArea, when property jsyntaxtextarea.font.family is set.
• Bug 59720 - Batch test file comparisons fail on Windows as XML files are generated as EOL=LF
• Code cleanups. Patches by Graham Russell (graham at ham1.co.uk)
• Bug 59722 - Use StandardCharsets to reduce the possibility of misspelling Charset names.
• Bug 59723 - Use jmeter.properties for testing whenever possible
• Bug 59726 - Unit test to check that CSV header text and sample format don't change unexpectedly
• Bug 59889 - Change encoding to UTF-8 in reports for dashboard.
• Bug 60053 - In Non GUI mode, a Stacktrace is shown at end of test while report is being generated
• Bug 60049 - When using Timers with high delays or Constant Throughput Timer with low throughput, Scheduler may take a lot of time to exit, same for Shutdown test
• Bug 60089 - Report / Dashboard : Bytes throughput Over Time has reversed Sent and Received bytes. Contributed by Ubik Load Pack (support at ubikloadpack.com)
• Bug 60090 - Report / Dashboard : Empty Transaction Controller should not count in metrics
• Bug 60103 - Report / Dashboard : Requests summary includes Transaction Controller leading to wrong percentage
• Bug 60105 - Report / Dashboard : Report requires Transaction Controller "generate parent sample" option to be checked, fix related issues
• Bug 60107 - Report / Dashboard : In StatisticSummary, TransactionController SampleResult makes Total line wrong
• Bug 60110 - Report / Dashboard : In Response Time Percentiles, slider is useless
• Bug 60135 - Report / Dashboard : Active Threads Over Time should be in OverTime section
• Bug 60125 - Report / Dashboard : Dashboard cannot be generated if the default delimiter is \t. Based on a report from Tamas Szabadi (tamas.szabadi at rightside.co)
• Bug 59439 - Report / Dashboard : AbstractOverTimeGraphConsumer.createGroupInfos() should be abstract
• Bug 59918 - Ant generated HTML report is broken (extras folder)
• Bug 60295 - JSON Extractor doesn't index array elements when only one element is found. Based on a patch by Roberto Braga (roberto.braga at sociale.it)
• Bug 60299 - Thread Group with Scheduler : Weird behaviour when End-Time is in the past

Thanks

We thank all contributors mentioned in bug and improvement sections above:

• Felix Draxler (felix.draxler at sap.com)
• Antonio Gomes Rodrigues (ra0077 at gmail.com)
• Graham Russell (graham at ham1.co.uk)
• Teemu Vesala (teemu.vesala at qentinel.com)
• Asier Lostalé (asier.lostale at openbravo.com)
• Thomas Peyrard (thomas.peyrard at murex.com)
• Benoit Wiart (b.wiart at ubik-ingenierie.com)
• Maxime Chassagneux (maxime.chassagneux at gmail.com)
• Ubik Load Pack
• Tamas Szabadi (tamas.szabadi at rightside.co)
• Roberto Braga (roberto.braga at soziale.it)
• jarek102 at gmail.com
• Wyatt Epp (wyatt.epp at gmail.com)
• Thomas Schapitz (ts-nospam12 at online.de)

We also thank bug reporters who helped us improve JMeter.
For this release we want to give special thanks to the following reporters for the clear reports and tests made after our fixes:

Apologies if we have omitted anyone else.

Known problems and workarounds

• The Once Only controller behaves correctly under a Thread Group or Loop Controller, but otherwise its behaviour is not consistent (or clearly specified).
• The numbers that appear to the left of the green box are the number of active threads / total number of threads, the total number of threads only applies to a locally run test, otherwise it will show 0 (see Bug 55510).
• Note that there is a bug in Java on some Linux systems that manifests itself as the following error when running the test cases or JMeter itself:

   [java] WARNING: Couldn't flush user prefs:
   java.util.prefs.BackingStoreException:
   java.lang.IllegalArgumentException: Not supported: indent-number
  

  This does not affect JMeter operation. This issue is fixed since Java 7b05.
• Note that under some windows systems you may have this WARNING:

  java.util.prefs.WindowsPreferences
  WARNING: Could not open/create prefs root node Software\JavaSoft\Prefs at root 0
  x80000002. Windows RegCreateKeyEx(…) returned error code 5.
  

  The fix is to run JMeter as Administrator, it will create the registry key for you, then you can restart JMeter as a normal user and you won't have the warning anymore.
• With Oracle Java 7 and Mac Book Pro Retina Display, the JMeter GUI may look blurry. This is a known Java bug, see Bug JDK-8000629. A workaround is to use a Java 7 update 40 runtime which fixes this issue.
• You may encounter the following error:

  java.security.cert.CertificateException: Certificates does not conform to algorithm constraints

  if you run a HTTPS request on a web site with a SSL certificate (itself or one of SSL certificates in its chain of trust) with a signature algorithm using MD2 (like md2WithRSAEncryption) or with a SSL certificate with a size lower than 1024 bits. This error is related to increased security in Java 7 version u16 (MD2) and version u40 (Certificate size lower than 1024 bits), and Java 8 too.
  To allow you to perform your HTTPS request, you can downgrade the security of your Java installation by editing the Java jdk.certpath.disabledAlgorithms property. Remove the MD2 value or the constraint on size, depending on your case.
  This property is in this file:

  JAVA_HOME/jre/lib/security/java.security

  See Bug 56357 for details.
• Under Mac OSX Aggregate Graph will show wrong values due to mirroring effect on numbers. This is due to a known Java bug, see Bug JDK-8065373 The fix is to use JDK7_u79, JDK8_u45 or later.
• View Results Tree may fail to display some HTML code under HTML renderer, see Bug 54586. This is due to a known Java bug which fails to parse "px" units in row/col attributes. See Bug JDK-8031109 The fix is to use JDK9 b65 or later.
• JTable selection with keyboard (SHIFT + up/down) is totally unusable with JAVA 7 on Mac OSX. This is due to a known Java bug JDK-8025126 The fix is to use JDK 8 b132 or later.