https://spec.org/jvm2008/docs/UserGuide.html#UsePJA
Version 1.0
Last modified: April 16, 2008
This document is a practical guide for setting up and running SPECjvm2008. To submit SPECjvm2008 results the benchmarker must adhere to the rules contained in the Run and Reporting Rules document contained in the kit.
This document is targeted at people trying to run the SPECjvm2008 benchmark in order to accurately measure their Java system, comprising a JRE and an underlying operating system and hardware.
java - The term java is used below in examples and in this context it refers to the launcher used to invoke the Java Virtual Machine.The main purpose of SPECjvm2008 is to measure the performance of a JRE (a JVM and associated libraries). It also measures the performance of the operating system and hardware in the context of executing the JRE. It focuses on the performance of the JRE executing a single application; it reflects the performance of the hardware processor and memory subsystem, but has minimal dependence on file I/O and includes no remote network I/O.
The SPECjvm2008 workload mimics a variety of common general purpose application computations. These characteristics reflect the goal that this benchmark be applicable to measuring basic Java performance on a wide variety of both client and server systems running Java.
SPEC also finds user experience of Java important and the suite therefore includes startup benchmarks and has a required run category called base, which has to be run without any tuning of the JVM to improve the out of the box performance.
Other SPEC benchmarks are available for measuring the performance of Java systems in more specialized enterprise scenarios. SPECjbb2005 is a Java program emulating a 3-tier system with emphasis on the middle tier. For a full-fledged multi-tier Java benchmark, SPEC has developed a comprehensive application server benchmark, SPECjAppServer2004. SPEC also has a Java messaging benchmark called SPECjms2007, which is the first industry-standard benchmark for evaluating the performance of enterprise message-oriented middleware servers based on JMS (Java Message Service)
SPECjvm2008 comprises a collection of workloads intended to represent a diverse set of common types of computation. In general, the algorithms and operations in the workloads are components of real-world applications and include text/character processing, numerical computations, and bitwise computation (e.g., media processing). Each of the workloads has a specific amount of work to do, making them each a small benchmark in itself and several of the benchmarks have sub-benchmarks.
The benchmarks are described in detail in the benchmark pages.
There are two run categories in SPECjvm2008 called Base and Peak and there is also an additional run category called Lagom. In order to create a compliant result, a run in the Base category must be included. It is optional to also include a run in the Peak category.
The Base category shows the performance of the system without any tuning of the JVM, the 'out of the box' performance. It does however allow tuning of the OS and hardware (including firmware like BIOS). The Base category has the limitation that you are not allowed to do any hand tuning of the JVM and you are not allowed to change the run time.
The Peak category shows what can be achieved with the system. Tuning of the JVM is allowed to achieve optimal performance.
The Base run and the Peak run are done with two seperate invocations of the SPECjvm2008 benchmarrk suite, and initially two different results are created. The results from these are later combined into one raw file, in order to submit.
The Lagom workload is a fixed size workload, meaning it does a certain number of operations of each benchmarks. This is a complement to the base and peak categories and can be used when it is preferred to run a fixed amount of work. The workload will run a specified number of operations (different for each benchmark) in each benchmark thread. The number of benchmark threads will by default be adjusted to the number of hardware threads the machine has, so in order to have an exact amount of work on two different systems of different size, the number of benchmark threads should be set.
The Lagom workload is intended for research usage, to be used as a tool to measure the progres, overhead, or whatever the intent of the research may be. A run of the Lagom category will not be a compliant run and SPEC will not review or post any of this results on the spec.org web.
Each invocation of a benchmark workload is one operation. The harness will call a benchmark several times, making it perform multiple operations in one iteration.
An iteration goes on for a certain duration, by default 240 seconds. During this time the harness will kick off several operations, one new one as soon as previous operation completed. It will never abort an operation, but wait until an operation is completed for stopping. The harness expects to complete at least 5 operations inside an iteration. The duration for an iteration is never less than specified time, but will be increased if the operations takes too long, based on performance in the warmup period.
The first iteration is a warmup iteration, run for 120 seconds by default. The result of the warmup iteration is not included in the benchmark result. To skip warmup, set the warmup time to 0.
Most of the benchmarks are run in parallel, where several operations are started at the same time in separate threads. From a harness point of view the threads work independently but the work loads are designed to introduce an interesting mix of problems, both by sharing data and work in the application level as well as using resources shared inside the JVM.
It is possible to use the SPECjvm2008 benchcmark harness in order to analyze what happens during a run, in order to understand and diagnose a product. An example would be to view the heap usage during a benchmark run. In order to achieve this, the framework can run one or several Analyzers during the benchmark run. Those will gather information and deliver the results together with the benchmark results. It will plot information on the details chart in the report, where each benchmark operation is plotted, and it can report a summary metric for each iteration. The analyzers can either poll for information during the benchmark run, or implement a callback method to report results based on events. Here is recommended further reading for configuration and implementation of analyzers.
The startup benchmarks measure JVM and application startup performance. The ops/m metric is calculated using the time taken for each workload to be run for one operation by a newly started JVM (via java.lang.Runtime.exec())). This tests both basic JVM startup and also the time to startup the benchmark workload, since in order to get best score in several of the benchmarks it is critical to optimize hot areas of code.
It is possible to configure both launcher and arguments for the startup benchmarks, see the Operational Configurations section.
The SciMark workloads are run with both small and large dataset sizes. The small dataset fits within the L2 cache available on most modern CPU architectures and is intended to test JVM code optimization and computation performance while ensuring the dataset is accessible in cache alone. The large dataset is large enough to not fit within a standard L2 cache and is intended to test JVM optimizations targeted at memory and the performance of the memory subsystem itself. The scimark.monte_carlo workload is run only once as it does not use a dataset for calcuation that can be modified.
SPECjvm2008 can be run on a system with only one logical CPU and 512 MB of memory.
The recommended minimum amount of disk space needed is 256 MB (including installation).
SPECjvm2008 requires a Java Runtime Environment supporting Java SE 5.0 features.
SPECjvm2008 has been tested on a large set of Hardware, OS and JRE combinations.
See the FAQ for details on what configurations are tested.
Note: SPECjvm2008 is designed to scale up the workload when a larger machine (determined by the number of logical CPUs) is used. In many of the work loads included in the suite, the amount of live data will be increased when the workload is increased and minimum amount of memory mentioned above will not be enough. More space on disk will also is required (since the derby benchmark stores data on disk).
The benchmark kit is distributed with an installer or in a zip file.
It is recommended to install the benchmark on the system under test, but it can be installed on and run from a file share. The installed benchmark suite will use about 150 MB of file resources.
To install using the installer, invoke the installer and follow the instructions, which will ask where the benchmark should be installed.
To install using the zip file, unzip it in a folder where the benchmark is to be installed. The zip file will create the sub-folder SPECjvm2008 where into which everything will be installed.
Note: Although the Java source code for the benchmark is provided in the kit, it is NOT necessary to recompile the Java code; the .jar files in the kit have everything required. In fact, rebuilt .jar files are not allowed to be used for compliant runs and they will fail the checksum tests. You may, of course, modify and recompile the benchmark for research purposes.
Note:If you are using Java for Mac OS X, see the Known Issues document for how to prepend javac.jar.
The following command line may be used to quickly check the installation.
It should be executed from a shell or command window and should be executed where the benchmark is installed, where the SPECjvm2008.jar file is located.
It is expected to complete within a few minutes.
It is expected to:
java -jar SPECjvm2008.jar -wt 5s -it 5s -bt 2 compress
Note: If there is no "java" command in your path, the "java" above would have to replaced by the full path to the java command, e.g. "d:\myjava\jre\bin\java.exe"
* Note: The check benchmark will verify that the javac version used by the JRE is from javac.jar included in the benchmark. If this verification fails, see the known issues document for a workaround, if it is the same issue already found.
Beginning of the output:SPECjvm2008 Peak Properties file: none Benchmarks: compress WARNING: Run will not be compliant. Not a compliant sequence of benchmarks for publication. Property specjvm.iteration.time must be at least 240 seconds for publication. ...
There are a number of parameters that control the operation of the SPECjvm2008 benchmark. Each parameter has a default value; the user may override these defaults by specifying parameter values in a properties file and/or on the command line.
The SPECJvm2008 harness is very flexible when it comes to tuning the workloads in order to be able to work with the benchmarks as effectively as possible. The complete set of parameters for SPECjvm2008 is documented in Appendix A and also in the files props/specjvm.properties and props/specjvm.reporter.properties.
The user may specify values for SPECjvm2008 parameters on the command line as arguments or as properties in a properties file. In general, the effective value of a parameter is determined as if by the following procedure:
If one or more command line options specify a value for the parameter, then the effective value is the one specified in the last such option; otherwise
if a properties file is loaded and one or more lines in the properties file specify a value for the parameter, then the effective value is the one specified by the last such line; otherwise
the effective value is the default value for the parameter.
As described in the next section, the parameter specjvm.propfile is an exception to this rule.
Whether a properties file is loaded is determined by what value, if any, for the parameter specjvm.propfile is specified on the command line. If no value for specjvm.propfile is specified on the command line, then no properties file is loaded. If one or more command line options specify a value for specjvm.propfile, then the value specified in the last such option, PROPFILE, is used as follows.
SPECJVM_HOME denotes the root of the directory subtree containing the SPECjvm2008 kit.
The properties file is a list of attribute-value pairs, specified one per line in the following form:
<parameter name>=<value>
Blank lines and lines beginning with the character '#' are ignored. Conversions are performed according to the type of value expected for each parameter.
Any SPECjvm2008 parameter may be set by using the -D command line option. In addition, specific command line options exist to make setting many of the more commonly used parameters more convenient. See Appendix A for details on command line options.
To assist testers and submitters in assembling all the pertinent information needed for reproducing SPECjvm2008 results, the benchmark defines a number of system attributes which are given values in the same way as operational parameters and the reporting mechanism automatically includes this information in the various reports it generates. The names of these system information properties are easy to distinguish, as the all begin with the prefix spec.jvm2008.reporter. It is important to note two things about these properties.
Although they are given values via the same mechanism as the operational parameters, SPECjvm2008 treats them very differently. They have no effect on the operation of the benchmark; they are simply passed through to the results reports. Except for the labeling supplied for them in the results reports, SPECjvm2008 does not interpret their values.
SPECjvm2008 is capable of automatically detecting and filling in values for a few of these properties, but for the most part it is up to the user to supply these values. In any case, the user should check them all for correctness.
Whereas it is common that all the operation parameters that a user wants to change are easily supplied via command line options, supplying all the required system information on the command line (while possible) would be very cumbersome. Therefore, the expected usage model is that these are supplied via a properties file.
An example properties file for this purpose is included as props/specjvm.reporter.properties. In addition, SPECjvm2008 has a mechanism that supports keeping separate property files for operational parameters and system information: if (as the result of command line options or the loading of a first property file) the operational parameter specjvm.additional.properties.file has a nonnull value, then SPECjvm2008 uses that value as the name of another property file to be loaded.
SPECjvm2008 runs as a single Java application on a single system. It understands a number of command line switches, though none of them are required. However, some of the workloads require heap space that is greater than the default maximum heap size for some particular JREs on particular systems. The benchmark will increase the workload (and the size of the live data) based on the number for CPUs (number of hardware threads). For all the JREs we have tested, 400 MB is sufficient for a system with 2 hardware threads. So a simple command line to run the benchmark is
java -Xmx400m -jar SPECjvm2008.jar
The general form of the command line to run the benchmark is
java [<jvm options>] -jar SPECjvm2008.jar [<SPECjvm2008 options>] [<benchmark name> ...]
Appendix A lists the valid SPECjvm2008 options and their meanings and the valid benchmark names.
For a run to be compliant, the SPECjvm2008 configuration, specified on the command line and/or in properties files must result in parameters having values conforming to the the run rules, section 2.3 and 2.4. The SPECjvm2008 framework checks the conformance of the parameter values at start of suite and indicates in the benchmark output any parameter values that do not pass these checks (which makes the run noncompliant).
For a run to be compliant it must also be valid, passing the result validations done by the SPECjvm2008 harness as part of each operation.
For a compliant run to be submitted for the base metric, the run must not use any jvm options or harness time tuning.
To run the base category, specify --base on command line. This category is default unless JVM command line arguments are used or run time is changed.
A compliant run submitted for the peak metric may use jvm options. The options used must be disclosed as described in the Run Rules.
This disclosure is done using the property spec.jvm2008.report.jvm.command.line.
To run the peak category, specify --peak on command line. This will also be selected automatically if JVM command line arguments are used or run time is changed.
SPECjvm2008 prints a record of its operation to the standard output stream as it runs. It optionally produces an XML file that records system information supplied by the user, some parameter values and results for the run. This XML file is required for submitting the results to SPEC, and the default behavior is that this XML file is produced.
The location to which the XML file is written is controlled by the value of the parameter specjvm.result.dir; the default is a subdirectory named “results” in the current working directory. In the following description, RESULTS_DIR denotes the value of specjvm.results.dir. If the XML file is to be created, SPECjvm2008 creates a subdirectory of RESULTS_DIR named SPECjvm2008.<num>, where <num> is the smallest positive integer that will result in a unique subdirectory name within RESULTS_DIR. The XML file is then written as a file named
RESULTS_DIR/SPECjvm2008.<num>/SPECjvm2008<num>.raw
Optionally, the SPECjvm2008 reporter will run at the end of the run and create text and/or HTML versions of the information in the XML results file. These files are written to the same directory as the XML results file. A more detailed description of the results files appears in the section on Results Reports.
Normally, the reporter is executed automatically by SPECjvm2008 at the end of a run to process the results file generated by that run. The reporter may also be run alone by using a command line of this form:
java -jar SPECjvm2008.jar --reporter <file name>
where <file name> is the name of a results file produced by a previous run. As in a regular run, which results file formats are created is controlled by the properties specjvm.create.html.report and specjvm.create.txt.report.
In a given run, each SPECjvm2008 sub-benchmark produces a result in ops/min (operations per minute) that reflects the rate at which the system was able to complete invocations of the workload of that sub-benchmark. At the conclusion of a run, SPECjvm2008 computes a single quantity intended to reflect the overall performance of the system on all the sub-benchmarks executed during the run. The basic method used to compute the combined result is to compute a geometric mean. However, because it is desired to reflect performance on various application areas more or less equally, the computation done is a little more complex than a straight geometric mean of the sub-benchmark results.
In order to include multiple sub-benchmarks that represent the same general application area while still treating various application areas equally, an intermediate result is computed for certain groups of the sub-benchmarks before they are combined into the overall throughput result. In particular, for these groups of sub-benchmarks
COMPILER: compiler.compiler, compiler.sunflow
CRYPTO: crypto.aes, crypto.rsa, crypto.signverify
SCIMARK: scimark.fft.large, scimark.lu.large, scimark.sor.large, scimark.sparse.large, scimark.fft.small, scimark.lu.small, scimark.sor.small, scimark.sparse.small, scimark.monte_carlo
STARTUP: {all sub-benchmarks having names beginning with startup. } See Appendix A for the complete list.
XML: xml.transform, xml.validation
the geometric mean of sub-benchmark results in each group is computed. The overall throughput result is then computed as the geometric mean of these group results and the results from the other sub-benchmarks.
While throughput results obtained from runs that execute only selected sub-benchmarks may be interesting and useful for research purposes, only the overall throughput results from runs that execute all the sub-benchmarks (and satify several other conditions as well) are deemed to represent the performance of the system on SPECjvm2008 per se. The conditions under which an overall throughput result represents a SPECjvm2008 metric is discussed in detail in the "Run and Reporting Rules".
The benchmark results reports for a single run are written to a single results directory as described above. The results files include some or all of the following.
SPECjvm2008.<num>.raw
SPECjvm2008.<num>.html
SPECjvm2008.<num>.txt
SPECjvm2008.<num>.sub
SPECjvm2008.<num>.summary
If the HTML results file exists, there will also be an images subdirectory containing .jpg files that are referenced by the HTML results file.
The XML results file (.raw) is generated from data in the benchmark's internal data structures at the end of a run; it is intended primarily for consumption by other software, and it is the form required for submission to SPEC for review and publication. The HTML and text reports are generated from the XML results file and are intended primarily for viewing by humans. All of them contain substantially the same information.
The results reports contain the system information supplied by the user, summary information about the run and detailed information about the run.
The system information in the results reports is categorized as general information about the run (such as time and date), information about the JRE on which the benchmark was run, information about the OS, and information about the hardware. All of the system information properties that SPECjvm2008 is capable of reporting are documented in the example reporter properties file (props/specjvm.reporter.properties).
The summary information includes the score on each workload, the composite score, and, if there were any, violations that make the run noncompliant. For a compliant run, the composite score represents a value of the SPECjvm2008 metric; for a noncompliant run the composite score is just a number that may or may not be interesting for research purposes.
For each workload executed in the run, the results report includes the values of the operational parameters in effect for that workload, the score for each iteration (or an indication that the iteration did not complete successfully), and the start and end times for each execution of the workload by each thread.
Most of the information in the HTML report is presented in a single main page. However, below the graph of the scores by iteration for each workload, there is a link named “details”. Clicking on this link will display a graph showing the performance of individual threads for that workload.
Recommended steps for producing a compliant result:
java -jar SPECjvm2008.jar --base --propfile props/specjvm.properties
java -Xms3000m -Xmx3000m -jar SPECjvm2008.jar --peak --propfile props/specjvm.properties
Here are the steps for submitting results:
java -jar SPECjvm2008.jar --reporter --prepare <base raw file> <optional peak raw file>This will first produce a new raw file, which is a merged version of previous files.
java -jar SPECjvm2008.jar --reporter --specprocess <zip file>This should pass without complaints and create a brief summary report, which links to the full reports for each run in subfolders.
To control SPECjvm2008 behavior you can use SPECjvm2008 options. These options can be specified either on the command line or in a property file.
To specify option by a property file use:
java -jar SPECjvm2008.jar -pf <your_file_name>
This will let the suite know what property file should be used. Desired property should be set in this property file.
To specify option by command line use:
java -jar SPECjvm2008.jar -D<property_name>=<property_value>
For the most of frequently used options simpler way can be used, use '-help' option, to see short names.
Properties specified on the command line override properties specified in a property file, even if a propoerty file specification goes after command line options. So, if props.file contains line props_a=value_b:
java -jar SPECjvm2008.jar -Dprops_a=value_c -pf props.file
the property props_a will be set to value_c.
For the benchmark specific properties, you can add benchmark name to use this property only for a certain benchmark, for example:
java -jar SPECjvm2008.jar -Dspecjvm.benchcmark.threads.scimark.monte_carlo=3 -Dspecjvm.benchmark.threads=2 all
This will run 2 benchmark threads for all benchmarks except scimark.monte_carlo which will have 3 and the startup benchmarks which always run the workload single threaded.
The suite uses the number of available logical CPUs to compute the number of benchmark threads to use for each benchmark. Some benchmarks scale the number of benchmark threads, for example the sunflow benchmark uses only half as many benchmark threads as the number of available logical CPUs, since each benchmark instance (thread) kicks off four threads internally. It is possible to override the number of available logical CPUs using the property specjvm.hardware.threads.override. Overriding it will leave the existing suite computations based on the new value and scale the workload. This therefore differs from using the argument -bt.
To disable checksum verification of the kit, use the '-ikv' option. This can be used for testing purposes, but can not be used in a compliant run.
It's possible to set the number of iterations, the iteration time and the warmup time.
To set the number of iterations, use the '-i <iter_num>' option. With this option the suite will run one warmup iteration and then <iter_num> iterations. If <iter_num> is set to -1, it will continue to run an infinite number of iterations.
To specify the duration of the warmup phase and an iteration one can use '-it ' option to specify the iteration time and '-wt ' to specify the warmup time.
So, for example:
java -jar SPECjvm2008.jar -i 4 -wt 17 -it 4711
will run 4 iterations with 17 seconds of warmup and each iteration will run for 4711 seconds.
Another example is:
java -jar SPECjvm2008.jar -i -1 -wt 0 -it 20
which will skip the warmup phase, then run an infinite number of iterations, 10 seconds each.
SPECjvm2008 results will by default be stored in a result folder, based on where the execution takes place. It is possible to redirect the results using the property specjvm.result.dir property. Example:
java -Dspecjvm.result.dir=/home/results/jvm08-results/ -jar /home/tests/SPECjvm2008/SPECjvm2008.jar
SPECjvm2008 can be run from any directory; however, specjvm.home.dir must be specified as a system property and point to the SPECjvm2008 location (where SPECjvm2008.jar is located). Example:
java -Dspecjvm.home.dir=/home/tests/SPECjvm2008 -jar /home/tests/SPECjvm2008/SPECjvm2008.jar
In the above example, the results will be produced where you execute, but can be controlled with property specjvm.result.dir. Example:
java -Dspecjvm.home.dir=/home/tests/SPECjvm2008 -Dspecjvm.result.dir=/home/results/jvm08-results/ -jar /home/tests/SPECjvm2008/SPECjvm2008.jar
By default the harness will produce a raw file (in xml format) with the results from the benchmark runs. The result is stored after each iteration, in order not to have an impact on the measurement period, but also done continuously in order to not store any extra data between benchmarks or even between iterations, which could affect the benchmark result.
After the full suite is run, the reporter will be invoked and produce an html report, a text report and a little summary file. In order to skip generating these reports, the commands '-ctf false' will skip the text report and '-chf false' will skip the html report.
In order to skip generating results over all, the command '-crf false' can be used in order not to print any results to file. This means that there will be no raw file, and it will not be possible to post-generate any html report or text report.
By default no analyzers will be run. In order to run one or more, set the property specjvm.benchmark.analyzer.names to contain which analyzer(s) to run. If more than one is specified, seperate with a blank space. Use property specjvm.benchmark.analyzer.frequency to control how often the analyzers will run when polling. The name of an analyzer is the same as the name of the Analyzer class.
An analyzer is a class that extends the class AnalyzerBase and has the package name spec.harness.analyzers. It must implement the method execute(). If the analyzer rather should listen to events than poll, it is recommended to implement an empty execute method(). In order to store a result, use the report method. To store a result that should be plotted in the graph over the run, pass a TYInfo (Time-axis, Y-axis) object. To store a summary result (recommended to do in the tearDown method) for the iteration, pass an AnalyzerResult object.
To implement your own, see this example of a polling analyzer and this for an event-based or listening callback-based analyzer.
The harness can be configured to use a property file for reporter information about the system. It is recommended to do so in an additional file, compared to the harness configuration and the additional file is specified by using the property specjvm.additional.properties.file. This properties file usually includes the JRE arguments. It is possible to parse the command line arguments can from the information available in the Runtime JMXBean, including parsing the initial and mximum heap settings. THis is done if the -pja option is specified on the command line for the harness. There is however no standard on what is passed in by the launcher and then reported in this field. Some JVM also includes properties information other than what is specified on command line, but by the launcher. So use this option as shorthand when possible.
specjvm.verify property.java -jar SPECjvm2008.jar compilerwill run the compiler.compiler and the compiler.sunflow benchmarks.
java -jar SPECjvm2008.jar --help
| Arg | Long arg | Value | Property name | Description |
|---|---|---|---|---|
| -h | --help | Show this help. | ||
| --version | Print SPECjvm2008 version and exit. | |||
| -sv | --showversion | Print SPECjvm2008 version and continue. | ||
| --base | Run the base compliant run of SPECjvm2008 (default, unless jvm args are specified). | |||
| --peak | Run the peak compliant run of SPECjvm2008. | |||
| --lagom | Run the Lagom benchmark suite, a version, of SPECjvm2008 that uses a fixed workload. | |||
| -pf | --propfile | string | specjvm.propfile | Use this properties file. |
| -i | --iterations | int | specjvm.miniter, specjvm.maxniter | How many iterations to run. 'inf' means an infinite number. |
| -mi | --miniter | int | specjvm.miniter | Minimum number of iterations. |
| -ma | --maxiter | int | specjvm.maxniter | Maximum number of iterations. |
| -it | --iterationtime | time | specjvm.iteration.time | How long one iteration should be. The time is specified as an integer, and assumed to be in seconds, or an integer with unit, for example 4m (4 minutes). Units available are ms, s, m and h. If the iteration time is too short, based on the warmup result, it will be adjusted to expect to finish at least 5 operations. |
| -fit | --forceIterationIime | time | specjvm.iteration.time, specjvm.iteration.time.forced | As iteration time, but the time will not be adjusted based on the warmup result. |
| -ja | --jvmArgs | string | specjvm.startup.jvm_options | JVM options used for startup subtests. |
| -jl | --jvmLauncher | path | specjvm.benchmark.startup.launcher | JVM launcher used for startup subtests. |
| -wt | --warmuptime | time | specjvm.benchmark.warmup.time | How long warmup time. The time format is the same as in iteration time. |
| -ops | --operations | int | specjvm.fixed.operations, specjvm.run.type | How many operations each iteration will consist of. It will then be a fixed workload and iteration time is ignored. |
| -bt | --benchmarkThreads | int | specjvm.benchmark.threads | How many benchmark threads to use. |
| -r | --reporter | raw file name | Invokes the reporter with given file(s). The benchmarks will not be run. | |
| -v | --verbose | specjvm.print.verbose, specjvm.print.progress | Print verbose info (harness only). | |
| -pja | --parseJvmArgs | Parse jvm arguments info from command line, including heap settings (uses JMXBean info). This is not done by default. | ||
| -coe | --continueOnError | specjvm.continue.on.error | Continue to run suite, even if one test fails. | |
| -ict | --ignoreCheckTest | specjvm.run.initial.check | Do not run check benchmark. | |
| -ikv | --ignoreKitValidation | specjvm.run.checksum.validation | Do not run checksum validition of benchmark kit. | |
| -crf | --createRawFile | boolean | specjvm.create.xml.report | Whether to generate a raw file. |
| -ctf | --createTextFile | boolean | specjvm.create.txt.report | Whether to generate text report. If raw is disabled, so is txt. |
| -chf | --createHtmlFile | boolean | specjvm.create.html.report | Whether to generate html report. If raw is disabled, so is html. |
| -xd | --xmlDir | path | specjvm.benchmark.xml.validation.input.dir | To set path to xml input files |
| <benchmark(s)> | specjvm.benchmarks | Name of benchmark(s) to run. By default all submission benchmarks will be selected. 'all' means all sumission benchmarks will be run. See SPECjvm2008 workload names for all values. |
| startup.helloworld | compiler.compiler | scimark.fft.small |
| startup.compiler.compiler | compiler.sunflow | scimark.lu.small |
| startup.compiler.sunflow | compress | scimark.sor.small |
| startup.compress | crypto.aes | scimark.sparse.small |
| startup.crypto.aes | crypto.rsa | scimark.monte_carlo |
| startup.crypto.rsa | crypto.signverify | serial |
| startup.crypto.signverify | derby | sunflow |
| startup.mpegaudio | mpegaudio | xml.transform |
| startup.scimark.fft | scimark.fft.large | xml.validation |
| startup.scimark.lu | scimark.lu.large | |
| startup.scimark.monte_carlo | scimark.sor.large | |
| startup.scimark.sor | scimark.sparse.large | |
| startup.scimark.sparse | ||
| startup.serial | ||
| startup.sunflow | ||
| startup.xml.transform | ||
| startup.xml.validation |