Java JMH Benchmark Tool

Benchmarking tool can be used to find for example you have two code implementations and wanna know what is the best implementation performance wise.

Photo by Markus Spiske on Unsplash

Create a simple Maven project and add below dependency to add JMH

<dependency>
  <groupId>org.openjdk.jmh</groupId>
  <artifactId>jmh-core</artifactId>
  <version>1.27</version>
</dependency>
<dependency>
  <groupId>org.openjdk.jmh</groupId>
  <artifactId>jmh-generator-annprocess</artifactId>
  <version>1.27</version>
  <scope>provided</scope>
</dependency>

Let’s say your main class name is App and you wanna benchmark below method

public void testMethod() {
        System.out.println( "Hello World!" );
    }

You can annotate it with

@Benchmark

Now to execute the benchmark you can add the instructions to main method

Options opt = new OptionsBuilder()
        .include(App.class.getSimpleName())
        .build();
new Runner(opt).run();

You may need to throw RunnerException when executing this.

If you run the class now output will show some information how the test is been carried out. As we didn’t say any it will load the default values.

# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: com.nuwan.fastloging.benchmark.App.testMethod
# Fork: 1 of 5

You can change theme as you like. For example if you wanna change Benchmark Mode to average time you can set it here,

Options opt = new OptionsBuilder()
                .include(App.class.getSimpleName())
                .mode(Mode.AverageTime)
                .build();

So you can change above default like this

Options opt = new OptionsBuilder()
                .include(App.class.getSimpleName())
                .mode(Mode.AverageTime)
                .warmupIterations(1)
                .warmupTime(TimeValue.microseconds(2))
                .measurementIterations(1)
                .forks(1)
                .measurementTime(TimeValue.microseconds(2))
                .build();

Mode

.mode(Mode.AverageTime)

There are 5 modes

Get Nuwan Zen’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Throughput, Average time, Sample time, Single shot time, All

Warmup

.warmupIterations(1)
.warmupTime(TimeValue.microseconds(2))

How many iterations to execute or how many times to do the warmup. And duration of a iteration.

Measurement

.measurementIterations(1)
.measurementTime(TimeValue.microseconds(2))

How many iterations to execute or how many times to do the test. And time duration of the iteration.

Fork

.forks(1)

Will use seperate jvm processors to get even more accurate resutls. If measurementIterations() value is set to x and if fork is set to y then total number of tests done will be x*y

NOTE

Should note that we aren’t going count and limit how many times our method is been executed instead we will run our method as much as we can for the given time frame and that will considered as an iteration.

Final code

public class App 
{
    public static void main( String[] args ) throws RunnerException
    {
        Options opt = new OptionsBuilder()
                .include(App.class.getSimpleName())
                .mode(Mode.AverageTime)
                .warmupIterations(1)
                .warmupTime(TimeValue.microseconds(2))
                .measurementIterations(1)
                .forks(1)
                .measurementTime(TimeValue.microseconds(2))
                .build();
        new Runner(opt).run();

    }

    @Benchmark
    public void testMethod() {
        System.out.println( "Hello World!" );
    }

}

Output

# JMH version: 1.27
# VM version: ...
# VM invoker: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
# VM options: ...
# JMH blackhole mode: full blackhole + dont-inline hint
# Warmup: 1 iterations, 2 us each
# Measurement: 1 iterations, 200 us each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.nuwan.fastloging.benchmark.App.testMethod
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 1
# Warmup Iteration   1: Hello World!
0.002 s/op
Iteration   1: Hello World!
Hello World!
Hello World!
≈ 10⁻⁴ s/op
Result "com.nuwan.fastloging.benchmark.App.testMethod":
  ≈ 10⁻⁴ s/op
# Run complete. Total time: 00:00:00
REMEMBER: ...
Benchmark       Mode  Cnt   Score   Error  Units
App.testMethod  avgt       ≈ 10⁻⁴           s/op

ERROR

ERROR: Unable to find the resource: /META-INF/BenchmarkList

If you see above error its an issue with dependency, try to reload the maven dependencies.

My other writings:

Java Blogs