59

I'm not able to run simple JMH benchmark inside eclipse. Maven dependencies:

            <dependency>
                <groupId>org.openjdk.jmh</groupId>
                <artifactId>jmh-core</artifactId>
                <version>1.12</version>
            </dependency>
            <dependency>
                <groupId>org.openjdk.jmh</groupId>
                <artifactId>jmh-generator-annprocess</artifactId>
                <version>1.12</version>
            </dependency>

Java code:

    public class BTest {
        @Benchmark
        public void test() {
            // todo
        }
    
        public static void main(String[] args) throws RunnerException {
            Options opt = new OptionsBuilder()
                     .include(BTest.class.getSimpleName())
                      .build();
    
            new Runner(opt).run();
        }
    }

Result of run:

> Exception in thread "main" java.lang.RuntimeException: ERROR: Unable
> to find the resource: /META-INF/BenchmarkList     at
> org.openjdk.jmh.runner.AbstractResourceReader.getReaders(AbstractResourceReader.java:96)
>   at org.openjdk.jmh.runner.BenchmarkList.find(BenchmarkList.java:104)
>   at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:256)   at
> org.openjdk.jmh.runner.Runner.run(Runner.java:206)    at
> com.test.BTest.main(BTest.java:24)

Maybe the problem is, that I'm running it from eclipse.

4
  • Have you tried not running it in eclipse? Since you mentioned it, it would make sense to try it Jun 27, 2016 at 14:56
  • @WillBarnwell not yet...
    – riva
    Jun 27, 2016 at 18:47
  • here may be an answer for idea
    – NanoNova
    Jun 21, 2020 at 4:57
  • @NanoNova not working Jul 28, 2021 at 12:31

10 Answers 10

28

If anyone is using Gradle, add the jmh-gradle-plugin to your plugins block:

plugins {
    id 'java'
    id 'me.champeau.jmh' version '0.6.8'
}

Then add all of the following in your dependencies block (check the latest version of JMH on Maven here):

dependencies {
    jmh 'org.openjdk.jmh:jmh-core:1.36'
    jmh 'org.openjdk.jmh:jmh-generator-annprocess:1.36'

    // this is the line that solves the missing /META-INF/BenchmarkList error
    jmhAnnotationProcessor 'org.openjdk.jmh:jmh-generator-annprocess:1.36'
}

Then just use the following to run your benchmarks through Gradle:

./gradlew jmh

Running through your IDE

If you also want to run the benchmarks from within your IDE instead of through Gradle, you can do either of the following:

Option 1 - main method

Just use a main method with no additional configuration and your IDE will respect your annotation configuration:

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Fork(value = 1)
@Warmup(iterations = 5, timeUnit = TimeUnit.MILLISECONDS, time = 5000)
@Measurement(iterations = 5, timeUnit = TimeUnit.MILLISECONDS, time = 5000)
public class MyBenchmark {

    public static void main(String[] args) throws RunnerException {
        Options options = new OptionsBuilder()
            .include(MyBenchmark.class.getSimpleName())
            .build();
        new Runner(options).run();
    }

    // benchmarks omitted
}

Option 2 - Install JMH Plugin

If you're using IntelliJ, install the JMH Java Microharness Benchmark Plugin from the Preferences > Plugins section, then you can omit your main method altogether and IntelliJ will give you a run button next to your class name:

JMH benchmark in IntelliJ

3
  • Why doesn't the plugin just set these things for you?
    – Chad
    Sep 8, 2022 at 22:54
  • jmhAnnotationProcessor is not required. If fact, none of the things shown in the dependencies block are required unless you want to change the version. Dec 6, 2022 at 21:52
  • 1
    As opposed to previous comment, without those dependencies, the JMH plugin for intelliJ produces the errors listed in the question Nov 13, 2023 at 15:54
16

Finally found it out. There was a problem with missing exec-maven-plugin plugin

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <executions>
        <execution>
            <id>run-benchmarks</id>
            <phase>integration-test</phase>
            <goals>
                <goal>exec</goal>
            </goals>
            <configuration>
                <classpathScope>test</classpathScope>
                <executable>java</executable>
                <arguments>
                    <argument>-classpath</argument>
                    <classpath />
                    <argument>org.openjdk.jmh.Main</argument>
                    <argument>.*</argument>
                </arguments>
            </configuration>
        </execution>
    </executions>
</plugin>
1
9

pom.xml must have the below dependencies and configurations to Java Micro-benchmark Harness (JMH) Framework

<properties>
    <jmh.version>1.21</jmh.version>
</properties>
<dependencies>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>${jmh.version}</version>
</dependency>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>${jmh.version}</version>
</dependency>
<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.11</version>
  <scope>test</scope>
</dependency>
</dependencies>
<build>
<finalName>java-jmh</finalName>
<plugins>
    <plugin>    
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
            <source>1.8</source>
            <target>1.8</target>
            <annotationProcessorPaths>
                <path>
                    <groupId>org.openjdk.jmh</groupId>
                    <artifactId>jmh-generator-annprocess</artifactId>
                    <version>${jmh.version}</version>
                </path>
            </annotationProcessorPaths>
        </configuration>
    </plugin>
    <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <executions>
            <execution>
                <id>run-benchmarks</id>
                <phase>integration-test</phase>
                <goals>
                    <goal>exec</goal>
                </goals>
                <configuration>
                    <classpathScope>test</classpathScope>
                    <executable>java</executable>
                    <arguments>
                        <argument>-classpath</argument>
                        <classpath />
                        <argument>org.openjdk.jmh.Main</argument>
                        <argument>.*</argument>
                    </arguments>
                </configuration>
            </execution>
        </executions>
    </plugin>
</plugins>

After this go to command line and run the command $mvn clean install

5

This could happen when your compiler plugin has not processed the JMH related annotations. For me, Gill's answer with the maven-compiler-plugin's <annotationProcessorPaths> update worked.

4

I realized that I already had exec-maven-plugin in my parent pom as mentioned in expected answer but I had to run mvn clean install as mentioned in https://stackoverflow.com/a/40748670 to fix the error

4

Having had the same error; and running the tests from maven or intellij didn't work. I realised that the problem was that I wrote the benchmark in Kotlin. Changing the code to java sorted the issue.

1
  • 2
    I banged my head against this error for hours trying to write a @Benchmark in Kotlin to no avail. Once I flipped to Java it all worked.
    – Technetium
    Aug 1, 2022 at 23:30
0

Add version as well. This works for me

        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>3.0.0</version>
0

If you are getting this error using the Intelij Idea's JMH Java Microharness Benchmark Plugin and you also using Gradle try changing the default running task from Gradle to Intelij Idea. To do this:

Settings -> Build, Execution, Deployment -> Build Tools -> Gradle

Change Build and Run using to Intelij IDEA

0

I tried all the above solution. I ended using jmh-maven-plugin:

<plugin>
    <groupId>pw.krejci</groupId>
    <artifactId>jmh-maven-plugin</artifactId>
    <version>0.2.2</version>
</plugin>

and I run it with:

mvn clean install jmh:benchmark

It seems that it does not require exec-maven-plugin, maven-compiler-plugin or maven-shade-plugin.

Notice that the benchmarks must be in the test package.

0

Here's everything I had to add to pom.xml fix mine.
This snippet ended up pretty close to Ishaq's answer minus a few extras and name-specifics.
I hope this is just a cleaner and easier to readily-use answer.

  <properties>
    <jmh.version>1.36</jmh.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.openjdk.jmh</groupId>
      <artifactId>jmh-core</artifactId>
      <version>${jmh.version}</version>
    </dependency>
    <dependency>
      <groupId>org.openjdk.jmh</groupId>
      <artifactId>jmh-generator-annprocess</artifactId>
      <version>${jmh.version}</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.2.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <finalName>benchmarks</finalName>
              <transformers>
                <transformer
                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                  <mainClass>org.openjdk.jmh.Main</mainClass>
                </transformer>
              </transformers>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.