6

This is a weird error. After adding the selenium dependencies to the pom of my maven project and upload it to a lambda, it says it is unable to unzip the file. However after removing the dependencies, the lambda is able to unzip the file just fine (however it comes up with a class not found afterwards). I have tried removing the dependencies one by one but each one triggers the error.

Any ideas on how to solve this?

Class not found error

org/openqa/selenium/WebDriver: java.lang.NoClassDefFoundError
java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver

lambda cannot zip error

Calling the invoke API action failed with this message: Lambda was not able to unzip the file

The dependencies causing the issue

    <dependency>
        <groupId>org.seleniumhq.webdriver</groupId>
        <artifactId>webdriver-common</artifactId>
        <version>0.9.7376</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-chrome-driver</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.141.59</version>
    </dependency>

updated dependancies (for Vishal)

    <dependency>
        <groupId>org.seleniumhq.webdriver</groupId>
        <artifactId>webdriver-common</artifactId>
        <version>0.9.7376</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-chrome-driver</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-api</artifactId>
        <version>2.0rc2</version>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-remote-driver</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-support</artifactId>
        <version>3.141.59</version>
    </dependency>

Configuration

 <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.6.0</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
          <encoding>UTF-8</encoding>
          <forceJavacCompilerUse>true</forceJavacCompilerUse>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
6
  • I have now added the jars as separate dependencies and still getting the issue Commented Oct 22, 2019 at 8:25
  • I think the chromedriver dependency is messing it up. Can you try without the chrome driver once? If it works then try to change the version of the chrome driver dependency
    – Vishal
    Commented Oct 23, 2019 at 2:53
  • @Vishal it still comes up with the unzip error with or without the chrome dependancy Commented Oct 23, 2019 at 10:03
  • Try including this in your pom: Selenium-api
    – Vishal
    Commented Oct 23, 2019 at 19:17
  • You will likely also need: selenium-remote-driver-<version>.jar and probably also selenium-support-<version>.jar, in addition to the selenium-java-<version>.jar you already have. But try with the above one first. As there are transitive dependencies for selenium. Also Make sure you do a mvn install before running your lambda
    – Vishal
    Commented Oct 23, 2019 at 19:17

3 Answers 3

1
+100

The shade plugin combines all dependencies with the developed code and plops them in one Uber JAR. The downside is that it can overwrite resource files, and doesn't play well with signed jars (in my experience at least).

I would recommend moving away from the shade plugin if at all possible.

That said, if you have to use it - you're issue may be with the combining the jar resources. There are many transformers that you can use to resolve this, and you'll need to investigate which one is really needed. I would start with something like this

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>2.4.3</version>
    <configuration>
        <shadedArtifactAttached>true</shadedArtifactAttached>
        <shadedClassifierName>${executable.classifier}</shadedClassifierName>
        <filters>
            <filter>
                <artifact>*:*</artifact>
                <excludes>
                    <exclude>META-INF/*.SF</exclude>
                    <exclude>META-INF/*.DSA</exclude>
                    <exclude>META-INF/*.RSA</exclude>
                </excludes>
            </filter>
        </filters>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>shade</goal>
            </goals>
            <configuration>
                <transformers>
                    <transformer
                        implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                    <transformer
                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                        <mainClass>fully.qualified.ClassName</mainClass>
                    </transformer>
                </transformers>
            </configuration>
        </execution>
    </executions>
</plugin>

You can find more tranformers on the Apache plugin here

The alternative that I would suggest is Spring Boot, which uses the Jar-in-Jar structure with a custom ClassLoader to load classes from the internal jar(s).

This is the easier method due to not having to re-write files as the Shade plugin approach and it handles the dependencies a little better.

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <version>1.3.6.RELEASE</version>
    <configuration>
        <classifier>${executable.classifier}</classifier>
        <layout>ZIP</layout>
        <mainClass>fully.qualified.ClassName</mainClass>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>repackage</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Seriously, look at the simpler configuration!

NOTE: Most of this came from my own notes - version numbers may be a little old...

1
  • I'm not sure if this is the true answer as I have no need to go with this method (see my answer). I rather stick with the shaded jar since that seems to work. However your answer is detailed and you clearly know what your talking about so i'm going to mark it as correct Commented Nov 1, 2019 at 8:40
1

Try to tell your dependencies to output zip, maybe jar is messing up with things

Add this to maven-assembly-plugin configuration:

     <formats>
        <format>zip</format>
     </formats>

For example:

<plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-assembly-plugin</artifactId>
     ...
     <configuration>
         ...
         <formats>
            <format>zip</format>
         </formats>
     </configuration>
</plugin>

Same is suggested here

3
  • Could you give more detail using the pom provided in my question as i'm using the shade plugin not assembly Commented Oct 25, 2019 at 14:59
  • Is there any reason why you need to use shade? Shade is meant to create an uber-jar, so maybe it will hard to manage. I think it should be easy to use assembly instead and make it work. Im sorry I dont have any lambda ready with maven now so I cannot test myself :/ Commented Oct 25, 2019 at 15:16
  • When creating the jar I followed AWS official walkthrough on setting it up and its suppose to save space i think, however if it will work without it then I don't need it. Could you please update your answer with how I should format it so I can test your answer? Commented Oct 28, 2019 at 8:15
1

I figured it out. The java selenium seemed to cause the major issue. Downgrading to 3.10 fixed the issue although I have no idea why.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.