3

Has anyone out in the community successfully created a Selenium build in Jenkins using Browserstack as their cloud provider, while requiring a local testing connection behind a firewall?

I can say for sure Saucelabs is surprisingly easy to execute builds with the Sauce Jenkins plugin in a continuous deployment environment as I have done it. I cannot however, say the same for Browserstack. The organization I work with currently uses Browserstack, and although their service does support automated testing using a binary application I find it troublesome with Jenkins. I need to make absolutely sure Browserstack is not a viable solution, if so. I love Saucelabs and what their organization provides, but if Browserstack works I don't want to switch if I don't need to.

The Browserstack documentation instructs you to run a command, with some available options, in order to create a local connection before execution.

nohup ./[binary file] -localIdentifier [id] [auth key] localhost,3000,0 &

I have added the above statement as a pre-build step shell command. I have to also add 'nohup' as once the binary creates a successful connection, the build never actually starts since I have not exited as displayed in the output below.

BrowserStackLocal v3.5

You can now access your local server(s) in our remote browser.

Press Ctrl-C to exit

Normally I can successfully execute the first build without a problem. Subsequent build configurations using the same command never connect. The above message displays, but during test execution Browserstack reports no local testing connection was established. This confuses me.

To give you a better idea of what's being executed, I have 15 build configurations for various projects suites and browser combinations. Two Jenkins executors exist and I have more than 5 Browserstack VM's available at any given time. Five of the builds will automatically begin execution when the associated project code is pushed to the staging server, filling up both executors. One of them will begins and end fine. None of the others will as Browserstack reports local testing is not available.

Saucelabs obviously has this figured out with their plugin, which is great. If Browserstack requires shell commands to create local testing connections, I must be doing something wrong, out of order, etc.

Environment:

  • Java 7
  • Selenium 2.45
  • JUnit 4.11
  • Maven 3.1.1
  • Allure 1.4.10
  • Jenkins 1.5

Can someone post some information who use Browserstack in a continuous testing environment while utilizing multiple parallel test executions and tell me how each build is configured?

Thanks,

1
  • @ BrianSmith: you're using Allure plugin in Jenkins? How did you ever get it to work as advertised? I would love to know how you configured it as I spent 2 weeks trying (and failing) to get it to work at all. Mar 29, 2016 at 11:43

5 Answers 5

8

I've recently looked into BrowserStack with Selenium and the BrowserStack Plugin has made this task much easier.

Features

  • Manage your BrowserStack credentials globally or per build job.
  • Set up and tear down BrowserStack Local for testing internal, dev or staging environments.
  • Embed BrowserStack Automate reports in your Jenkins job results.

Much easier integration all round.

4

This is Umang replying on behalf of BrowserStack.

To start with, you are using the correct command for setting up the Local Testing connection. Although you do not need to specify the ‘localhost,3000,0’ details. We would also suggest you use the “-forcelocal” parameter while initiating the connection. The command should be as follows:

nohup ./[binary file] [auth key] -localIdentifier [id] -forcelocal &

The parameter “-forcelocal” will route all traffic via your IP address. Also, the process to initiate the connection before running your tests is correct.

However, here I’d like on confirm on the “id” you’ve specified while creating the connection. As you shared, there are 15 build configurations and I understand that each build has a different “id” specified. Please make sure that “id” specified while setting up the Local Testing connection and in the tests (“browserstack.localIdentifier” = “id”) is the same. Else, you will receive the error “[browserstack.local] is set to true but local testing through BrowserStack is not connected”

4
  • Using your recommended statement, I can successfully complete execution of each build one at time manually. The problem arises when multiple builds trigger and try to run two at a time, back-to-back. When that happens none of them run at all and the message about local testing not being connected displays in each log. Mar 11, 2015 at 13:13
  • 1
    I think I have a handle on this now. I have updated the pre-build step for each configuration with the statement in the answer above along with a few others to wait for a successful connection before execution proceeds. I will update this post once I test a bit more this week. Mar 11, 2015 at 21:29
  • It would be really useful if the local binary logged when connections were being made. I couldn't see this even when running with the -v flag. (p.s. BrowserStack is excellent!) Mar 8, 2016 at 11:44
  • @RalphCowling The -vv parameter should generate verbose logs. Could you give that a shot? Mar 8, 2016 at 12:30
1

Integrating BrowserStack with Jenkins is a little bit tricky, but don't worry, it's perfectly doable :-)

The BrowserStackLocal client needs to be started as a background process, as per Umang's suggestion, and that's pretty much how the SauceLabs plugin works as well.

The trouble is that when Jenkins sees that you start daemon processes all by yourself and not via a plugin, it kills them. That's why you need to convince it otherwise.

I've described how to go about it step by step in this article, but if you're using the Pipeline Plugin, you can use the below script as a starting point:

node {
    with_browser_stack 'linux-x64', {
        // Execute tests: here's where a step like
        //  sh 'mvn clean verify'
        // would go
    }
}

// ----------------------------------------------
def with_browser_stack(type, actions) {
    // Prepare the BrowserStackLocal client
    if (! fileExists("/var/tmp/BrowserStackLocal")) {
        sh "curl -sS https://www.browserstack.com/browserstack-local/BrowserStackLocal-${type}.zip > /var/tmp/BrowserStackLocal.zip"
        sh "unzip -o /var/tmp/BrowserStackLocal.zip -d /var/tmp"
        sh "chmod +x /var/tmp/BrowserStackLocal"
    }

    // Start the connection
    sh "BUILD_ID=dontKillMe nohup /var/tmp/BrowserStackLocal 42MyAcc3sK3yV4lu3 -onlyAutomate > /var/tmp/browserstack.log 2>&1 & echo \$! > /var/tmp/browserstack.pid"

    try {
        // Execute tests
        actions()
    }
    finally {
        // Stop the connection
        sh "kill `cat /var/tmp/browserstack.pid`"
    }
}

You'd of course need to replace the fake access key (42MyAcc3sK3yV4lu3) with yours, or provide it via an environmental variable.

The important part here is the BUILD_ID, because that's what the Jenkins ProcessTreeKiller looks for when it decides whether to kill your daemon process or not.

Hope this helps!

Jan

1

Paul Whelan's answer to use the Jenkin's BrowserStack Plugin is currently the simplest way to integrate Jenkins with BrowserStack. The plugin supports all Jenkins versions >1.580.1.

To ensure that you get BrowserStack test reports you will need to configure your project's pom.xml as documented on the plugin wiki.

-1

Just in case anyone else was having problems with this

For BrowserStackLocal v4.8 I found that -localidentifier has been removed from the binary options. (This is probably old news!)

When I removed the capabilities['browserstack.localIdentifier'] property from our automated tests the connection started working.

local binary

browserstack <key> -v -forcelocal

selenium setup

Capybara.register_driver :browserstack do |app|
    capabilities = Selenium::WebDriver::Remote::Capabilities.new

    # If we're running the BrowserStackLocal binary, we need to
    # tell the driver as well
    capabilities['browserstack.local'] = true

    # other useful options
    capabilities['browserstack.debug'] = true
    capabilities['browserstack.javascriptEnabled'] = true
    capabilities['javascriptEnabled'] = true

    # etc ...
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.