29

What's the proper way to launch a script from jenkins, don't get the build hanging, and leave the process running? I can't seem to get it to work. Either the script doesn't run or the build hangs.

If I put in the build's "Execute shell" step bash relaunch.sh & or relaunch.sh > output.log & or nohup bash relaunch.sh &, nothing happens; build finishes, but the process doesn't run. I guess it can be related to Jenkins waiting for the error pipe to close.

If I do nohup bash relaunch.sh 2>&1 > output.log as suggested here, the output is properly redirected, but the build hangs (doesn't finish), and the process dies when I kill the build.

Adding export BUILD_ID=dontKillMe, as suggested here, here, and here, either to the "Execute shell" step or the script itself doesn't help either. The build hangs and the process dies when I kill the build. Needless to say, my knowledge of linux is very limited.

How do people do this in a clean way?

1
  • I'm nominating for reopen, the answers to this question are correct and the duplicate has none of the correct information Oct 29, 2018 at 19:37

6 Answers 6

35

A convenient way to achieve that is to change the environment variable BUILD_ID under Execute shell which Jenkins's ProcessTreeKiller is looking for.

By doing,

BUILD_ID=dontKillMe nohup bash relaunch.sh &

Jenkins will assume that the background job is not spawned by the build and will not kill them after finishing the job.

Thanks to Joshua for his observation that you could also use JENKINS_NODE_COOKIE as

JENKINS_NODE_COOKIE=dontKillMe
4
  • 2
    Thanks, this helped us tremendously today! Mar 13, 2017 at 18:01
  • 3
    If this doesn't work, try setting JENKINS_NODE_COOKIE=dontKillMe instead - devops.stackexchange.com/questions/1473/…
    – Joshua
    Feb 13, 2018 at 17:56
  • 2
    FYI, you want to use JENKINS_NODE_COOKIE=dontKillMe for Pipeline projects, as BUILD_ID=dontKillMe does not work there. Source: wiki.jenkins.io/display/JENKINS/ProcessTreeKiller
    – Serge
    Jul 1, 2019 at 14:38
  • nohup didn't seem to output a file named nohup.out like I expected it to. I needed to include >scan.log 2>&1 just before the & so that I could actually check the output from this script later in the pipeline (or manually by browsing the file in the workspace.) May 12, 2021 at 18:46
9

I was having the exact same problem. I ended up fixing this by placing the following in the Jenkins execute shell box:

BUILD_ID=dontKillMe ./grid.sh

I moved the & inside the script file. Here's what the script looks like:

#!/bin/bash
java -jar selenium-server-standalone-3.0.1.jar -role hub &

Hopefully this helps someone!

2

Interesting solutions. Has anyone tried screen?

Here's how I run it (for RTL simulation using ModelSim-ASE):

screen -S $testname -d -m -L bash -c 'cd build_sim ; make; make sim'
echo Waiting for simulator ...

I then wait for our simulator to get to a waiting state by watching the screenlog.0

until ((`fgrep -c 'Ready for simulation' screenlog.0`)) ;do
    sleep 1
done

Then, when the other jobs that need the simulation are done running (or Jenkins detects a failure), in the post, run:

screen -S $testname -X kill

The only caveat is that if Jenkins somehow dies before the job cleans up, in theory you could have a simulator chewing up resources while waiting for something to happen.

To break it down a little: With screen, '-S' assigns a title to the session (or addresses a running session with that title), '-d' runs detached, '-m' spawn a fork first (which hides from Jenkins watchful eyes), and -L turns on logging (which is also handy in case something in the simulator breaks - you now have a log file artifact with the output). The rest is simply bash inside of the screen session.

0

I've solved a similar issue by executing:

mycommand -parameters &> /dev/null &

Somehow Jenkins was attached to the stdout of my script.

So I tried redirecting the standard output/error to /dev/null using &> /dev/null then adding the process to background using & and solved the problem, Jenkins ended the build without killing the process spawned.

PD: I tried adding an exit 0 at the end of the script, and it was executed, but that didn't free the Jenkins build, it kept attached to the subprocess output.

0

your Jenkins command box should only contain ksh /path/to/script/script.sh

your script file should look like this:

#!/bin/bash
command > output.log &
0

I have implemented the solution as follows:

stage('Start AdminServer') {
  sh 'export JENKINS_NODE_COOKIE=dontKillMe;export BUILD_ID=dontKillMe;/app/jenkins/scripts/applyPatch/applyPatch.sh startAdmin' 
}
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.