12

Say I submit a job using something like bsub pwd. Now I would like to get the job ID of that job in order to build a dependency for the next job. Is there some way I can get bsub to return the job ID?

6 Answers 6

9

Nils and Andrey have the answers to this specific question in shell and C/C++ environments respectively. For the purposes of building dependencies, you can also name your job with -J then build the dependency based on the job name:

bsub -J "job1" <cmd1>
bsub -J "job2" <cmd2>
bsub -w "done(job1) && done(job2)" <cmd>

There's a bit more info here.

This also works with job arrays:

bsub -J "ArrayA[1-10]" <cmd1>
bsub -J "ArrayB[1-10]" <cmd2>
bsub -w "done(ArrayA[3]) && done(ArrayB[5])" <cmd>

You can even do element-by-element dependency. The following job's i-th element will only run when the corresponding element in ArrayB reaches DONE status:

bsub -w "done(ArrayB[*])" -J "ArrayC[1-10]" <cmd3>

You can find more info on the various things you can specify in -w here.

2
  • (+1) Good thing you noticed the actual intent of the question. Sep 12, 2013 at 18:14
  • @jsmedmar Actually, it does. I'll add some detail to the answer.
    – Squirrel
    Jan 9, 2016 at 1:19
8

Just as a reference, this is the best solution I could come up with so far. It takes advantage of the fact that bsub write a line containing the ID to STDOUT.

function nk_jobid {
    output=$($*)
    echo $output | head -n1 | cut -d'<' -f2 | cut -d'>' -f1
}

Usage:

jobid=$(nk_jobid bsub pwd)
2
  • @AndreyRubshtein thats bash.
    – jsmedmar
    Jan 8, 2016 at 22:19
  • I'm wondering how to pass a variable inside the $(nk_jonid bsub $VARIABLE)
    – jsmedmar
    Jan 8, 2016 at 22:22
4

In case you are using C++, you can use the lsblib, LSF C API to submit jobs. The input and the output are structs. In particular, the output struct contains the job id.

#include <lsf/lsbatch.h>    
LS_LONG_INT lsb_submit (struct submit *jobSubReq, struct submitReply *jobSubReply)
1

If you just want to view the JOBID after submission, most of the time I will just use bhist or bhist -l to view the running jobs and details.

$ bhist
Summary of time in seconds spent in various states:
JOBID   USER    JOB_NAME  PEND    PSUSP   RUN     USUSP   SSUSP   UNKWN   TOTAL
8664    F14r3   sample       2       0    187954  0       0       0       187956 
0
$jobid = "0"
bsub pwd > $jobid
cat $jobid
0

you can put the job id into the name of the log file.

bsub -oo '%J' <rest of your command>

a log file will appear in the current directory that is the job id. You can then do what you want.

there are other values you can use besides %J. have a look at the bsub man page.

simple example:

> for job in {1..4} ; do bsub -oo '%J' sleep 1s ; done
> bjobs -o 'id cresreq' *
JOBID COMBINED_RESREQ
13217166 select[(type = any ) && (hi&&type == any )] order[r15s:pg]
13217167 select[(type = any ) && (hi&&type == any )] order[r15s:pg]
13217169 select[(type = any ) && (hi&&type == any )] order[r15s:pg]
13217170 select[(type = any ) && (hi&&type == any )] order[r15s:pg]

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.