48

We have a project in a Github repository with multiple Jenkinsfiles:

my-project
  app
    Jenkinsfile
  lib1
    Jenkinsfile
  lib2
    Jenkinsfile

We have created 3 Jenkins pipelines each referring to a Jenkinsfile. enter image description here

Question: How to avoid triggering "app" and "lib1" pipelines when there is a new commit in "lib2"? We don't want to run N jobs every time a commit happens.

I've seen that the issue is addressed in https://issues.jenkins-ci.org/browse/JENKINS-43749, but I haven't found a solution there.

1
  • is it really advisable to do that? what if you only have 1 file change in lib1 folder, will it also rebuild the other apps?
    – KD.S.T.
    Oct 4, 2021 at 7:04

4 Answers 4

19

RECENT UPDATE:

I later fixed this issue using following code snippet: If you see the command dir('servicelayer'), using this to move into the directory, executing git command to find the difference between the commits and raising a flag. This way i have managed 3 Jenkins files in a single repository.

stage('Validation') {
steps {
        //Moving in to the directory to execute the commands
        dir('servicelayer') {
            script {
                //Using the git command to check the difference between previous successful commit. ${GIT_PREVIOUS_SUCCESSFUL_COMMIT} is an environment variable comes with GIT Jenkins plugin
                //There is a drawback though, if it is the first time you are running this job, this variable is not available and fails the build
                //For the first time i had to use ${env.GIT_COMMIT} itself at both places to pass the build. A hack but worth it for future builds.

                def strCount = sh(returnStdout: true, script: "git diff --name-only ${env.GIT_COMMIT} ${GIT_PREVIOUS_SUCCESSFUL_COMMIT} | grep servicelayer | wc -l").trim()
                if(strCount=="0") {
                    echo "Skipping build no files updated"
                    CONTINUE_BUILD = false
                } else {
                    echo "Changes found in the servicelayer module"
                }
            }
        }
    }
}

OLD ANSWER:

You can do it in 2 ways:

a) Configure your build jobs by adding "Additional Behaviour" -> "Polling ignore commits in certain region" This behaviour will let you add "Whitelisted regions" or "Blacklist the regions" which you do not want to poll for triggering the build job.

b) Write a custom shell script to verify the files changed per commit and verify the location. This shell script can then be added to your Jenkinsfile and be it Declarative or Scripted you can tweak the behaviour accordingly.

I will recommend option a) as it is simpler to configure and maintain as well. Hope this helps.

Additional Behaviour

4
  • I assume you might have already figured it out, but one more change you will have to do here is to change the directory context of the Jenkinsfile where it will be executed. You must use dir('folder_name') after "steps" tag to set the directory where Jenkinsfile need to be executed. Jul 9, 2018 at 2:24
  • Surrounding the block inside script {} with try {} catch (err) {} and continuing the build on error (depending on your use case of course) may help with the non-availability of GIT_PREVIOUS_SUCCESSFUL_COMMIT on first execution of the job.
    – maria vill
    Nov 27, 2019 at 15:34
  • As an aside, grep | wc -l is usually better written grep -c, though stackoverflow.com/questions/46978924/… shows how to do it in a more idiomatic way as far as the shell scripting is concerned, i.e. use grep -q and simply examine its return status.
    – tripleee
    Aug 3, 2021 at 12:00
  • Can you check if GIT_PREVIOUS_SUCCESSFUL_COMMIT is empty, and then continue if it is?
    – AnnanFay
    Nov 3 at 9:55
11

I'd suggest using the multi-branch pipeline plugin, then specifying the path to the appropriate jenkinsfile. So you'd have three pipeline jobs, one for app, lib1, and lib2.

enter image description here

10
  • 2
    You are NOT tackling the problem described in the question, of NOT starting all jobs for changes in the same repository where other Jenkinsfiles are contained. Jun 27, 2019 at 6:30
  • 3
    @RaúlSalinas-Monteagudo you are incorrect. using a multi-branch pipeline lets you only trigger a single job despite a repository commit. That's because in the multi-branch pipeline, you can specify the path to which the Jenkinsfile applies. Did you read my answer or try this out? This is a workflow I'm using right now (had started using around the time I wrote the answer) so you'll need to explain why it works for me and nobody else.
    – iheanyi
    Jun 27, 2019 at 14:53
  • Dear Mr iheanyi: this is the original question: "Question: How to avoid triggering "app" and "lib1" pipelines when there is a new commit in "lib2"? We don't want to run N jobs every time a commit happens." Jun 28, 2019 at 5:56
  • 9
    @RaúlSalinas-Monteagudo my solution only triggers pipelines for which there are commits that affect the referenced path. I'm not sure what you're not understanding. You set up three multi-branch pipeline jobs, one for app, one for lib1, and one for lib2. The app pipeline will only run when there are commits to app. The lib1 pipeline will only run when there are commits to lib1. Same behavior for lib2. This is exactly the workflow the question is asking for advice on achieving.
    – iheanyi
    Jun 28, 2019 at 14:00
  • 1
    @Normal yes, that's the referenced plugin. I'm not sure about dependencies, you'd need to experiment. Even the approach in this answer wasn't strictly the problem the plugin was designed to solve - I just found a way to make it work for my situation.
    – iheanyi
    Sep 28, 2020 at 18:36
5

I achieved to have multiple pipelines (Jenkinsfile) in a single repository. It's not hacky but it was not exactly trivial, requires recent versions of Jenkins with updated Lockable Resources plugin and may require additional steps in the build procedure.

  • Create a "Source Code Management" only project. In Jenkins terminology it can be a Freestyle project that will only care about keeping the source code updated, with a SCM section and a triggered configured (example name "MyProjectSCM"). In my case I have a git repository with a polling schedule. Ensure the "Execute concurrent builds if necessary" flag is not checked;

  • In the project repository commit one or more Jenkins pipeline, for example "JenkinsfileBuild", "JenkinsfileRelease". Ensure the pipelines:

    1. has the the skipDefaultCheckout in the options section;

    2. has a trigger on the SCM project configured ("MyProjectSCM");

    3. locks the SCM project during build.

       pipeline {
           agent any
      
           options {
               disableConcurrentBuilds()   
               skipDefaultCheckout()
           }
      
           triggers {
               upstream(upstreamProjects: 'MyProjectSCM')
           }
      
           stages {
               stage('Build') {
                   lock('MyProjectSCM') {
                       steps {
                           echo 'Build within MyProjectSCM lock...'
                       }
                   }
               }
           }
       }
      
  • Create Pipeline projects, one per each Jenkinsfile you committed in the repository. The pipeline should have a configured "Pipeline script from SCM" (in this example the pipeline can be named "MyProject_Build", with a configured "JenkinsfileBuild"), with the "Lightweight checkout" flag checked (it's actually the default):

enter image description here

The skipDefaultCheckout directive in "MyProject_Build" pipeline will prevent the checkout of the repository in that workspace, "Lightweight checkout" flag itself strangely it's not enough to prevent the checkout. Because the "MyProject_Build" workspace it's actually empty you have to refer to the SCM only project directory in your build scripts, by means of environment variables or relative paths. The lock directive is useful so a new commit can't happen while actually building the project.

3

I think easiest way to use multibranch pipeline with Multibranch build strategy extension plugin https://plugins.jenkins.io/multibranch-build-strategy-extension/

It gives a way to trigger build only if change occurred in specific folder/region

1
  • This answer is much better than all the others. If the build doesn't even trigger, you don't have to stop or even delete it afterwards.
    – rü-
    Nov 23 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.