18

As both node step and stage step provide scoped {} syntax, what is the best practice for defining their topology inside groovy code?

Exhibit A

node ("NodeName") {
    stage ("a stage inside node"){
        // do stuff here
    }
}

Exhibit B

stage ("a stage holding a node") {
    node ("NodeName"){
        // do stuff here
    }
}

2 Answers 2

18

This depends on your actual needs.

As long as you can run your complete pipeline on a single node, I would wrap the stages in a node so that the pipeline is not blocked by busy executors.

As soon as you use the parallel step, then you don't really have a choice besides having stage around node allocations.

There are (at least for me) no issues around mixing that, i.e., have the first 2-3 stages executed on the same node and then one stage that executes on multiple nodes within parallel.

4
  • That's interesting. I am having 5 stages inside a single node. I need to have the final stage on a different node (but not as parallel).
    – Rakib
    Jan 15, 2017 at 11:34
  • So where do you see the problem? Jan 15, 2017 at 22:21
  • No problem... Just a remark.... That i might not really need the "parallel" step at all.
    – Rakib
    Jan 15, 2017 at 22:37
  • Yes, since 1.2 (Sep 2017), it's possible to have parallel stages: jenkins.io/blog/2017/09/25/declarative-1 Dec 7, 2017 at 13:22
13

With node { stage { ... } } each stage will share the same working folder and all the files from the previous stage will be there for the next stage.

With stage { node { ... } } you need to stash/unstash files between each stage. If you have a large repository, and especially if you have a large folder of dependencies like node_modules, this repeated stash/unstash could end up being a significant, or even majority, or your build time.

IMO I would generally start with the first syntax, node { stage { ... } } as preferred. If you have individual build stages that take time and can benefit from parallelism, then switching to stage { node { ... } } might be better, as long as the time gained in parallelization is not lost in stashing.

Update:

I tested the exact effect of swapping nesting on one of our builds. with a bunch of stages inside a node, the total build time is just over one minute. With a node inside each stage, total build time is almost five minutes. Big difference.

Build stage timing example

5
  • 1
    great stuff... thanks for sharing your findings. And the matter of stash / unstash is very helpful here :)
    – Rakib
    Oct 15, 2019 at 9:47
  • This is great example/explanation. Thanks for sharing. Nov 5, 2019 at 21:16
  • Thanks for the great explanation. In case of node { stage { ... } }, we are sure that stages will be executed by the same executor as well on the same node? If we have a node with 10 executors and sequential stages inside a node.
    – Minee
    Nov 22, 2023 at 10:54
  • @Minee Sorry, I can't say. I haven't used Jenkins in a few years now. I'd suggest posting a new and you can reference this one for background. Good luck. 🤞 Nov 28, 2023 at 20:42
  • @SamuelNeff According to Jenkins webpage the node keyword reserves an executor: "For more advanced usage with Scripted Pipeline, the example above node is a crucial first step as it allocates an executor and workspace for the Pipeline"
    – Minee
    Dec 1, 2023 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.