21

I'm trying to run a shell script from my template file located in another project via my include.

How should this be configured to work? Below scripts are simplified versions of my code.

Project A

template.yml

deploy:
  before_script:
    - chmod +x ./.run.sh
    - source ./.run.sh

Project B

gitlab-ci.yml

include:
 - project: 'project-a'
    ref: master
    file: '/template.yml'

stages:
  - deploy

Clearly, the commands are actually being run from ProjectB and not ProjectA where the template resides. This can further be confirmed by adding ls -a in the template file.

So how should we be calling run.sh? Both projects are on the same GitLab instance under different groups.

3
  • So you want to execute shell script by Project B pipeline, but in scope of Project A? For example ls -a should print files from Project A in the pipeline that runs in Project B? Is my understanding correct?
    – rkm
    Sep 2, 2020 at 15:08
  • @rkm that is correct. In my example, Project A contains run.sh. But since I'm including template.yml in Project B, it can't see run.sh. I'm trying to include Project A and its dependencies.
    – ctwheels
    Sep 2, 2020 at 15:58
  • the question is duplicated here: gitlab.com/gitlab-org/gitlab/-/issues/241808
    – john k
    Jan 18, 2023 at 16:42

5 Answers 5

16

If you have access project A and B, you can use multi-project pipelines. You trigger a pipeline in project A from project B.

In project A, you clone project B and run your script.

Project B

job 1:
  variables:
    PROJECT_PATH: "$CI_PROJECT_PATH"
    RELEASE_BRANCH: "$CI_COMMIT_BRANCH"
  trigger:
    project: project-a
    strategy: depend

Project A

job 2:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "pipeline" && $PROJECT_PATH && $RELEASE_BRANCH'
  script:
    - git clone -b "${RELEASE_BRANCH}" --depth 50 https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}/${PROJECT_PATH}.git $(basename ${PROJECT_PATH})
    - cd $(basename ${PROJECT_PATH})
    - chmod +x ../.run.sh
    - source ../.run.sh
5
  • Thank you! This is exactly how I’ve accomplished this. It works really well and I wasn’t aware of this feature. Now I have a CI repository doing all my fancy stuff for a bunch of other projects (builds, deploys, etc.) this is fantastic. If anyone is wondering, you can also set a trigger branch. I found this to be extremely useful for different versions or languages (e.g. I have a build project that works for different languages, a python branch for building python and a nodejs branch to build nodejs).
    – ctwheels
    Jan 12, 2021 at 15:25
  • How do the variables from Project B become accessible in Project A? I do not understand how that works in this answer.
    – Jake Stout
    Jul 28, 2021 at 13:37
  • I actually want the stuff to run from project B to show up on project A's CI logs. If not the developer of project A can ignore the templates that were setup on project B. Nov 16, 2021 at 7:38
  • 2
    Doesn't multi-project pipelines mean running external scripts on the external pipeline? I think the user is asking how to run external scripts on the current pipeline.
    – john k
    Mar 25, 2022 at 15:26
  • 1
    @john-ktejik, When you use the CI_JOB_TOKEN to trigger pipelines (or use trigger key), GitLab recognizes the source of the job token. The pipelines become related, so you can visualize their relationships on pipeline graphs. Mar 27, 2022 at 13:02
13

We've also run into this problem, and kinda wish Gitlab allowed includes to "import" non-yaml files. Nevertheless the simplest workaround we've found is to build a small docker image in repo A, which contains the script you want to run, and then repo B's job uses that docker image as the image, so the file run.sh is available :)

Minimal Dockerfile:

FROM bash:latest
COPY run.sh /usr/local/bin/
CMD run.sh

(Note: make sure you chmod +x run.sh before building your image, or add a RUN chmod +x /usr/local/bin/run.sh step)

Then, you'd just add this to your Project B's .gitlab-ci.yml:

stages:
  - deploy

deploy:
  image: registry.gitlab.com/...  # Wherever you pushed your docker image to
  script: run.sh
3

it's also possible to request a script by curl instead of copying a whole repository:

- curl -H "PRIVATE-TOKEN:$PRIVATE_TOKEN" --create-dirs "$CI_API_V4_URL/projects/$CI_DEPLOY_PROJECT_ID/repository/archive?path=pathToFolderWithScripts" -o $TEMP_DIR/archive.tar.gz
- tar zxvf $TEMP_DIR/archive.tar.gz -C $TEMP_DIR --strip-components 3
- bash $TEMP_DIR/run.sh
  • to make a curl request
  • to archive a folder with scripts
  • to unzip scripts in a temporary folder
  • to execute sh
-1

As hinted by the answer above, multi project pipelines is the right approach for it.

Here's how it worked for me:

GroupX/ProjectA - contains reusable code

# .gitlab-ci.yml

stages:
  - deploy

reusable_deploy_job:
  stage: deploy
  rules:
    - if: '$CI_PIPELINE_SOURCE == "pipeline"'   # run only if triggered by a pipeline
  script:
    - bash ./src/run.sh $UPSTREAM_CUSTOM_VARIABLE

GroupY/ProjectB - job that will reuse a code

# .gitlab-ci.yml

stages:
  - deploy

deploy_job:
  stage: deploy
  variables:
    UPSTREAM_CUSTOM_VARIABLE: CUSTOM_VARIABLE   # pass this variable to downstream job
  trigger: groupx/projecta
2
  • This does not provide an answer to the question. You can search for similar questions, or refer to the related and linked questions on the right-hand side of the page to find an answer. If you have a related but different question, ask a new question, and include a link to this one to help provide context. See: Ask questions, get answers, no distractions Sep 8, 2021 at 4:44
  • Doesn't this trigger scripts in the other pipeline? This does technically answer the question as asked, but I think the user is asking how to run external scripts on the current pipeline.
    – john k
    Mar 25, 2022 at 15:24
-1

ref This :: https://docs.gitlab.com/ee/api/repository_files.html#get-file-from-repository

GET /projects/:id/repository/files/:file_path/raw

curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fmodels%2Fkey%2Erb?ref=master" 

It will display the file.

To download this file just add >> <filename> as below

curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fmodels%2Fkey%2Erb?ref=master" >> file.extension

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.