58

I have an AWS CodePipeline that invokes CodeBuild in the Build Stage.

The question is how do I pass in an environment variable from CodePipeline that can be read in the CodeBuild's buildspec.yml?

I know I can set environment variables in CodeBuild, but I want to use the same CodeBuild project for dev, qa, and prod environments. I don't see how I can pass an environment variable from CodePipeline that makes it all the way to the buildspec.yml

Example buildspec.yml

version: 0.1

phases:   
  build:
    commands:
      - npm install
      - npm build -- --env ${CURRENT_ENVIRONMENT}

Where CURRENT_ENVIRONMENT would be the variable I set in the CodePipeline Stage action.

3
  • 2
    For anyone interested, the way I got around this is you can set environment variables in CodeBuild that get passed to buildspec.yml. So as long as you can get your parameters into the AWS::CodeBuild::Project CloudFormation declaration, they are picked up when the buildspec.yml gets run. Feb 7, 2017 at 20:20
  • 2
    This is a rather unfortunate limitation. I've added to an AWS forum thread about it here Consider chiming in to let AWS know if this is important to you.
    – jandersen
    Mar 3, 2017 at 20:26
  • 1
    how to use environment variables in buildspec.yml: npm build --env $THE_VARIABLE
    – ricka
    Mar 15, 2017 at 22:51

10 Answers 10

21

As of today, you can set environment variables for CodeBuild build jobs in your pipeline. This improvement makes it possible to reuse the same build project for multiple actions and simplify deployments to staging and production environments in your pipeline.

name: Build
actions:
  - name: MyBuildJob
    actionTypeId:
      category: Build
      owner: AWS
      provider: CodeBuild
      version: '1'
    runOrder: 1
    configuration:
      ProjectName: my-build-project
      PrimarySource: MyApplicationSource1
      EnvironmentVariables: '[{"name":"CURRENT_ENVIRONMENT","value":"Production","type":"PLAINTEXT"},
                              {"name":"UseParamStore","value":"CURRENT_ENVIRONMENT","type":"PARAMETER_STORE"}]'

https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodeBuild.html#action-reference-CodeBuild-config

3
  • This is the best answer in 2021 in the "official AWS way". Note the (rather odd) JSON-as-string value of EnvironmentVariables. Also, all Cloudfront YAML keys should be CamelCase (I'm surprised the example works as is).
    – kino1
    Jan 22, 2021 at 16:53
  • To "pass in an environment variable from CodePipeline" as asked, use {"name":"CURRENT_ENVIRONMENT","value":"#{StageNamespace.CurrentEnvironment}","type":"PLAINTEXT"}] and add Namespace: StageNamespace to the action providing the variable.
    – kino1
    Jan 22, 2021 at 16:59
  • Hi, which file does this go in please?
    – user13524876
    Mar 25, 2022 at 12:37
12

You can actually pass environment variables in CodeBuild cloudformation as below:

Build:
    Type: AWS::CodeBuild::Project
    Properties:
      Name:
        !Sub Build-${AWS::StackName}
      Description: Build your project
      Environment:
        Type: LINUX_CONTAINER
        ComputeType: BUILD_GENERAL1_SMALL
        Image: node8
        EnvironmentVariables:
          - Name: CURRENT_ENVIRONMENT
            Type: PLAINTEXT
            Value: staging

And in your buildspec.yml you can reference the environment like this,

version: 0.2

phases:
  install:
    commands:
      - npm install

  build:
    commands:
      - npm build -- --env ${CURRENT_ENVIRONMENT}
1
  • Not helpful, as one would typically want to define a single AWS::CodeBuild::Project to use across all environments. Your approach would mean creating one AWS::CodeBuild::Project for each environment, where the only difference between them is the environment variables.
    – Jim Tough
    Jan 12, 2021 at 16:19
9

This feature isn't available today.

A workaround would be to create a different CodeBuild project for each stage with different environment variables.

You can find details on using an environment variable in your builspec.yml commands here: http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

3
  • 15
    One whole year, this feature still unavailable? Jan 23, 2018 at 11:30
  • Wasn't able to figure out better idea than creating new build per environment...
    – BartusZak
    Oct 25, 2019 at 12:50
  • this answer is invalid. There are three ways to get Env Vars into a CodeBuild step via a CloudFormation template now: in the CodeBuild resource Environment.EnvironmentVariables section, in the Action step of a CodePipeline resource that uses CodeBuild, or in the CodeBuilds buildSpec file itself: docs.aws.amazon.com/codebuild/latest/userguide/…
    – airtonix
    May 22, 2020 at 3:43
7
  1. Create a Codebuild project.
  2. Create a pipeline project inside Codepipline.
  3. Add a CodeBuild action and define your variables in this action.
    For example:
    Variable name is: TAG
    Variable value is: staging

enter image description here

Using the variable you defined in the Codepipline Buildspec file in the Codebuild project: enter image description here

the logging results: enter image description here

1
  • Tried this and worked like a charm. This one should be the accepted answer as it explains what the question asks, in the best and simplest way with today's (August 2021) available options in AWS.
    – SebaGra
    Aug 12, 2021 at 16:09
5

You can use build environment variables to conditionally pass from buildspec to npm build by detecting which CodeBuild job or CodePipeline job is running. If you have one CodePipeline listening to /dev commits and another listening to /master commits, for example, this works perfectly.

Here's an example which runs a different PROD vs DEV build:

  build:
    commands:
   - |
     if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProjectDev-" >/dev/null; then
       yarn run build-dev;
     fi
   - |
     if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProject-" >/dev/null; then
       yarn run build-prod;
     fi
4

If you don't want to use CF, you can set ENV vars on your CodeBuild project in the AWS UI.

In AWS, go to your code builder project, in the top right corner click "edit" and select "environment." On the edit environment page, click the "Additional Configuration" drop down. In there you will see inputs for "name" and "value." "Name" is where you set your ENV and "value" is where you set your value for that variable.

Example: set API_ENV in "name" and development in "value." Then, in your buildspec.yml, you can use $API_ENV.

4

I am sure there are many answers posted above but putting some extra info which I found useful.

CodePipeline started supporting the environment variable at the Pipeline level where some of have native support via Pipeline (CodePipeline-generated pipeline variables)

So for example if we need to get the CommitId in any stage then we can declare the environment variable as per below.

  1. Give any name to variable namespace on source stage for example "SourceVariables" enter image description here
  2. Use that namespace in any stage as "#{SourceVariables.CommitId}" in this case its getting use in build stage. enter image description here

Also we can generate our own environment variable from any stage which will be available to use in next stages. More info is available here.

Going back to now original question, we can set the environment variable on build action which will help us to use it along with build spec file. But still it will not allow us to dynamically pass it during every push of code from source. And due to that it require two steps like update pipeline and then trigger the Pipeline so that it can go to respective stage.

1
  • This is indeed correct, but you didn't emphasize enough that the Environment Variables must be set from CodePipeline -> Edit: Build -> Environment variables - optional. If you set these variables in CodeBuild -> Edit -> Environment -> Additional configuration -> Environment variables it WON'T WORK! This is so dumb, but I spent too much time on this myself. Sep 14, 2022 at 19:38
1

I created a small python script to parse the $CODEBUILD_INITIATOR variable that is passed. Below is the buildspec.yml and below that is the python script that I include with the build and call.

      build:
        commands:
          - |
            PIPELINE_ENV=$(python3 codebuild_env_parser.py $CODEBUILD_INITIATOR)
            OUTPUT_STATUS=$?
            if [ "$OUTPUT_STATUS" = "0" ]; then
              echo "Success finding a valid environment from codebuild_env_parser.py."
            else
              echo "Failure finding a valid environment from codebuild_env_parser.py. Check the script to see if the codepipeline env was passed correctly."
            fi

Python script (codebuild_env_parser.py):

    import sys
    
    
    def main():
        args = sys.argv
        if len(args) == 2:
            env_list = ["dev", "prod"]
            pipeline_invoker = args[1].lower()
            code_pipeline_name = pipeline_invoker.split("codepipeline/")[1]
            env_name = code_pipeline_name.split("-project-name")[0]
            if env_name in env_list:
                print("{}".format(env_name))
                sys.exit(0)
            else:
                sys.exit(1)
        else:
            sys.exit(1)
    
    
    main()

You'll have to tweak some variable values here if you want this to work. Namely, "-project-name".

0

I've created a lambda function that updates an existing codebuild project's environment variables. You can the start the build (codebuild.start) after updating the variables. Looks something like this (nodejs):

var params = {
    "name": "Build-project-name",
    "description": "Build this project.",

    "environment": {
        "type": "LINUX_CONTAINER",
        "image": "aws/codebuild/standard:1.0",
        "computeType": "BUILD_GENERAL1_LARGE",
        "environmentVariables": [
        {
            "name": "MY_ENVIRONMENT_VARIABLE",
            "value": "VALUE_OF_ENVIRONMENT_VARIABLE",
            "type": "PLAINTEXT"
        }]
    }
}

codebuild.updateProject(params, function(err, data) {
    if (err) {
        console.log(err, err.stack); // an error occurred
    }
    else     {
        console.log(data);           // successful response
    }
});
0

The CodeBuild initiator is an environment variable in CodeBuild and this can be used to read in the CodePipeline name.

So if you include your environment in your CodePipeline name as a suffix (-dev or -prod, for instance), then you can parse it out like so.

version: 0.2
phases:
  build:
    commands:
      - CURRENT_ENVIRONMENT=`echo $CODEBUILD_INITIATOR | cut -d '-' -f2 | tr '[:upper:]' '[:lower:]'`
      - echo "My env is $CURRENT_ENVIRONMENT"


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.