12

We have a CodePipeline set up to do a build, deploy to a QA ECS environment, then a manual approval step to deploy to Prod.

What gets confusing though, is when there are several builds running one after another. Several builds get deployed to QA in sequence, but then the Approval button seems to approve them one at a time, and it's not clear which build you're approving when you click on it.

What I would like to be able to do is to approve the latest build, in case the earlier builds had issues that were fixed by the later builds. What would be the best way to accomplish that?

5
  • Where can we find this Approval button you are talking about?
    – JMA
    Commented Dec 28, 2017 at 1:29
  • The button that appears on a manual approval step.
    – kos
    Commented Dec 28, 2017 at 1:41
  • But this is a button in a platform of yours right? I cannot find a manual approval step on the AWS console.
    – JMA
    Commented Dec 28, 2017 at 2:06
  • 1
    This is a standard step. New Step -> Action Category -> Approval
    – kos
    Commented Dec 28, 2017 at 2:21
  • I have exactly the same requirement. Thanks for the question!
    – user389955
    Commented Jun 26, 2018 at 1:04

5 Answers 5

8
+50

I had the same problem. Manual approvals are confusing since several pipeline executions can get queued and it's easy to lose track of things. I think we can blame this on CodePipeline's bad UX.

The workaround I settled with is to have two identical pipelines for the same project. They have the same source stage (same repo/branch) but different deploy stages (one deploys to QA, one deploys to prod). No more manual approval stages. The QA pipeline is set to auto-execute when changes in the source (repo/branch) are detected while the Prod pipeline needs to be manually released.

Basically, we replaced the Manual Approval with Manual Release. Manual release always releases the latest from source unlike manual approvals.

1
  • 5
    The problem with this approach is that when you do the manual deploy to prod it might pull new changes from source control that have not yet been deployed to test and have therefore not been verified. It would be better if you referenced the artifacts built from the QA pipeline so that you are not at risk of accidentally releasing unverified changes
    – Sam Shiles
    Commented Feb 12, 2019 at 17:07
5

You should place the deploy and approval action in the same stage. This lets you approve exactly what you tested. Why? Because exactly one pipeline execution can be in a pipeline stage at any given time.

...approve the latest build, in case the earlier builds had issues that were fixed by the later builds.

If you want to let later builds catch up, reject the earlier build that is waiting for approval.

3
  • Aaron: but it is even hard to tell which stage I am reject and approve. Besides, I have to reject several times.
    – user389955
    Commented Jun 26, 2018 at 1:05
  • Which part is hard? Perhaps aws codepipeline get-pipeline-state (docs.aws.amazon.com/cli/latest/reference/codepipeline/…) would help?
    – Aaron
    Commented Jun 28, 2018 at 18:30
  • Presumably you're rejecting several times because you want to approve the latest pipeline execution. One way to avoid this is to automate the approval step (i.e. run some automated test instead of asking a human to manually approve).
    – Aaron
    Commented Jun 28, 2018 at 18:33
2

One option if you don't want to have multiple pipelines is to by default disable stage transitions into your environments that required controlled releases.

When you are ready to deploy into an environment, you enable the stage transition to allow the most recent release from the previous stage to be processed and then disable the transitions again.

It's still a bit clunky, but reasonably effective once you get used to it. Having to reject each change that comes through becomes very slow and cumbersome to manage, so by disabling transitions you choose when to promote a release.

IMO, CodePipeline should have an option to automatically supersede executions if they are paused at the manual approval stage.

2
  • mixja: how to automate this two steps: manual approval and disable/enable the stage after the approval stage? and will you put disable/enable transition before the approval stage of after the approval? if put it before approval, manager will not get email, if put it after approval, then are you sure manager will only need to approve once?
    – user389955
    Commented Jun 26, 2018 at 1:15
  • 1
    We've just put codepipeline to use at my work, and yes, the manual approval is quickly showing its limitations. For now I'd be happy to just find a way to see how many actions are queued on a specific manual-approval step; if it's N I can reject N-1 and then approve the last one. However, I can't find that info neither in the Console nor in the CLI. Is there a way at all?
    – Magnus
    Commented Dec 26, 2018 at 13:45
2

In the CodePipeline UI, you can see the history of Manual approvals in your pipelines' History. Click on History to see what's in progress (Manual Approvals that haven't timed out will always be in progress) and the source (git) short-sha that triggered it (if you need to narrow down to the relevant commit).

To know which Manual approval you're approving, in Pipeline view, click on View current revisions next to the Manual step (to get the Execution ID), then find the matching Execution ID in History (should be the oldest one).

Only way I found to get to the latest Approval is to hit reject n-1 times in the pipeline (where n is how many manual approvals are still in progress) until I only have 1 approval left (or until I find matching Execution ID).

0

Well, we can solve this problem as how you describe it with development, but it might also be a process glitch.

For example: If we have a development branch, a release branch (staging) and a master branch ( production ) we could easily solve this issue.

Development branch Things we develop will be going through the development branch stage where we don't need the manual approval, as we don't want to check every changes. We have setup automated unit tests for that.

Release branch This will deploy to the staging environment where we extensively test the software quality, also based on the regression tests on an acceptance chain with acceptance systems. This should prevent all the big issues before merging towards master branch. Next to that, we could also manually test the release branch on the staging environment. If this works, be happy and easily migrate towards master

Master branch This will deploy to the production environment with a manual approval before the actual deployment is taking place, knowing for sure you will only push 1 change, being the merge from release to master, preventing the issues you've summarized in the ticket.

Another way is to develop a new AWS feature where you can uncheck or check a checkbox saying: always take the latest release, but that will not help adding value to the pipeline integration as things will be pushed without testing well enough.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.