40

According to this and this GitHub issues, currently there is no native way how to supply multiple tags for a service's image when using docker-compose to build one or multiple images.

My use case for this would be to build images defined in a docker-compose.yml file and tag them once with some customized tag (e.g. some build no. or date or similar) and once as latest.

While this can be easily achieved with plain docker using docker tag, docker-compose only allows to set one single tag in the image key. Using docker tag together with docker-compose is not an option for me since I want to keep all my docker-related definitions in the docker-compose.yml file and not copy them over into my build script.

What would be a decent work-around to achieve setting of multiple tags with docker-compose and without having to hardcode/copy the image names first?

6 Answers 6

29

I have some nice and clean solution using environment variables (bash syntax for default variable value, in my case it is latest but you can use anything ), this is my compose:

version: '3'
services:
  app:
    build: .
    image: myapp-name:${version:-latest}

build and push (if you need to push to the registry) with the default tag, change the version using environment variable and build and push again:

docker-compose build
docker-compose push
export version=0.0.1
docker-compose build
docker-compose push
5
26

You can also take the following approach:

# build is your actual build spec
build:
  image: myrepo/myimage
  build:
  ...
  ...
# these extend from build and just add new tags statically or from environment variables or 
version_tag:
  extends: build
  image: myrepo/myimage:v1.0
some_other_tag:
  extends: build
  image: myrepo/myimage:${SOME_OTHER_TAG}

You can then just run docker-compose build and docker-compose push and you will build and push the correct set of tagged imaged

5
  • 13
    It's a pity it only works in a compose file version < 3
    – whoan
    Aug 31, 2018 at 16:04
  • 1
    Agreed, this syntax is no longer supported by docker-compose.yml. @Maoz Zadok has the right approach below. If only for clarity, use your own environment variable that doesn't conflict with any Docker keywords (e.g. version). Jul 1, 2019 at 17:57
  • 3
    @whoan there is extension field support in the 3.4+ file formats. You can use it to perform the same steps albeit with more obscure YAML formatting. Ref: docs.docker.com/compose/compose-file/#extension-fields Jan 23, 2020 at 18:20
  • @JordanDeyton Good idea, I've posted an answer based on your suggestion. Thanks!
    – Romain
    Jan 25, 2020 at 17:11
  • For those people viewing this thread later - the extends and image syntax appears to still work with docker-compose file version 3.8, even though the official docker-compose documentation says that the syntax has been deprecated... (tested myself) ref: forums.docker.com/t/…
    – kachow6
    Aug 9, 2022 at 17:44
17

As suggested by @JordanDeyton extends cannot by used anymore in Compose file format > 3 and the Extension fields capability added in the version 3.4 can replace it to achieve the same goal. Here is an example.

version: "3.4"
# Define common behavior
x-ubi-httpd:
  &default-ubi-httpd
  build: ubi-httpd
  # Other settings can also be shared
  image: ubi-httpd:latest

# Define one service by wanted tag
services:
  # Use the extension as is
  ubi-httpd_latest:
    *default-ubi-httpd
  # Override the image tag
  ubi-httpd_major:
    << : *default-ubi-httpd
    image: ubi-httpd:1
  ubi-httpd_minor:
    << : *default-ubi-httpd
    image: ubi-httpd:1.0
  # Using an environment variable defined in a .env file for e.g.
  ubi-httpd_patch:
    << : *default-ubi-httpd
    image: "ubi-httpd:${UBI_HTTPD_PATCH}"

Images can be built now with all the defined tags

$ docker-compose build
# ...

$ docker images | grep ubi-httpd
# ubi-httpd  1       8cc412411805  3 minutes ago  268MB
# ubi-httpd  1.0     8cc412411805  3 minutes ago  268MB
# ubi-httpd  1.0.1   8cc412411805  3 minutes ago  268MB
# ubi-httpd  latest  8cc412411805  3 minutes ago  268MB
4
  • For those reading, support for file format version 3.4 was added to docker-compose 1.17.0 per github.com/docker/compose/releases/tag/1.17.0-rc1 The compose file reference page includes a matrix of formats to and Docker engine releases, but it was never clear to me how to map Docker engine releases to docker-compose versions. Jan 26, 2020 at 18:11
  • Well, this works for me in building two tags, but it has two flaws: it builds twice (not a big deal), but it also creates a second service that is started on "docker-compose up" and that of course is not the intended goal.
    – benelgiac
    Sep 30, 2021 at 14:04
  • Updated extension fields link: docs.docker.com/compose/compose-file/compose-file-v3/…
    – agconti
    Dec 14, 2021 at 17:29
  • With Docker version 24.0.4 and docker compose plugin v2.19.1 I got different image hashes using this workaround suggestion. Cause of it is that it adds default label com.docker.compose.service filling it with name of service. link. Should be no problem since images differs only by that label, all layers are identical.
    – hraphrap
    Jul 25 at 20:10
15

I came up with a couple of work-arounds of different complexity. They all rely on the assumption that ${IMAGE_TAG} stores the customized tag that represents e.g. a build no. and we want to tag all services' images with this tag as well as with latest.

grep the image names from the docker-compose.yml file

images=$(cat docker-compose.yml | grep 'image: ' | cut -d':' -f 2 | tr -d '"')
for image in $images
do
  docker tag "${image}":"${IMAGE_TAG}" "${image}":latest
done

However, this is error prone if somebody adds a comment in docker-compose.yml which would e.g. look like # Purpose of this image: do something useful....

Build twice

Use ${IMAGE_TAG} as an environment variable in your docker-compose.yml file as described here in the first example.

Then, simply run the build process twice, each time substituting ${IMAGE_TAG} with a different value:

IMAGE_TAG="${IMAGE_TAG}" docker-compose build
IMAGE_TAG=latest docker-compose build

The second build process should be much faster than the first one since all image layers should still be cached from the first run.

Drawback of this approach is that it will flood your log output with two subsequent build processes for each single service which might make harder to search through it for something useful.

Besides, if you have any command in your Dockerfile which always flushes the build cache (e.g. an ADD command fetching from a remote location with auto-updating last-modified headers, adding files which are constantly updated by an external process etc.) then the extra build might slow things down significantly.

Parse image names from the docker-compose.yml file with some inline Python code

Using a real yaml parser in Python (or any other language such as Ruby or perl or whatever is installed on your system) is more robust than the first mentioned grep approach since it will not get confused by comments or strange but valid ways of writing the yml file.

In Python, this could look like this:

images=$(python3 <<-EOF # make sure below to indent with tabs, not spaces; or omit the "-" before "EOF" and use no indention at all
    import yaml
    content = yaml.load(open("docker-compose.build.yml"))
    services = content["services"].values()
    image_names = (service["image"].split(":")[0] for service in services)
    print("\n".join(image_names))
EOF
)

for image in ${images}
do
docker tag ${image}:${IMAGE_TAG} ${image}:latest
done

Drawback of this approach is that the machine executing the build has to have Python3 installed, along with the PyYAML library. As already mentioned, this pattern could similarly be used with Python2 or any other programming language that is installed.

Get image names with combination of some docker commands

The following approach using some native docker and docker-compose commands (using go-templates) is a bit more complex to write but also works nicely.

# this should be set to something unique in order to avoid conflicts with other running docker-compose projects
compose_project_name=myproject.tagging

# create containers for all services without starting them
docker-compose --project-name "${compose_project_name}" up --no-start

# get image names without tags for all started containers
images=$(docker-compose --project-name "${compose_project_name}" images -q | xargs docker inspect --format='{{ index .RepoTags 0}}' | cut -d':' -f1)

# iterate over images and re-tag
for image in ${images}
do
    docker tag "${image}":"${IMAGE_TAG}" "${image}":latest
done

# clean-up created containers again
docker-compose --project-name "${compose_project_name}" down

While this approach does not have any external dependencies and is more safe than the grep method, it might take a few more seconds to execute on large setups for creating and removing the containers (typically not an issues though).

2
  • 7
    You can run docker-compose config in your first example which would strip all comments and leave only config statements. Ie. docker-compose config | grep 'image: ' | awk -F ':' '{ print "image "$2", tag "$3 }' does it for me. Nov 17, 2017 at 7:43
  • docker-compose images <service_name> outputs Container | Repository | Tag | Image Id | Size, with values like project_foo_1 | project_foo | latest | 590caffee098 | 1.1 GB. (The | was added to accommodate for this garbled line layout.) Maybe that can be used to improve the last example and make --project-name unneeded. May 4, 2019 at 11:43
8

There is now a built-in solution using buildx bake, released in v.0.7.0. This feature was implemented following to my suggestion in https://github.com/docker/buildx/issues/396.

Docker comes bundled with buildx installed, however, if you are on Mac and running Docker Desktop the bundled buildx version is older at the time of writing this and you will need to install the correct version of buildx in addition to Docker.

Add the x-bake extension field to your docker-compose.yaml:

version: '3.9'

services:
  my-app:
    image: my-repo/my-image:latest
    build:
      context: .
      dockerfile: Dockerfile
      x-bake:
        tags:
        - my-repo/my-image:${MY_TAG_1}
        - my-repo/my-image:${MY_TAG_2}
        - my-repo/my-image:${MY_TAG_3}

        - my-other-repo/my-image:${MY_TAG_1}
        - my-other-repo/my-image:${MY_TAG_2}
        - my-other-repo/my-image:${MY_TAG_3}

To build and tag the image run:

buildx bake --load

To build, tag and push to image to the repository or even to multiple repositories:

buildx bake --push
1
  • This should be the accepted answer and it seems to work under build without the extra x-bake part. Docs here
    – RiverHeart
    Jul 19 at 19:49
1

Using extends like many have suggested is sub optimal as it results in different checksums for each image produced. This is generally bad as you can be sure you are using consistent images in a production environment.

# cat docker-compose.yml 
version: "3.8"
services:

  container:
    build:
      context: .
    image: test/image:1

 # adds a copy of the image tagged with the build id
  extra-tags:
    extends: container
    image: test/image:2
  latest:
    extends: container
    image: test/image:3# 
# cat Dockerfile 
# syntax=docker/dockerfile:1
FROM alpine:3.15

# docker compose build
[+] Building 2.9s (13/13) FINISHED                                                                                                                                                                                                               
 => [latest internal] load build definition from Dockerfile                                                                                                                                                                                 0.0s
 => => transferring dockerfile: 84B                                                                                                                                                                                                         0.0s
 => [latest internal] load .dockerignore                                                                                                                                                                                                    0.0s
 => => transferring context: 2B                                                                                                                                                                                                             0.0s
 => [latest] resolve image config for docker.io/docker/dockerfile:1                                                                                                                                                                         1.2s
 => [container internal] load build definition from Dockerfile                                                                                                                                                                              0.0s
 => => transferring dockerfile: 84B                                                                                                                                                                                                         0.0s
 => [container internal] load .dockerignore                                                                                                                                                                                                 0.0s
 => => transferring context: 2B                                                                                                                                                                                                             0.0s
 => [extra-tags internal] load build definition from Dockerfile                                                                                                                                                                             0.0s
 => => transferring dockerfile: 84B                                                                                                                                                                                                         0.0s
 => [extra-tags internal] load .dockerignore                                                                                                                                                                                                0.0s
 => => transferring context: 2B                                                                                                                                                                                                             0.0s
 => CACHED [container] docker-image://docker.io/docker/dockerfile:1@sha256:39b85bbfa7536a5feceb7372a0817649ecb2724562a38360f4d6a7782a409b14                                                                                                 0.1s
 => => resolve docker.io/docker/dockerfile:1@sha256:39b85bbfa7536a5feceb7372a0817649ecb2724562a38360f4d6a7782a409b14                                                                                                                        0.0s
 => [extra-tags internal] load metadata for docker.io/library/alpine:3.15                                                                                                                                                                   1.0s
 => CACHED [container 1/1] FROM docker.io/library/alpine:3.15@sha256:3362f865019db5f14ac5154cb0db2c3741ad1cce0416045be422ad4de441b081                                                                                                       0.1s
 => => resolve docker.io/library/alpine:3.15@sha256:3362f865019db5f14ac5154cb0db2c3741ad1cce0416045be422ad4de441b081                                                                                                                        0.0s
 => [latest] exporting to image                                                                                                                                                                                                             0.2s
 => => exporting layers                                                                                                                                                                                                                     0.0s
 => => exporting manifest sha256:67e737bfe3d8955deeebd531025697ed4e8c4a4c263b9f1ea9bc7b4bc0330105                                                                                                                                           0.0s
 => => exporting config sha256:b2fe3df223ec66ebfa2c8e2dac5760b42a79eefb9abc83c2f0c5972e371c6e52                                                                                                                                             0.0s
 => => exporting attestation manifest sha256:1a18ee3574147462eea9d4b46ffaa30a69b5211b70238ba3f548e67b24a06aa8                                                                                                                               0.1s
 => => exporting manifest list sha256:19a2d2908947fa148c536b1d7b487e01d0aaa5ae4a8250fae41ac53ce2ae103d                                                                                                                                      0.0s
 => => naming to docker.io/test/image:3                                                                                                                                                                                                     0.0s
 => => unpacking to docker.io/test/image:3                                                                                                                                                                                                  0.0s
 => [container] exporting to image                                                                                                                                                                                                          0.2s
 => => exporting layers                                                                                                                                                                                                                     0.0s
 => => exporting manifest sha256:1f4ea8de74e009308a75606c3eb0adfad16bd5a0c495b1e5e790414adbb9e09e                                                                                                                                           0.0s
 => => exporting config sha256:021dc0c803e68082939d5a0ecd37d6f8e25be82eb9425b45296ad3cbd2373c29                                                                                                                                             0.0s
 => => exporting attestation manifest sha256:a297fc379c4b1a509de3b106f3f4e478d8f64b95abb1f516d584ce148f42cbec                                                                                                                               0.1s
 => => exporting manifest list sha256:73cb361e578dab7451a61a3381bb1d3fa228273de97dc7ed8ff5032498ac7049                                                                                                                                      0.0s
 => => naming to docker.io/test/image:1                                                                                                                                                                                                     0.0s
 => => unpacking to docker.io/test/image:1                                                                                                                                                                                                  0.0s
 => [extra-tags] exporting to image                                                                                                                                                                                                         0.2s
 => => exporting layers                                                                                                                                                                                                                     0.0s
 => => exporting manifest sha256:03f127f826024b1a0e194aa00bae4feba34af4471832d6d4b19d6b4cebd26d7f                                                                                                                                           0.0s
 => => exporting config sha256:3e45d21e99ff3ca466cf3197927e614d542da5d548f929422dfb7889fa55e546                                                                                                                                             0.0s
 => => exporting attestation manifest sha256:419d1dff28c183ab66a3c7724cf26248c58d4e452eae54e605cf3504ce15eefb                                                                                                                               0.1s
 => => exporting manifest list sha256:d671e4950218e44569fca1a74aebb100999deb24acaa2003149bc743cf1316d4                                                                                                                                      0.0s
 => => naming to docker.io/test/image:2                                                                                                                                                                                                     0.0s
 => => unpacking to docker.io/test/image:2                                                                                                                                                                                                  0.0s
# docker images --digests | grep ^test/
test/image                                        1                                                       sha256:73cb361e578dab7451a61a3381bb1d3fa228273de97dc7ed8ff5032498ac7049   73cb361e578d   2 minutes ago    9.07MB
test/image                                        2                                                       sha256:d671e4950218e44569fca1a74aebb100999deb24acaa2003149bc743cf1316d4   d671e4950218   2 minutes ago    9.07MB
test/image                                        3                                                       sha256:19a2d2908947fa148c536b1d7b487e01d0aaa5ae4a8250fae41ac53ce2ae103d   19a2d2908947   2 minutes ago    9.07MB

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.