6

since 2 weeks or so I have an issue with docker compose while changing anything on my compose files (or at least i do not remember).

When docker compose up is run, sometimes (rarely) a container does not get added to one or multiple of the networks defined in the yml file. For example, I have some backend containers, that are all connected to a database container using a network. However, sometimes one of the backend containers ends up without a connection to the DB. When analyzing with docker network inspect I found out that the relevant backend container did not "join" the network.

I made sure that the db container is up before the backend containers are started, however if this was the problem, i think there should have been an error message anyways, since the db compose file defines the network, which is external in the backend compose file.

DB compose file:

version: "3.5"

services:
  db:
    restart: always
    image: postgres:15.2-alpine
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./postgres.conf:/etc/postgresql/postgresql.conf:ro
    environment:
      - POSTGRES_PASSWORD=${DB_MASTER_PWD}
    command: postgres -c config_file=/etc/postgresql/postgresql.conf -c logging_collector=on -c log_directory=/logs/postgresql -c log_filename=postgresql.log -c log_statement=ddl
    ports:
      - 127.0.0.1:5432:5432
    networks:
      - db
    container_name: postgres

volumes:
  postgres_data:

networks:
  db:

Backend(1) compose file:

version: "3.5"

services:
  backend_1:
    restart: always
    build:
      context: ${DOCKER_FILES_PATH}/customer-compose/backend/
      dockerfile: Dockerfile
      args:
        ...
    image: ${BE_IMAGE_NAME}
    volumes:
      ...
    networks:
      - postgres_db
    environment:
      ...
    container_name: backend_1

networks:
  postgres_db:
    external: true
1
  • We had a similar issue in our project. docker-compose suddenly wanted our networks to become external docs.docker.com/compose/compose-file/06-networks/#external and now the documentation become true here: We have to create our networks manually and compose itself does not create them anymore. I wanted to proof this via the changelogs of docker but I did't find a real answer here too Commented Jul 19, 2023 at 9:23

2 Answers 2

8

This is now also a known issue for Docker Compose v2.24.6

https://github.com/docker/compose/issues/11533

This issue might be connected, too. https://github.com/docker/compose/issues/11510

Temporary quick fix

Download this binary of Docker Compose (two days older than the original): https://github.com/docker/compose/actions/runs/7994137875/artifacts/1264074044

Install it, for example in Ubuntu 20.04

Stop the Docker service, just in case:

sudo systemctl stop docker
sudo systemctl stop docker.service

Check that your current version is 2.24.6:

docker compose version
Docker Compose version v2.24.6

Find your current Docker Compose binary:

sudo find / -type f -name 'docker-compose*'

See what permissions and owner are set on your current binary:

ll /usr/libexec/docker/cli-plugins/docker-compose
-rwxr-xr-x 1 root root 61548555 feb 23 03:38 /usr/libexec/docker/cli-plugins/docker-compose*

Change the downloaded permissions and owner so they match:

chmod a+x docker-compose-linux-x86_64
chown root:root docker-compose-linux-x86_64

Check that the new binary has the same permissions and owner as the original:

ll docker-compose*
-rwxr-xr-x 1 root root 61439138 feb 21 18:59 docker-compose-linux-x86_64

Backup is always a good idea, for quick reverting. Backup the original binary:

cp -rp /usr/libexec/docker/cli-plugins/docker-compose docker-compose-ORIG

Copy the downloaded version, overwriting the existing binary:

sudo cp docker-compose-linux-x86_64 /usr/libexec/docker/cli-plugins/docker-compose

Check the new version:

docker compose version
Docker Compose version c5a88de

Start the Docker service:

sudo systemctl start docker
2
3

This is a known issue for the Docker Compose plugin v2.19.1. It was fixed in v2.20.1, which might not be available in the package repositories of your system at the time of writing. If you require an immediate fix, you can install the plugin manually by following the instructions here.

1
  • 1
    Hello. Running version 2.24.6 and have the same issue. I can not reproduce it. After a "docker compose up -d --force-recreate" the problem is gone. Commented Mar 4 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.