25

I have a following setup in docker compose

  • nginx for proxying to frontend, backend and serving static content
  • backend app on port 8080 (spring boot)
  • frontend app on port 4000 (node for SSR)
  • mysql used by backend

Frontend can be updated relatively fast using

docker-compose up -d --no-deps frontend

Unfortunately backend takes about 1 minute to start.

Is there an easy way to achieve lower downtime without having to change the current setup too much? I like how simple it is right now.

I would imagine something like this:

  1. Start a new instance of backend
  2. Wait till it starts (it could be per timer or a healthtest)
  3. Close the previously running instance
2
  • 3
    You might be better off with Swarm, but also could look into why it takes the backend service so long to start.
    – ldg
    Mar 4, 2018 at 0:11
  • This is extremely interesting! Would you share your docker-compose.yml file? Sep 26, 2019 at 15:09

2 Answers 2

17

Swarm is the right solution to go, but this is still painfully doable with docker-compose.

First, ensure your proxy can do service discovery. You can't use container_name (as you can't use it in swarm) because your will increase the number of container of the same service. Proxy like traefik or nginx-proxy uses labels to do this.

Then, docker-compose up -d --scale backend=2 --no-recreate this creates a new container with the new image without touching the running one.

After it's up and running, docker kill old_container, then docker-compose up -d --scale backend=1 --no-recreate just to reset the number.


EDIT 1

docker kill old_container should be docker rm -f old_container

EDIT 2

how to handle even and not even runs

We want to always kill the oldest containers

docker rm -f $(docker ps --format "table {{.ID}}  {{.Names}}  {{.CreatedAt}}" | grep backend | (read -r; printf "%s\n" "$REPLY"; sort -k 3 ) | awk -F  "  " '{print $1}' | head -1)
11
  • I've tried this and unfortunately it doesn't work. Scaling up works perfectly, but killing the container causes problems. The service stops responding at all instead of forwarding the traffic to the running container. It is a shame that scaling down to 1 container always stops the newer container instead of the older one. Jan 21, 2019 at 11:40
  • @MarcinKunert I think the issue is on your nginx config. How do you config your reverse proxy?
    – Siyu
    Jan 21, 2019 at 11:42
  • server { listen 80; location /api { proxy_pass backend:8080; } Jan 21, 2019 at 11:43
  • 1
    You are reading my mind! I was looking how to kill the last container. Jan 21, 2019 at 13:21
  • 1
    It finally works. Instead of looking for the older container I've decided to save the id of the existing container before starting the script. I've also added a chance for the backend to exit gracefully docker kill -s SIGTERM $PREVIOUS_CONTAINER. Thanks for help! Jan 21, 2019 at 13:41
12

Here is the script I've ended up using:

PREVIOUS_CONTAINER=$(docker ps --format "table {{.ID}}  {{.Names}}  {{.CreatedAt}}" | grep backend | awk -F  "  " '{print $1}')
docker-compose up -d --no-deps --scale backend=2 --no-recreate backend
sleep 100
docker kill -s SIGTERM $PREVIOUS_CONTAINER
sleep 1
docker rm -f $PREVIOUS_CONTAINER
docker-compose up -d --no-deps --scale backend=1 --no-recreate backend
docker-compose stop http-nginx
docker-compose up -d --no-deps --build http-nginx
5
  • Why do you need to restart http-nginx? Sep 26, 2019 at 15:10
  • 1
    @AugustinRiedinger without restarting nginx is unable to route the traffic to newly started backend. I've read about plugins to do it properly, but restarting was a lot easier for me Sep 27, 2019 at 8:35
  • @MarcinKunert this should solve the dns cache problem without restarting nginx serverfault.com/a/916786/130859
    – Jacer Omri
    Feb 14, 2020 at 15:50
  • 3
    A simpler way to get the ID of the currently running container for a service called backend, is to use docker-compose ps -q backend. Also, in case you have other Docker containers running containing that name, then it won't return their container IDs, it will just return the ID of the backend from your Docker Compose file :) Oct 10, 2020 at 23:41
  • 1
    @Bart van Oort thanks for the tip! I'll try to incorporate it :) Oct 12, 2020 at 10:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.