19

I have a dockerized setup running a Django app within which I use Celery tasks. Celery uses Redis as the broker.

Versioning:

  • Docker version 17.09.0-ce, build afdb6d4
  • docker-compose version 1.15.0, build e12f3b9
  • Django==1.9.6
  • django-celery-beat==1.0.1
  • celery==4.1.0
  • celery[redis]
  • redis==2.10.5

Problem:

My celery workers appear to be unable to connect to the redis container located at localhost:6379. I am able to telnet into the redis server on the specified port. I am able to verify redis-server is running on the container.

When I manually connect to the Celery docker instance and attempt to create a worker using the command celery -A backend worker -l info I get the notice:

[2017-11-13 18:07:50,937: ERROR/MainProcess] consumer: Cannot connect to redis://localhost:6379/0: Error 99 connecting to localhost:6379. Cannot assign requested address.. Trying again in 4.00 seconds...

Notes:

I am able to telnet in to the redis container on port 6379. On the redis container, redis-server is running.

Is there anything else that I'm missing? I've gone pretty far down the rabbit hole, but feel like I'm missing something really simple.

DOCKER CONFIG FILES:

docker-compose.common.yml here
docker-compose.dev.yml here

9
  • Try replacing localhost with 127.0.0.1
    – warvariuc
    Commented Nov 13, 2017 at 19:49
  • Do you have a docker-compose file? Share it.
    – Opal
    Commented Nov 13, 2017 at 19:54
  • @Opal added config files to bottom of post.
    – user1026996
    Commented Nov 13, 2017 at 20:01
  • @warvariuc same result when I change to 127.0.0.1. Also of note, using the docker IP address for the redis container also does not work.
    – user1026996
    Commented Nov 13, 2017 at 20:02
  • It will definitely not be localhost since redis is not working doesn't work on the same host as celery. Can you ping redis via IP from celery?
    – Opal
    Commented Nov 13, 2017 at 20:07

3 Answers 3

42

When you use docker-compose, you aren't going to be using localhost for inter-container communication, you would be using the compose-assigned hostname of the container. In this case, the hostname of your redis container is redis. The top level elements under services: are your default host names.

So for celery to connect to redis, you should try redis://redis:6379/0. Since the protocol and the service name are the same, I'll elaborate a little more: if you named your redis service "butter-pecan-redis" in your docker-compose, you would instead use redis://butter-pecan-redis:6379/0.

Also, docker-compose.dev.yml doesn't appear to have celery and redis on a common network, which might cause them not to be able to see each other. I believe they need to share at least one network in common to be able to resolve their respective host names.

Networking in docker-compose has an example in the first handful of paragraphs, with a docker-compose.yml to look at.

4
  • They did not share a common network, which helped resolve the issue.
    – user1026996
    Commented Nov 14, 2017 at 0:42
  • if two of my containers are in the same network, this redis://butter-pecan-redis:6379/0 way of connecting still work??(I don't want to use docker-compose) Commented Jan 4, 2021 at 16:36
  • @AATHITHRAJENDRAN No, it won't. Docker inherits the DNS of the host, so unless you have a DNS entry for butter-pecan-redis mapped to localhost:<port> somewhere outside of Docker, one container won't resolve another container by name. Docker-compose does something extra that allows this to work, I can't remember if it's something with a DNS per Docker network in compose, or if docker-compose adjusts hosts or resolv.conf. Either way, the simple by-name resolution is a docker-compose feature.
    – bluescores
    Commented Jan 6, 2021 at 20:39
  • This was the missing piece of my puzzle to finally getting Docker, Django, Celery and Redis to work together from a docker-compose.yml file. Thanks for explaining this!
    – Jarad
    Commented Apr 30, 2021 at 7:32
3

You may need to add the link and depends_on sections to your docker compose file, and then reference the containers by their hostname.

Updated docker-compose.yml:

version: '2.1'
services:
    db:
        image: postgres
    memcached:
        image: memcached
    redis:
        image: redis
        ports:
          - '6379:6379'
    backend-base:
        build:
            context: .
            dockerfile: backend/Dockerfile-base
        image: "/backend:base"
    backend:
        build:
            context: .
            dockerfile: backend/Dockerfile
        image: "/backend:${ENV:-local}"
        command: ./wait-for-it.sh db:5432 -- gunicorn backend.wsgi:application -b 0.0.0.0:8000 -k gevent -w 3
        ports:
            - 8000
        links:
            - db
            - redis
            - memcached
        depends_on:
            - db
            - redis
            - memcached
    celery:
        image: "/backend:${ENV:-local}"
        command: ./wait-for-it.sh db:5432 -- celery worker -E -B --loglevel=INFO --concurrency=1
        environment:
            C_FORCE_ROOT: "yes"
        links:
            - db
            - redis
            - memcached
        depends_on:
            - db
            - redis
            - memcached
    frontend-base:
        build:
            context: .
            dockerfile: frontend/Dockerfile-base
            args:
                NPM_REGISTRY: http://.view.build
                PACKAGE_INSTALLER: yarn
        image: "/frontend:base"
        links:
            - db
            - redis
            - memcached
        depends_on:
            - db
            - redis
            - memcached
    frontend:
        build:
            context: .
            dockerfile: frontend/Dockerfile
        image: "/frontend:${ENV:-local}"
        command: 'bash -c ''gulp'''
        working_dir: /app/user
        environment:
            PORT: 3000
        links:
            - db
            - redis
            - memcached
        depends_on:
            - db
            - redis
            - memcached

Then configure the urls to redis, postgres, memcached, etc. with:

  • redis://redis:6379/0
  • postgres://user:pass@db:5432/database
1
  • Hey, thanks for your contribution. What do you mean exactly by "...configure the urls to redis, postgres, memcached..."? Commented Jun 18, 2019 at 20:55
0

The issue for me was that all of the containers, including celery had a network argument specified. If this is the case the redis container must also have the same argument otherwise you will get this error. See below, the fix was adding 'networks':

  redis:
    image: redis:alpine
    ports:
      - '6379:6379'
    networks:
      - server

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.