615

I have two separate docker-compose.yml files in two different folders:

  • ~/front/docker-compose.yml
  • ~/api/docker-compose.yml

How can I make sure that a container in front can send requests to a container in api?

I know that --default-gateway option can be set using docker run for an individual container, so that a specific IP address can be assigned to this container, but it seems that this option is not available when using docker-compose.

Currently I end up doing a docker inspect my_api_container_id and look at the gateway in the output. It works but the problem is that this IP is randomly attributed, so I can't rely on it.

Another form of this question might thus be:

  • Can I attribute a fixed IP address to a particular container using docker-compose?

But in the end what I'm looking after is:

  • How can two different docker-compose projects communicate with each other?
2
  • 9
    I just looked into this today again. The devs have finally relented and allowed arbitrary network naming. Using compose file version 3.5 you can specify a name for the default network under the 'networks' key. This will create a named network without the usual project name prefix if it doesn't exist..
    – cstrutton
    Dec 29, 2017 at 14:17
  • be aware of what kind of connection you're planning to set. one-way (A->B or B->A; described in this question) or bidirectional A <-> B which is harder to achieve (requires wrapping docker files with extra logic and verifying whether the network exists or not due to mutual dependencies) Aug 17, 2022 at 18:24

20 Answers 20

717

You just need to make sure that the containers you want to talk to each other are on the same network. Networks are a first-class docker construct, and not specific to compose.

# front/docker-compose.yml
version: '2'
services:
  front:
    ...
    networks:
      - some-net
networks:
  some-net:
    driver: bridge

...

# api/docker-compose.yml
version: '2'
services:
  api:
    ...
    networks:
      - front_some-net
networks:
  front_some-net:
    external: true

Note: Your app’s network is given a name based on the “project name”, which is based on the name of the directory it lives in, in this case a prefix front_ was added

They can then talk to each other using the service name. From front you can do ping api and vice versa.

16
  • 4
    Robert Moskal only if you hack around to get the ip of your docker host into the containers. Better to have them communicate on a common docker-defined network. Jun 29, 2016 at 17:15
  • 16
    Please note that the "front_" prefix to the network is created automatically from the folder its running on. So if your first docker-compose file would be located in "example/docker-compose.yml" it would be called "example_default" instead. Dec 6, 2018 at 17:09
  • 66
    You can also provide a name to a network using name property, which will disable automatic prepending with project name. Then either project can use that network and create it automatically if it does not yet exist.
    – SteveB
    Feb 13, 2019 at 18:40
  • 7
    @SteveB - Note that the name property only works from docker-compose files version 3.5 and up
    – kramer65
    Feb 22, 2020 at 11:30
  • 8
    Just curious, what happen if we have same service name but on different network, can the caller container, be more explicit about which network it want to use to call a particular service? Apr 5, 2020 at 14:05
263

UPDATE: As of compose file version 3.5:

This now works:

version: "3.5"
services:
  proxy:
    image: hello-world
    ports:
      - "80:80"
    networks:
      - proxynet

networks:
  proxynet:
    name: custom_network

docker-compose up -d will join a network called 'custom_network'. If it doesn't exist, it will be created!

root@ubuntu-s-1vcpu-1gb-tor1-01:~# docker-compose up -d
Creating network "custom_network" with the default driver
Creating root_proxy_1 ... done

Now, you can do this:

version: "2"
services:
  web:
    image: hello-world
    networks:
      - my-proxy-net
networks:
  my-proxy-net:
    external:
      name: custom_network

This will create a container that will be on the external network.

I can't find any reference in the docs yet but it works!

14
  • 9
    The first service (proxy above) creates the network. The syntax in the second example joins it.
    – cstrutton
    Jan 6, 2019 at 0:27
  • 7
    @slashdottir You can not mark the network as external in second service and it will be created if it does not yet exist.
    – SteveB
    Feb 13, 2019 at 19:03
  • 2
    It does work. I just spun up a DO droplet with the latest docker compose. I have edited the example to an actual working example.
    – cstrutton
    May 15, 2019 at 23:57
  • 4
    Here are the refs in the docs: docs.docker.com/compose/networking/#use-a-pre-existing-network
    – jeanggi90
    Mar 13, 2020 at 20:35
  • 5
    In my case, this turned out to be a more suitable solution than the accepted answer. The problem with external network was, that it required to start containers in predefined order. For my client, this was not acceptable. A named network (since 3.5) turned out to be perfect solution. Thanks.
    – ygor
    May 28, 2020 at 7:37
124

Just a small adittion to @johnharris85's great answer, when you are running a docker compose file, a "default" network is created so you can just add it to the other compose file as an external network:

# front/docker-compose.yml 
version: '2' 
  services:   
    front_service:
    ...

...

# api/docker-compose.yml
version: '2'
services:
  api_service:
    ...
    networks:
      - front_default
networks:
  front_default:
    external: true

For me this approach was more suited because I did not own the first docker-compose file and wanted to communicate with it.

3
  • just wandering the correct way to assign static IP for this external network. I manged to do it within services: tag, the sintax would be networks: then nested front_default: (remove the "-") and then we nest a static IP: ipv4_address: '172.20.0.44' Feb 15, 2018 at 16:47
  • 1
    Although this is true, there may come a time when this does not work. You are relying on an implementation detail. Who is to say that they won't change the way default networks are named in a future version. Creating an explicitly named network is documented and will likely be supported for some time.
    – cstrutton
    Jul 24, 2020 at 15:49
  • @cstrutton The default network name is also documented.
    – Navin
    Oct 17 at 10:13
47

All containers from api can join the front default network with following config:

# api/docker-compose.yml

...

networks:
  default:
    external:
      name: front_default

See docker compose guide: using a pre existing network (see at the bottom)

39

Everybody has explained really well, so I'll add the necessary code with just one simple explanation.

Use a network created outside of docker-compose (an "external" network) with docker-compose version 3.5+.

Further explanation can be found here.

First docker-compose.yml file should define network with name giveItANamePlease as follows.

networks:
  my-network:
    name: giveItANamePlease
    driver: bridge

The services of first docker-compose.yml file can use network as follows:

networks:
  - my-network

In second docker-compose file, we need to proxy the network by using the network name which we have used in first docker-compose file, which in this case is giveItANamePlease:

networks:
  my-proxy-net:
    external:
      name: giveItANamePlease

And now you can use my-proxy-net in services of a second docker-compose.yml file as follows.

networks:
  - my-proxy-net
2
  • 4
    Finally a modern/easy-to-use answer
    – ciurlaro
    Mar 11, 2022 at 14:47
  • 2
    Make the networks section in the second file match the first (remove reference to external) and it won't matter which project starts first (allow either to auto-create the network)
    – rymo
    Jan 11 at 20:32
29

The previous posts information is correct, but it does not have details on how to link containers, which should be connected as "external_links".

Hope this example make more clear to you:

  • Suppose you have app1/docker-compose.yml, with two services (svc11 and svc12), and app2/docker-compose.yml with two more services (svc21 and svc22) and suppose you need to connect in a crossed fashion:

  • svc11 needs to connect to svc22's container

  • svc21 needs to connect to svc11's container.

So the configuration should be like this:

this is app1/docker-compose.yml:


version: '2'
services:
    svc11:
        container_name: container11
        [..]
        networks:
            - default # this network
            - app2_default # external network
        external_links:
            - container22:container22
        [..]
    svc12:
       container_name: container12
       [..]

networks:
    default: # this network (app1)
        driver: bridge
    app2_default: # external network (app2)
        external: true

this is app2/docker-compose.yml:


version: '2'
services:
    svc21:
        container_name: container21
        [..]
        networks:
            - default # this network (app2)
            - app1_default # external network (app1)
        external_links:
            - container11:container11
        [..]
    svc22:
       container_name: container22
       [..]

networks:
    default: # this network (app2)
        driver: bridge
    app1_default: # external network (app1)
        external: true
3
  • Thank you for this details explanation in docker network it is very useful. Whole day I was struggling for proper explanation and solution but now I understood the concept. Jul 10, 2020 at 15:31
  • Yes it does. I am using this configuration from 2 to 3.7 Sep 17, 2020 at 4:52
  • "Links" became an obsolete Docker concept once Docker networks were introduced and Compose file version 2 made them available. If you do have an external_links: option as suggested in this answer, it's probably safe to delete it.
    – David Maze
    Jul 12 at 11:10
15

Since Compose 1.18 (spec 3.5), you can just override the default network using your own custom name for all Compose YAML files you need. It is as simple as appending the following to them:

networks:
  default:
    name: my-app

The above assumes you have version set to 3.5 (or above if they don't deprecate it in 4+).

Other answers have pointed the same; this is a simplified summary.

3
  • 1
    important to note here is that the identifier of the network is still default. you are not able to set aliases or something like that to the network my-app. You need to use default instead
    – gustavz
    Sep 14, 2021 at 7:47
  • 1
    Adding to the above comment from @gustavz, using default as the network identifier allows not needing to specify a network to use in services. The default network name my-app could be anything to help group multiple Compose services.
    – emyller
    Nov 17, 2021 at 16:08
  • Beware that all of the services are also accessible by their keys as aliases on this same network and there is no guarantee which will resolve for a particular request. See the note in the aliases section here: docs.docker.com/compose/compose-file/#networks
    – rymo
    Jan 11 at 20:35
15

So many answers!

First of all, avoid hyphens in entities names such as services and networks. They cause issues with name resolution.

Example: my-api won't work. myapi or api will work.

What worked for me is:

# api/docker-compose.yml
version: '3'

services:
  api:
    container_name: api
    ...
    ports:
      - 8081:8080
    networks:
      - mynetwork

networks:
  mynetwork:
    name: mynetwork

and

# front/docker-compose.yml
version: '3'

services:
  front:
    container_name: front
    ...
    ports:
      - 81:80
    networks:
      - mynetwork

networks:
  mynetwork:
    name: mynetwork

NOTE: I added ports to show how services can access each other, and how they are accessible from the host.

IMPORTANT: If you don't specify a network name, docker-compose will craft one for you. It uses the name of the folder the docker_compose.yml file is in. In this case: api_mynetwork and front_mynetwork. That will prevent communication between containers since they will by on different network, with very similar names.

Note that the network is defined exactly the same in both file, so you can start either service first and it will work. No need to specify which one is external, docker-compose will take care of managing that for you.

From the host

You can access either container using the published ports defined in docker-compose.yml.

You can access the Front container: curl http://localhost:81

You can access the API container: curl http://localhost:8081

From the API container

You can access the Front container using the original port, not the one you published in docker-compose.yml.

Example: curl http://front:80

From the Front container

You can access the API container using the original port, not the one you published in docker-compose.yml.

Example: curl http://api:8080

4
  • Seems like name attribute was added from version 3.5. Executing docker network list shows the correct network name. Thanks!
    – Lukenzo
    Feb 15, 2022 at 17:40
  • @Gael I've just tried to test your solution to connect Laravel (sail up) as api and vue as front projects. But looks like there are still two different containers and no connections between them. Have you test these frameworks together?
    – Andrew
    May 30, 2022 at 10:26
  • @Andrew No I have never tried those. Try docker ps to see what container are running and what ports are open, to make sure you have set the ports properly.
    – Gael
    May 31, 2022 at 14:06
  • The most simple config to connect two (or more) docker compose projects.
    – QkiZ
    Oct 5 at 17:07
11

UPDATE: As of docker-compose file version 3.5:

I came across a similar problem and I solved it by adding a small change in one of my docker-compose.yml project.

For instance, we have two API's scoring and ner. Scoring API needs to send a request to the ner API for processing the input request. In order to do that they both are supposed to share the same network.

Note: Every container has its own network which is automatically created at the time of running the app inside docker. For example ner API network will be created like ner_default and scoring API network will be named as scoring default. This solution will work for version: '3'.

As in the above scenario, my scoring API wants to communicate with ner API then I will add the following lines. This means Whenever I create the container for ner API then it automatically added to the scoring_default network.

networks:
  default:
      external:
        name: scoring_default

ner/docker-compose.yml

version: '3'
services:
  ner:
    container_name: "ner_api"
    build: .
    ...

networks:
  default:
      external:
        name: scoring_default

scoring/docker-compose.yml

version: '3'
services:
  api:
    build: .
    ...

We can see this how the above containers are now a part of the same network called scoring_default using the command:

docker inspect scoring_default

{
    "Name": "scoring_default",
        ....
    "Containers": {
    "14a6...28bf": {
        "Name": "ner_api",
        "EndpointID": "83b7...d6291",
        "MacAddress": "0....",
        "IPv4Address": "0.0....",
        "IPv6Address": ""
    },
    "7b32...90d1": {
        "Name": "scoring_api",
        "EndpointID": "311...280d",
        "MacAddress": "0.....3",
        "IPv4Address": "1...0",
        "IPv6Address": ""
    },
    ...
}
3
  • 1
    Nice now if u want scoring_api to talk to ner_api would it be http://scoring_api:port? or should it still be localhost?
    – Kay
    Aug 21, 2020 at 13:20
  • 2
    both apps can use their names to communicate with each other. For example, the scoring app can call ner app using "ner_api:port/extract" and vice versa
    – Nomiluks
    Aug 21, 2020 at 23:05
  • Thank you very much for docker inspect and pointing that endpoints should be accessed by containers name_ not by localhost! May 25 at 23:16
6

You can add a .env file in all your projects containing COMPOSE_PROJECT_NAME=somename.

COMPOSE_PROJECT_NAME overrides the prefix used to name resources, as such all your projects will use somename_default as their network, making it possible for services to communicate with each other as they were in the same project.

NB: You'll get warnings for "orphaned" containers created from other projects.

1
  • This is a fantastic tip, thanks a ton!
    – Gavin Ray
    Mar 14, 2022 at 17:57
5

For using another docker-compose network you just do these(to share networks between docker-compose):

  1. Run the first docker-compose project by up -d
  2. Find the network name of the first docker-compose by: docker network ls(It contains the name of the root directory project)
  3. Then use that name by this structure at below in the second docker-compose file.

second docker-compose.yml

version: '3'
services:
  service-on-second-compose:  # Define any names that you want.
    .
    .
    .
    networks:
      - <put it here(the network name that comes from "docker network ls")>

networks:
  - <put it here(the network name that comes from "docker network ls")>:
    external: true
5

To connect two docker-compose you need a network and putting both docker-composes in that network, you could create netwrok with docker network create name-of-network,

or you could simply put network declaration in networks option of docker-compose file and when you run docker-compose (docker-compose up) the network would be created automatically.

put the below lines in both docker-compose files

networks:
   net-for-alpine:
     name: test-db-net

Note: net-for-alpine is internal name of the network and it will be used inside of the docker-compose files and could be different, test-db-net is external name of the network and must be same in two docker-compose files.

Assume we have docker-compose.db.yml and docker-compose.alpine.yml

docker-compose.apline.yml would be:

version: '3.8'

services:

  alpine:
    image: alpine:3.14
    container_name: alpine
    networks:
      - net-for-alpine
  
    # these two command keeps apline container running
    stdin_open: true # docker run -i
    tty: true # docker run -t



networks:
  net-for-alpine:
    name: test-db-net

docker-compose.db.yml would be:

version: '3.8'

services:

  db:
    image: postgres:13.4-alpine
    container_name: psql
    networks:
      - net-for-db
  
networks:
  net-for-db:
    name: test-db-net

To test the network, go inside alpine container

docker exec -it alpine sh 
      

then with following commands you could check the network

# if it returns 0 or see nothing as a result, network is established
nc -z psql (container name)  

or

ping pgsql
4

I would ensure all containers are docker-compose'd to the same network by composing them together at the same time, using:

docker compose --file ~/front/docker-compose.yml --file ~/api/docker-compose.yml up -d
10
  • Will that allow me, for instance, to make a link or depends_on from one container of front to one container of api?
    – Jivan
    Jun 29, 2016 at 10:11
  • actually when I do what you suggest, docker-compose replies either build path ~/front/api either does not exist or is not accessible or with the other way around, build path ~/api/front either does not exist or is not accessible
    – Jivan
    Jun 29, 2016 at 11:48
  • 1
    If you're composing them at the same time you shouldn't need to. A network will be created with all your containers on it, they will all be able to communicate via the service name from the compose file (not the container name). Jun 29, 2016 at 11:50
  • It might be easier if the two compose files are in the same folder. But I don't think that's necessary - I think it should work either way. Jun 29, 2016 at 11:52
  • 2
    This solution does not work, see my comment on this thread: github.com/docker/compose/issues/3530#issuecomment-222490501 Jun 29, 2016 at 17:17
3

If you are

  • trying to communicate between two containers from different docker-compose projects and don't want to use the same network (because let's say they would have PostgreSQL or Redis container on the same port and you would prefer to not changing these ports and not use it at the same network)
  • developing locally and want to imitate communication between two docker compose projects
  • running two docker-compose projects on localhost
  • developing especially Django apps or Django Rest Framework (drf) API and running app inside container on some exposed port
  • getting Connection refused while trying to communicate between two containers

And you want to

  • container api_a communicate to api_b (or vice versa) without the same "docker network"

(example below)

you can use "host" of the second container as IP of your computer and port that is mapped from inside Docker container. You can obtain IP of your computer with this script (from: Finding local IP addresses using Python's stdlib):

import socket
def get_ip():
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    try:
        # doesn't even have to be reachable
        s.connect(('10.255.255.255', 1))
        IP = s.getsockname()[0]
    except:
        IP = '127.0.0.1'
    finally:
        s.close()
    return IP

Example:

project_api_a/docker-compose.yml:

networks:
  app-tier:
    driver: bridge

services:
  api:
    container_name: api_a
    image: api_a:latest
    depends_on:
      - postgresql
    networks:
      - app-tier

inside api_a container you are running Django app: manage.py runserver 0.0.0.0:8000

and second docker-compose.yml from other project:

project_api_b/docker-compose-yml :

networks:
  app-tier:
    driver: bridge

services:
  api:
    container_name: api_b
    image: api_b:latest
    depends_on:
      - postgresql
    networks:
      - app-tier

inside api_b container you are running Django app: manage.py runserver 0.0.0.0:8001

And trying to connect from container api_a to api_b then URL of api_b container will be: http://<get_ip_from_script_above>:8001/

It can be especially valuable if you are using even more than two(three or more) docker-compose projects and it's hard to provide common network for all of it - it's good workaround and solution

3

I'm running multiple identical docker-compose.yml files in different directories, using .env files to make a slight difference. And use Nginx Proxy Manage to communicate with other services. here is my file:

make sure you have created a public network

docker network create nginx-proxy-man

/domain1.com/docker-compose.yml, /domain2.com/docker-compose.yml, ...

version: "3.9"

services:
  webserver:
    build:
      context: ./bin/${PHPVERSION}
    container_name: "${COMPOSE_PROJECT_NAME}-${PHPVERSION}"
    ...
    networks:
      - default    # network outside
      - internal   # network internal
  database:
    build:
      context: "./bin/${DATABASE}"
    container_name: "${COMPOSE_PROJECT_NAME}-${DATABASE}"
    ...
    networks:
      - internal   # network internal


networks:
  default:
    external: true
    name: nginx-proxy-man
  internal:
    internal: true

.env file just change COMPOSE_PROJECT_NAME

COMPOSE_PROJECT_NAME=domain1_com
.
.
.
PHPVERSION=php56

DATABASE=mysql57

webserver.container_name: domain1_com-php56 - will join the default network (name: nginx-proxy-man), previously created for Nginx Proxy Manager to be accessible from the outside.

Note: container_name is unique in the same network.

database.container_name: domain1_com-mysql57 - easier to distinguish

In the same docker-compose.yml, the services will connect to each other via the service name because of the same network domain1_com_internal. And to be more secure, set this network with the option internal: true

Note, if you don't explicitly specify networks for each service, but just use a common external network for both docker-compose.yml, then it's likely that domain1_com will use domain2_com's database.

1

Follow up of JohnHarris answer, just adding some more details which may be useful to someone: Lets take two docker-compose file and connect them through networks:

  1. 1st foldername/docker-compose.yml:
version: '2'
services:
  some-contr:
    container_name: []
    build: .
    ...
    networks:
      - somenet
    ports:
      - "8080:8080"
    expose:
      # Opens port 8080 on the container
      - "8080"
    environment:
      PORT: 8080
    tty: true
networks:
  boomnet:
    driver: bridge
  1. 2nd docker-compose.yml:
version: '2'
services: 
  pushapiserver:
    container_name: [container_name]
    build: .
    command: "tail -f /dev/null"
    volumes:
      - ./:/[work_dir]
    working_dir: /[work dir]
    image: [name of image]
    ports:
      - "8060:8066"
    environment:
      PORT: 8066
    tty: true
    networks:
      - foldername_somenet
networks:
  foldername_somenet:
    external: true

Now you can make api calls to one another services(b/w diff containers)like: http://pushapiserver:8066/send_push call from some code in files for 1st docker-compose.yml

Two common mistakes (atleast i made them few times):

  1. take note of [foldername] in which your docker-compose.yml file is present. Please see above in 2nd docker-compose.yml i have added foldername in network bc docker create network by [foldername]_[networkname]
  2. Port: this one is very common. Please note i have used 8066 when trying to make connection i.e. http://pushapiserver:8066/... 8066 is port of docker container(2nd docker-compose.yml) so when trying to talk with different docker compose.

docker will use docker container port[8066] and not host machine mapped port [8060]

0

Another option is just running up the first module with the 'docker-compose' check the ip related with the module, and connect the second module with the previous net like external, and pointing the internal ip

example app1 - new-network created in the service lines, mark as external: true at the bottom app2 - indicate the "new-network" created by app1 when goes up, mark as external: true at the bottom, and set in the config to connect, the ip that app1 have in this net.

With this, you should be able to talk with each other

*this way is just for local-test focus, in order to don't do an over complex configuration ** I know is very 'patch way' but works for me and I think is so simple some other can take advantage of this

0

I have had a similar example where I was working with separate docker-compose files working on a docker swarm with an overlay network to do that all I had to do is change the networks parameters as so:

first docker-compose.yaml

version: '3.9'
.
.
.

networks:
  net:
    driver: overlay
    attachable: true
docker-compose -p app up

since I have specified the app name as app using -p the initial network will be app_net. Now in order to run another docker-compose with multiple services that will use the same network you will need to set these as so:

second docker-compose.yaml

version: '3.9'
.
.
.
networks:
  net-ref:
    external: true
    name: app_net
docker stack deploy -c docker-compose.yml mystack

No matter what name you give to the stack the network will not be affected and will always refer to the existing external network called app_net.

PS: It's important to make sure to check your docker-compose version.

0

Here is an example that uses IP Addresses. The first docker compose should create the network that future containers can join. Here is a snippet code snippet.

version: "3"
services:
  app:
    image: "jc21/nginx-proxy-manager:latest"
    restart: unless-stopped
    ports:
      - "80:80"
      - "81:81"
      - "443:443"
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
    networks:
      customnetwork:
        ipv4_address: 172.20.0.10
networks:
  customnetwork:
    ipam:
      config:
        - subnet: 172.20.0.0/24

The second docker-compose should join the network that was created:

version: "3"
services:
  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    command: -H unix:///var/run/docker.sock
    ports:
      - 9000:9000
      - 9443:9443
    volumes:
      - portainer_data:/data
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      nginxproxymanager_customnetwork:
        ipv4_address: 172.20.0.11
    restart: unless-stopped
volumes:
  portainer_data:
networks:
  nginxproxymanager_customnetwork:
    external: true

Source: WordPress/MYSQL Docker Compose with Networking

1
  • You should almost never need to manually set the container-private IP addresses and that's not necessary for this setup.
    – David Maze
    May 1 at 12:37
-2
version: '2'
services:
  bot:
    build: .
    volumes:
      - '.:/home/node'
      - /home/node/node_modules
    networks:
      - my-rede
    mem_limit: 100m
    memswap_limit: 100m
    cpu_quota: 25000
    container_name: 236948199393329152_585042339404185600_bot
    command: node index.js
    environment:
      NODE_ENV: production
networks:
  my-rede:
    external:
      name: name_rede_externa

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.