353

I have a directory apkmirror-scraper-compose with the following structure:

.
├── docker-compose.yml
├── privoxy
│   ├── config
│   └── Dockerfile
├── scraper
│   ├── Dockerfile
│   ├── newnym.py
│   └── requirements.txt
└── tor
    └── Dockerfile

I'm trying to run the following docker-compose.yml:

version: '3'

services:
  privoxy:
    build: ./privoxy
    ports:
      - "8118:8118"
    links:
      - tor

  tor:
    build:
      context: ./tor
      args:
        password: ""
    ports:
      - "9050:9050"
      - "9051:9051"

  scraper:
    build: ./scraper
    links:
      - tor
      - privoxy

where the Dockerfile for tor is

FROM alpine:latest
EXPOSE 9050 9051
ARG password
RUN apk --update add tor
RUN echo "ControlPort 9051" >> /etc/tor/torrc
RUN echo "HashedControlPassword $(tor --quiet --hash-password $password)" >> /etc/tor/torrc
CMD ["tor"]

that for privoxy is

FROM alpine:latest
EXPOSE 8118
RUN apk --update add privoxy
COPY config /etc/privoxy/config
CMD ["privoxy", "--no-daemon"]

where config consists of the two lines

listen-address 0.0.0.0:8118
forward-socks5 / tor:9050 .

and the Dockerfile for scraper is

FROM python:2.7-alpine
ADD . /scraper
WORKDIR /scraper
RUN pip install -r requirements.txt
CMD ["python", "newnym.py"]

where requirements.txt contains the single line requests. Finally, the program newnym.py is designed to simply test whether changing the IP address using Tor is working:

from time import sleep, time

import requests as req
import telnetlib


def get_ip():
    IPECHO_ENDPOINT = 'http://ipecho.net/plain'
    HTTP_PROXY = 'http://privoxy:8118'
    return req.get(IPECHO_ENDPOINT, proxies={'http': HTTP_PROXY}).text


def request_ip_change():
    tn = telnetlib.Telnet('tor', 9051)
    tn.read_until("Escape character is '^]'.", 2)
    tn.write('AUTHENTICATE ""\r\n')
    tn.read_until("250 OK", 2)
    tn.write("signal NEWNYM\r\n")
    tn.read_until("250 OK", 2)
    tn.write("quit\r\n")
    tn.close()


if __name__ == '__main__':
    dts = []
    try:
        while True:
            ip = get_ip()
            t0 = time()
            request_ip_change()
            while True:
                new_ip = get_ip()
                if new_ip == ip:
                    sleep(1)
                else:
                    break
            dt = time() - t0
            dts.append(dt)
            print("{} -> {} in ~{}s".format(ip, new_ip, int(dt)))
    except KeyboardInterrupt:
        print("Stopping...")
        print("Average: {}".format(sum(dts) / len(dts)))

The docker-compose build builds successfully, but if I try docker-compose up, I get the following error message:

Creating network "apkmirrorscrapercompose_default" with the default driver
ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network

I tried searching for help on this error message, but couldn't find any. What is causing this error?

11
  • 1
    No, docker ps shows no containers running.
    – Kurt Peek
    Commented May 1, 2017 at 14:08
  • 181
    Do you have a VPN connected? Also, have you tried restarting your compueter? (I am googling) github.com/moby/moby/issues/30295
    – Robert
    Commented May 1, 2017 at 14:18
  • 4
    Can you try docker network ls and confirm if networks were already created on your host. Commented May 1, 2017 at 14:37
  • 1
    @Robert, in my case was the VPN. Disabling it, solved the issue.
    – user1329187
    Commented Nov 4, 2017 at 20:06
  • 73
    docker network prune. This will resolve your issue Commented Jul 9, 2019 at 9:35

20 Answers 20

616

I've seen it suggested docker may be at its maximum of created networks. The command docker network prune can be used to remove all networks not used by at least one container.

My issue ended up being, as Robert commented about: an issue with openvpn service openvpn stop 'solved' the problem.

4
  • 29
    If you need to run docker alongside vpn here is possible solution: stackoverflow.com/q/45692255/7918.
    – jb.
    Commented Aug 15, 2017 at 14:34
  • 2
    To add to the above answer, if you get any issue like this pruning the system can really help. Docker system prune can also be a fix, but please be careful, this can remove your DB, only use this if you don't care for you DB, or if your DB container is running then this command is safe as it only prunes things not being used by at least one container. Commented Jul 25, 2018 at 23:11
  • Docker Can Only Create 31 Default Networks Commented Nov 29, 2019 at 19:19
  • For anyone who still has the issue after stopping VPN and running docker network prune, you may have to restart your PC. That solved it for me!
    – Benny64
    Commented Jan 27, 2021 at 2:24
253

I ran into this problem because I had OpenVPN running. As soon as I killed OpenVPN, docker-compose up fired right up, and the error disappeared.

4
  • 1
    I've added routes rather than redirect-gateway def1 to get around the issue without killing my openvpn service. Commented Feb 2, 2019 at 14:50
  • 2
    same problem because because I had VPN called globalprotect from paloaltonetworks running. As soon as I disconnect VPN the error disappeared. After the docker network is created you can enable your VPN again.
    – Sma Ma
    Commented Feb 10, 2020 at 16:25
  • same problem with Speedify VPN v9.9.0.8564 on Ubuntu 20.04. I don't think I saw this issue on Ubuntu 18.04 whilst running Speedify though. Commented May 25, 2020 at 22:19
  • GlobalProtect causes same issue.
    – Xoyce
    Commented Mar 3, 2022 at 14:59
176

I ran in this problem with OpenVPN working as well and I've found a solution where you should NOT stop/start OpenVPN server.

Idea that You should specify what exactly subnet you want to use. In docker-compose.yml write:

networks:
  default:
    driver: bridge
    ipam:
      config:
        - subnet: 172.16.57.0/24

That's it. Now, default network will be used and if your VPN did not assign you something from 172.16.57.* subnet, you're fine.

8
  • 7
    This is great! My server box can't stand without OpenVPN so advices on (even temporarily) disabling VPN are all nonsense to me.
    – iBug
    Commented Jun 23, 2019 at 14:04
  • 31
    this should be the accepted answer, as simply killing VPN is stupid
    – michnovka
    Commented Jun 28, 2019 at 16:59
  • 2
    Isn't 172.177.57.0/24 a routed newtork? If so, it could cause problems contacting a limited number of hosts on the internet.
    – dstromberg
    Commented Dec 5, 2019 at 23:50
  • 3
    yep, a glitch here. Would be better to use one from 172.16.*.* subnet
    – Arenim
    Commented Dec 8, 2019 at 12:00
  • 1
    I don't think that using 172.177.57.* is a good idea, because it not in the en.wikipedia.org/wiki/Private_network#Private_IPv4_addresses. Use addresses within this range.
    – lucidyan
    Commented Feb 10, 2020 at 20:37
75

Following Peter Hauge's comment, upon running docker network ls I saw (among other lines) the following:

NETWORK ID          NAME                                    DRIVER              SCOPE
dc6a83d13f44        bridge                                  bridge              local
ea98225c7754        docker_gwbridge                         bridge              local
107dcd8aa889        host                                    host                local

The line with NAME and DRIVER as both host seems to be what he is referring to with "networks already created on your host". So, following https://gist.github.com/bastman/5b57ddb3c11942094f8d0a97d461b430, I ran the command

docker network rm $(docker network ls | grep "bridge" | awk '/ / { print $1 }')

Now docker-compose up works (although newnym.py produces an error).

3
  • 12
    Doesn't work in more recent docker versions- they've disallowed removing built-in networks (such as default)
    – pospi
    Commented Jun 29, 2017 at 2:50
  • If you're using Traefik, make sure to shut down that container before running this command. Otherwise it will think all of your networks are active. Commented Jul 17, 2019 at 14:44
  • 6
    if I run docker network rm $(docker network ls | grep "bridge" | awk '/ / { print $1 }')I get Error response from daemon: bridge is a pre-defined network and cannot be removedon Ubuntu 20.04 Docker version 19.03.9, build 9d988398e7 docker-compose version 1.25.0, build unknown Commented May 25, 2020 at 22:23
67

I have the same problem. I ran docker system prune -a --volumes, docker network prune, but neither helped me.

I use a VPN, I turned off the VPN and, after it docker started normal and was able to create a network. After that, you can enable VPN again.

3
  • 4
    My VPN connection was on the same subnet as the one docker was trying to use. Disconnecting solved the problem for me. :) Commented Dec 13, 2018 at 0:59
  • 1
    Also, solve for me after off the open vpn connection. Commented Jan 31, 2022 at 15:20
  • 2
    Careful with docker system prune -a --volumes , next time, you launch Docker Compose, it will have to redownload all volumes Commented Nov 23, 2022 at 13:31
57

TL;DR

Add

version: '3.7'
services:
  web:
    ...
    network_mode: bridge

Read about network_mode in the documentation.

Long version

Disclaimer: I am not very knowledgeable about Docker networking, but this did the trick for me. YMMV.

When I ran docker run my-image the networking gave me no problems, but when I converted this command to a docker-compose.yml file, I got the same error as the OP.

I read Arenim's answer and some other stuff on the internet that suggested to re-use an existing network.

You can find existing networks like this:

# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
ca0415dfa442        bridge              bridge              local
78cbbda034dd        host                host                local
709f13f4ce2d        none                null                local

I wanted to reuse the default bridge network, so I added

services:
  web:
    ...

networks:
  default:
    external:
      name: bridge

to the the root of my docker-compose.yml (so not inside one of my services, but at the root indentation).

I now got the following error:

ERROR: for your-container network-scoped alias is supported only for containers in user defined networks

This led met to this Docker Github issue, that plainly stated that I should add the network_mode object to my docker-compose:

version: '3.7'
services:
  web:
    ...
    network_mode: bridge

This was tested on Docker version 18.09.8, docker-compose version 1.24.1 and the compose file format 3.7.

3
  • 3
    is this a question or an answer?
    – chovy
    Commented Apr 29, 2020 at 6:31
  • 3
    More a: my adventures in Docker-network land and how I stumbled onto something that stuck. Commented Apr 29, 2020 at 7:22
  • Not everyone wants to use the bridge network
    – user903589
    Commented Jun 10, 2021 at 15:40
30

If you want lots of networks, then you can control how much IP space docker hands out to each network via the default-address-pools daemon setting, so you could add this to your /etc/docker/daemon.json:

{
  "bip": "10.254.1.1/24",
  "default-address-pools":[{"base":"10.254.0.0/16","size":28}]
}

Here I've reserved 10.254.1.1/24 (254 IP addresses) for the bridge network.

For any other network I create, docker will partition up the 10.254.0.0 space (65k hosts), giving out 16 hosts at a time ("size":28 refers to the CIDR mask, for 16 hosts).

If I create a few networks and then run docker network inspect <name> on them, it might display something like this:

   ...
   "Subnet": "10.254.0.32/28",
   "Gateway": "10.254.0.33"
   ...

The 10.254.0.32/28 means this network can use 16 ip addresses from 10.254.0.32 - 10.254.0.47.

2
  • 2
    Thanks a lot for the answer. Restarting the docker service is required after making the changes to /etc/docker/daemon.json. Commented Sep 15, 2020 at 18:55
  • This should be marked as the answer instead of ``` docker network prune ``` or re-configuring the default bridge in the compose which are not scalable. In this case, pool can always be added.
    – Dexter
    Commented Apr 2, 2021 at 13:40
20

Killing the vpn is not needed.

This other comment about using a new network comes pretty close to the solution for me, and was working for a while, but I found a better way thanks to some talk over in another question

Create a network with:

docker network create your-network --subnet 172.24.24.0/24

Then, at the bottom of docker-compose.yaml, put this:

networks:
  default:
    external: 
      name: your-network

Done. No need to add networks to all container definitions etc. and you can re-use the network with other docker-compose files as well if you'd like.

13

You can try

$sudo service network-manager restart

Worked for me.

0
13

I had an identical problem with the same error message but the solution with removal of unused docker networks didn't help me. I've deleted all non-default docker networks (and all images and containers as well) but it didn't help - docker still was not able to create a new network.

The cause of the problem was in network interfaces that were left after OpenVpn installation. (It was installed on the host previously.) I found them by running ifconfig command:

...
tun0  Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
      inet addr:10.8.0.2  P-t-P:10.8.0.2  Mask:255.255.255.0
      UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
      RX packets:75 errors:0 dropped:0 overruns:0 frame:0
      TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:100 
      RX bytes:84304 (84.3 KB)  TX bytes:0 (0.0 B)

tun1  Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
      inet addr:10.8.0.2  P-t-P:10.8.0.2  Mask:255.255.255.0
      UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
      RX packets:200496 errors:0 dropped:0 overruns:0 frame:0
      TX packets:148828 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:100 
      RX bytes:211583838 (211.5 MB)  TX bytes:9568906 (9.5 MB)
...

I've found that I can remove them with a couple of commands:

ip link delete tun0
ip link delete tun1

After this the problem has disappeared.

1
  • This worked for me after trying several options - stopped openvpn, restarted pc, reinstalled docker, network prune, etc...
    – Matt
    Commented Aug 12, 2019 at 2:42
10
  1. Check if any other container is running, If yes, do: docker-compose down
  2. If VPN is connected, then disconnect it and try again to up docker container:

    docker-compose up -d container_name
    
1
  • I've no VPN running but docker-compose down fixed it for me
    – IMB
    Commented Feb 4, 2020 at 16:32
9

I encoutered the same problem, the reason why is that you reached the max of networks:

do an : docker network ls Choose one to remove using: docker network rm networkname_default

9

Very good explanation of this issue is on this link.

As short the issue was appeared because the default Docker daemon has small IP addresses pool. And this issue can be solved by config /etc/docker/daemon.json with the next content:

 {
  "default-address-pools" : [
    {
      "base" : "172.17.0.0/12",
      "size" : 20
    },
    {
      "base" : "192.168.0.0/16",
      "size" : 24
    }
  ]
}
1
  • after editing Docker daemon config, you should also restart the daemon and probably re-create existed docker networks to shrink them
    – AntonioK
    Commented May 31 at 13:28
8

This happened to me because I was using OpenVPN. I found a way that I don't need to stop using the VPN or manually add a network to the docker-compose file nor run any crazy script.

I switched to WireGuard instead of OpenVPN. More specifically, as I am running the nordvpn solution, I installed WireGuard and used their version of it, NordLynx.

2
  • 1
    The specific command to switch from OpenVPN to the WireGuard protocol for NordVPN is nordvpn set technology NordLynx. It is not a separate product, and currently is only available on Linux & iOS.
    – tephyr
    Commented Feb 16, 2020 at 1:14
  • Thank you for this! It worked perfectly and no additional installs were required for me, just running nordvpn set technology NordLynx did the trick. Commented Mar 30, 2021 at 14:59
5

If you have a lot of networks in docker network ls you need to run docker system prune -f

This line removes unused data, unused images and all unused local volumes. I have it in my crontab:

# cleanup docker
0 3 * * * /usr/bin/docker system prune -f;/usr/bin/docker image prune -a --filter "until=24h" -f; /usr/bin/docker volume prune -f
2
  • 2
    I'd be careful with this. This is a destructive crontab to have. Commented May 5, 2022 at 7:46
  • 1
    Thanks @AdamKDean, I added a disclaimer.
    – karser
    Commented May 6, 2022 at 8:19
4

I ran into this issue on a corporate development VM which wasn't running OpenVPN. Checking out etc/docker/daemon.json, I found

...
"default-address-pools": [
  {
    "base": "192.168.11.0/24",
    "size": 24
  }
],
...

Strangely, removing the default-address-pools field and then restarting docker with sudo systemctl restart docker fixed the issue for me. I'm assuming this let docker choose a more suitable default, but I don't know what the problem was with the chosen default.

3

I ran in this problem because of forcepoint vpn addresses.

1 - check your in use addresses using nmcli command

2 - choose a non overlapping CIDR address (x.x.x.x/xx) #google for details

3 - docker container prune #to destroy all created container

4 - docker network prune #to destroy all created networks

5 - modify (or create if not present) /etc/docker/daemon.json and add the follwing entry (changing eventually the address with your choosed one):

{
    "default-address-pools":[
        {
            "base":"173.5.0.0/16",
            "size":24
        }
    ]
}


N.B. be careful using 173.x.x.x because it is'n a private address and may cause problem if you need to go to an external address pointing to the same ip but it is one solution when your vpn already take control of all other internal ips

6 - sudo systemctl restart docker

6.error - If the service doesn't start it may be caused by another overlapping network.

Use journalctl -xe to see the error.

You can check again your networks using nmcli and retry.

N.B. After too many retry you may need to reset error count sudo systemctl reset-failed servicename.service

1
  • same thing for me, vpn blocks docker compose start, this the only solution that works :3
    – Kokizzu
    Commented Dec 28, 2022 at 10:24
1

I found this problem with one of enterprise network I was working with. The specific requirement was not be use the default from docker 172.17.0.0/16 and 172.18.0.0/16.

Private IP4 network, such as 10.28.160.0/24 and 10.28.161.0/24 was supposed to be used which I configured in docker daemon and it was resulting in non-overlapping issue. Changing subnet mask to /16 such as 10.28.160.0/16 solved the issue. It clearly provided more number of hosts and thus polling easily achieved.

1
1

Running the docker network prune command did't help me.
I also use a VPN and can’t disable it because images for docker-compose are downloaded only through a VPN.

I added a non-overlapping address space to the file /etc/docker/daemon.json:

# /etc/docker/daemon.json
{
  "default-address-pools":[
    {
      "base":"173.5.0.0/16",
      "size":24
    }
  ],
  "experimental": true
}

I try to restart docker with command sudo systemctl restart docker.socker, but still not working.

After that I finded all application containers:

docker container ls -a | grep my-app-name
=>
0d55efcf2c69   ... my-app-name_postgres
9b73192fb131   ... my-app-name_redis

And delete all:

docker container rm 0d55efcf2c69
docker container rm 9b73192fb131

After restarting docker.socket, docker-compose started successfully.

0

In my case, I had many docker networks in my system, so I listed all the docker networks by running the command docker network ls.

Then i removed one of the unwanted network by running the command docker network rm {network_id}.

Later i run docker-compose up command and it worked.

Not the answer you're looking for? Browse other questions tagged or ask your own question.