263

I had it working allright but now it stopped. I tried the following commands with no avail:

docker run -dns 8.8.8.8 base ping google.com

docker run base ping google.com

sysctl -w net.ipv4.ip_forward=1 - both on the host and on the container

All I get is unknown host google.com. Docker version 0.7.0

Any ideas?

P.S. ufw disabled as well

9
  • 26
    Your question fixed my problem: had to run sysctl -w net.ipv4.ip_forward=1 (on Centos 6)
    – qwertzguy
    Oct 5, 2015 at 14:01
  • Since you may have the problem with docker dns routing, check this similar solution stackoverflow.com/questions/35515203/… Aug 10, 2018 at 8:41
  • Same here, after I fixed the /etc/resolv.conf on the host box it didn't won't to work without sysctl -w net.ipv4.ip_forward=1
    – Reeebuuk
    Feb 9, 2019 at 13:13
  • Also check that you have the correct values for /etc/resolv.conf on the host machine
    – Hanxue
    Aug 27, 2019 at 10:09
  • 7
    for me after sysctl -w net.ipv4.ip_forward=1 I had to run sudo service docker restart.
    – Asif Ali
    Feb 24, 2020 at 9:43

37 Answers 37

216

First thing to check is run cat /etc/resolv.conf in the docker container. If it has an invalid DNS server, such as nameserver 127.0.x.x, then the container will not be able to resolve the domain names into ip addresses, so ping google.com will fail.

Second thing to check is run cat /etc/resolv.conf on the host machine. Docker basically copies the host's /etc/resolv.conf to the container everytime a container is started. So if the host's /etc/resolv.conf is wrong, then so will the docker container.

If you have found that the host's /etc/resolv.conf is wrong, then you have 2 options:

  1. Hardcode the DNS server in daemon.json. This is easy, but not ideal if you expect the DNS server to change.

  2. Fix the hosts's /etc/resolv.conf. This is a little trickier, but it is generated dynamically, and you are not hardcoding the DNS server.


1. Hardcode DNS server in docker daemon.json

  • Edit /etc/docker/daemon.json

    {
        "dns": ["10.1.2.3", "8.8.8.8"]
    }
    
  • Restart the docker daemon for those changes to take effect:
    sudo systemctl restart docker

  • Now when you run/start a container, docker will populate /etc/resolv.conf with the values from daemon.json.


2. Fix the hosts's /etc/resolv.conf

A. Ubuntu 16.04 and earlier

  • For Ubuntu 16.04 and earlier, /etc/resolv.conf was dynamically generated by NetworkManager.

  • Comment out the line dns=dnsmasq (with a #) in /etc/NetworkManager/NetworkManager.conf

  • Restart the NetworkManager to regenerate /etc/resolv.conf :
    sudo systemctl restart network-manager

  • Verify on the host: cat /etc/resolv.conf

B. Ubuntu 18.04 and later

  • Ubuntu 18.04 changed to use systemd-resolved to generate /etc/resolv.conf. Now by default it uses a local DNS cache 127.0.0.53. That will not work inside a container, so Docker will default to Google's 8.8.8.8 DNS server, which may break for people behind a firewall.

  • /etc/resolv.conf is actually a symlink (ls -l /etc/resolv.conf) which points to /run/systemd/resolve/stub-resolv.conf (127.0.0.53) by default in Ubuntu 18.04.

  • Just change the symlink to point to /run/systemd/resolve/resolv.conf, which lists the real DNS servers:
    sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf

  • Verify on the host: cat /etc/resolv.conf

Now you should have a valid /etc/resolv.conf on the host for docker to copy into the containers.

18
  • 2
    This solved the issue on Ubuntu 16.04 with Docker 17.09. Dec 4, 2017 at 16:12
  • 2
    This resolved my issue (same as OP, Ubuntu 14.04 / Docker 18.01.0-ce). This link can be usefull test internet connection without ping if you don't have ping command on your docker image. If your host don't have systemctl (Ubuntu 14.04) try How to restart the networking service? and/or restart you computer.
    – Benjamin
    Jan 31, 2018 at 11:15
  • 5
    This works on Ubuntu 18.04 (option B). However docker didn't transfer the now correct /etc/resolv.conf to the container on construction, I had to manually copy the file into the container.
    – glaux
    Aug 22, 2018 at 11:00
  • 2
    On my machine (RedHat 7.4) the host's config file is correct, but the containers file is still pointing towards 172.0.0.11. So what to do now? Sep 4, 2018 at 11:22
  • 4
    Are there any changes in Ubuntu 20? could you update, please?
    – mahyard
    Apr 5, 2021 at 18:50
167

Fixed by following this advice:

[...] can you try to reset everything?

pkill docker
iptables -t nat -F
ifconfig docker0 down
brctl delbr docker0
docker -d

It will force docker to recreate the bridge and reinit all the network rules

https://github.com/dotcloud/docker/issues/866#issuecomment-19218300

Seems the interface was 'hung' somehow.

Update for more recent versions of docker:

The above answer might still get the job done for you but it has been quite a long time since this answer was posted and docker is more polished now so make sure you try these first before going into mangling with iptables and all.

sudo service docker restart or (if you are in a linux distro that does not use upstart) sudo systemctl restart docker

11
  • 52
    docker -d fails. There is no -d flag. Dec 4, 2017 at 16:09
  • 1
    For those still having the issue, there's an open issue on Moby's github that has been opened for over a year now: github.com/moby/moby/issues/26567
    – Nepoxx
    Jan 30, 2018 at 18:42
  • 4
    @Pawan: ip link del docker0 Aug 1, 2018 at 6:15
  • 1
    or install bridge-utils Nov 6, 2018 at 10:53
  • 18
    docker -d doesn't exist in newer versions. Instead: service docker stop, then dockerd, then service docker start Apr 5, 2019 at 16:07
88

The intended way to restart docker is not to do it manually but use the service or systemctl command:

service docker restart

or

systemctl restart docker
7
  • 8
    if you are in a linux distro that does not use upstart, sudo systemctl restart docker worked for me
    – jeffrey
    Mar 16, 2016 at 17:31
  • 1
    restarting worked fine. I don't know if it has to do with the fact that I enabled it to "auto start" (systemctl enable docker) Apr 6, 2016 at 17:53
  • 2
    It kinda does, because in the situation OP is describing, resetting docker reinitialises the network interfaces, hence reenabling internet access. It is true that this does not address WHY it sometimes breaks, but it provides a solution to the problem.
    – bitmask
    Oct 2, 2018 at 19:59
  • But in production environment, restart docker is impossible. How to solve the problem on this case?
    – Suyanhanx
    Dec 23, 2018 at 7:20
  • 1
    Thank god I tried this before messing w/ DNS and mangling IP tables. Works in Aug 2023 Aug 15 at 14:50
27

Tried all answers, none worked for me.

After trying everything else I could find, this did the trick:

reboot
6
  • 9
    This is a ridiculous answer to upvote but I just did ... I spent hours trying to figure out why bizarre things were happening only to discover that my containers using a "bridge" network couldn't access any internet (not just DNS). After many answers above this one I rebooted and the problem is gone. Sigh. Apr 5, 2022 at 20:35
  • 8
    I know, it felt even more ridiculous to post it. It's called the Windows technique.
    – Mauro
    Apr 6, 2022 at 12:26
  • 1
    I spent >4 hours tonight trying this and that, having doubts that the changes I did to some docker compose scripts were wrong and this morning I thought what the hell, and rebooted. Now it works again. Tnx
    – Dietrich
    Sep 18, 2022 at 10:08
  • I had no internet in my container created by docker-compose. Reboot of server solved it. I still don't know the root cause, but it was something that affected the networking rules in between, because I had older containers started with similar image/compose configuration and they still worked.
    – CRK
    Dec 17, 2022 at 11:13
  • 1
    I tried everything else first because I didn't want to resort to the Windows Technique... I just wasted time. don't make my mistake, reboot first, then think :D
    – caesarsol
    May 26 at 21:55
22

Updating this question with an answer for OSX (using Docker Machine)

If you are running Docker on OSX using Docker Machine, then the following worked for me:

docker-machine restart

<...wait for it to restart, which takes up to a minute...>

docker-machine env
eval $(docker-machine env)

Then (at least in my experience), if you ping google.com from a container all will be well.

2
  • Also worked in windows to get network access working again. May 28, 2018 at 10:33
  • 1
    That worked for me. I have a docker-icon in the top menu bar, in the menu I had a "restart" option. After that, networking was fine again
    – olidem
    Oct 18, 2018 at 10:44
16

I do not know what I am doing but that worked for me :

OTHER_BRIDGE=br-xxxxx # this is the other random docker bridge (`ip addr` to find)    
service docker stop

ip link set dev $OTHER_BRIDGE down
ip link set dev docker0 down
ip link delete $OTHER_BRIDGE type bridge
ip link delete docker0 type bridge
service docker start && service docker stop

iptables -t nat -A POSTROUTING ! -o docker0 -s 172.17.0.0/16 -j MASQUERADE
iptables -t nat -A POSTROUTING ! -o docker0 -s 172.18.0.0/16 -j MASQUERADE

service docker start
4
  • 2
    nice duct tape !
    – user1270589
    Mar 25, 2019 at 23:56
  • 2
    You answer helped to solve similar issue. I spent hours on that! After incomplete Kubespray installation, Docker containers lost internet with the message "Temporary failure resolving" when trying to ping any public host or IP. So I didn't have this rule which is mandatory - iptables -t nat -A POSTROUTING ! -o docker0 -s 172.17.0.0/16 -j MASQUERADE . You can check if you have this rule with iptables -t nat -L POSTROUTING
    – laimison
    Dec 16, 2019 at 14:15
  • whatever this problem is I hate it. I hate it so much lol. Docker never connects to the internet on startup but if I run these things it magically works. Until the next reboot... Jun 24, 2021 at 14:21
  • Auto grab the first bridge id with OTHER_BRIDGE=$(ip addr | grep -E -m 1 -o 'br-[0-9a-f]+') Sep 28, 2021 at 15:13
8

I was using DOCKER_OPTS="--dns 8.8.8.8" and later discovered and that my container didn't have direct access to internet but could access my corporate intranet. I changed DOCKER_OPTS to the following:

DOCKER_OPTS="--dns <internal_corporate_dns_address"

replacing internal_corporate_dns_address with the IP address or FQDN of our DNS and restarted docker using

sudo service docker restart

and then spawned my container and checked that it had access to internet.

8

No internet access can also be caused by missing proxy settings. In that case, --network host may not work either. The proxy can be configured by setting the environment variables http_proxy and https_proxy:

docker run -e "http_proxy=YOUR-PROXY" \
           -e "https_proxy=YOUR-PROXY"\
           -e "no_proxy=localhost,127.0.0.1" ... 

Do not forget to set no_proxy as well, or all requests (including those to localhost) will go through the proxy.

More information: Proxy Settings in the Archlinux Wiki.

5
  • 1
    This was the solution for me. Beware though: I was using alpine, which has a busybox implementation of wget that appears to ignore the proxy settings, so I wasn't seeing the benefit of having the environment variables set.
    – pelson
    Dec 4, 2017 at 17:01
  • Thanks for the hint about busybox; I did not know about it yet! Dec 6, 2017 at 9:54
  • 1
    be aware that some os need upper case like in the documentation link.
    – Flo
    Feb 14, 2020 at 14:33
  • This syntax gave me docker: invalid reference format. I think the correct one was in the link mentioned by @Flo : -e HTTP_PROXY="proxy.."... Also, the `\` to distribute into multiple lines gives a similar error
    – Bersan
    May 13, 2021 at 14:27
  • @Bersan This is probably a shell specific issue, the above command works in Bash, which shell are you in? Jan 7 at 7:03
6

I was stumped when this happened randomly for me for one of my containers, while the other containers were fine. The container was attached to at least one non-internal network, so there was nothing wrong with the Compose definition. Restarting the VM / docker daemon did not help. It was also not a DNS issue because the container could not even ping an external IP. What solved it for me was to recreate the docker network(s). In my case, docker-compose down && docker-compose up worked.

Compose

This forces the recreation of all networks of all the containers:

docker-compose down && docker-compose up

Swarm mode

I suppose you just remove and recreate the service, which recreates the service's network(s):

docker service rm some-service

docker service create ...

If the container's network(s) are external

Simply remove and recreate the external networks of that service:

docker network rm some-external-network

docker network create some-external-network

0
6

Other answers have stated that the docker0 interface (bridge) can be the source of the problem. On Ubuntu 20.04 I observed that the interface was missing its IP address (to be checked with ip addr show dev docker0). Restarting Docker alone did not help. I had to delete the bridge interface manually.

sudo ip link delete docker0
sudo systemctl restart docker
1
  • In my case (Synology NAS DS220+), not even this helped. I had to restart the NAS to fix it.
    – meridius
    Dec 13, 2022 at 16:02
5

For me it was the host's firewall. I had to allow DNS on the host's firewall. And also had to restart docker after changing the host firewall setting.

1
  • Or you could disable the iptables by sudo service iptables stop and sudo chkconfig iptables off (on CentOS/RHEL).
    – MichaelZ
    Aug 17, 2015 at 2:19
5

for me, my problem was because of iptables-services was not installed, this worked for me (CentOS):

sudo yum install iptables-services
sudo service docker restart
2
  • remember start and enable iptable services too
    – Jay
    Jul 12, 2020 at 18:51
  • also works on red hat 7.3 Sep 20, 2021 at 15:05
5

Sharing a simple and working solution for posterity. When we run a docker container without explicitly mentioning the --network flag, it connects to its default bridge network which prohibits connecting to the outside world. To resolve this issue, we have to create our own bridge network(user-defined bridge) and have to explicitly mention it with the docker run command.

docker network create --driver bridge mynetwork
docker run -it --network mynetwork image:version
2
5

it help me:

sudo ip link delete docker0
sudo systemctl stop docker.socket
sudo systemctl stop docker.service

sudo systemctl start docker.socket
sudo systemctl start docker.service

NOTE: after this, interface docker0 must have ip adress smth like that:

inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
4

You may have started your docker with dns options --dns 172.x.x.x

I had the same error and removed the options from /etc/default/docker

The lines:

# Use DOCKER_OPTS to modify the daemon startup options.
DOCKER_OPTS="--dns 172.x.x.x"
4

On centos 8, My problem was that I did not install & start iptables before starting docker service. Make sure iptables service is up and running before you start docker service.

1
  • I had the same issue on centos8 Sep 7, 2020 at 17:04
3

If you're on OSX, you might need to restart your machine after installing Docker. This has been an issue at times.

3

For me it was an iptables forwarding rule. For some reason the following rule, when coupled with docker's iptables rules, caused all outbound traffic from containers to hit localhost:8080:

iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8080
iptables -t nat -I OUTPUT -p tcp -d 127.0.0.1 --dport 80 -j REDIRECT --to-ports 8080
2
  • 3
    So...what's the solution? :) I have the first rule and need it to redirect inbound traffic on 80 to 8080. How do I change this to not affect outbound traffic?
    – mrooney
    Mar 26, 2017 at 22:14
  • Sorry, my memory of iptables is faded, but in my setup script I see iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080 as the very last line, after the defaults (iptables -P INPUT DROP etc). No OUTPUT...REDIRECT line.
    – brandones
    Aug 5, 2020 at 0:57
3

I had the problem on Ubuntu 18.04. However the problem was with the DNS. I was in a corporate network that has its own DNS server and block other DNS servers. This is to block some websites (porn, torrents, ... so on )

To resolve your problem

  1. find your DNS on host machine
  2. use --dns your_dns as suggested by @jobin

    docker run --dns your_dns -it --name cowsay --hostname cowsay debian bash

3

For Ubuntu 19.04 using openconnect 8.3 for VPN, I had to symlink /etc/resolve.conf to the one in systemd (opposite of answerby wisbucky )

sudo ln -sf /etc/resolv.conf /run/systemd/resolve/resolv.conf

Steps to debug

  1. Connect to Company VPN
  2. Look for correct VPN settings in either /etc/resolv.conf or /run/systemd/resolve/resolv.conf
  3. Whichever has the correct DNS settings, we'll symlink that to the other file ( Hint: Place one with correct settings on the left of assignment )

Docker version: Docker version 19.03.0-rc2, build f97efcc

1
  • 2
    Thank you. With Ubuntu 18.04, upon connecting to company VPN only the /etc/resolve.conf was getting updated by the DHCP and the /run/systemd/resolve/resolve/conf remained constant/static. This solution helped. Now, containers in local machine connect to servers in VPN (which was not happening earler for me)
    – nashter
    Mar 25, 2020 at 20:33
3

for me, using centos 7.4, it was not issue of /etc/resolve.conf, iptables, iptables nat rules nor docker itself. The issue is the host missing the package bridge-utils which docker require to build the bridge using command brctl. yum install -y bridge-utils and restart docker, solve the problem.

1
  • 1
    which is why I upvoted it. But my remark was meant as a gentle reminder that cause and defect are sometimes a bit apart and that filing a ticket could help solve it for others, even though the first instinct would be to work around the underlying issue. Dec 15, 2020 at 15:22
2

On windows (8.1) I killed the virtualbox interface (via taskmgr) and it solved the issue.

2

Originally my docker container was able to reach the external internet (This is a docker service/container running on an Amazon EC2).

Since my app is an API, I followed up the creation of my container (it succeeded in pulling all the packages it needed) with updating my IP Tables to route all traffic from port 80 to the port that my API (running on docker) was listening on.

Then, later when I tried rebuilding the container it failed. After much struggle, I discovered that my previous step (setting the IPTable port forwarding rule) messed up the docker's external networking capability.

Solution: Stop your IPTable service:

sudo service iptables stop

Restart The Docker Daemon:

sudo service docker restart

Then, try rebuilding your container. Hope this helps.


Follow Up

I completely overlooked that I did not need to mess with the IP Tables to forward incoming traffic to 80 to the port that the API running on docker was running on. Instead, I just aliased port 80 to the port the API in docker was running on:

docker run -d -p 80:<api_port> <image>:<tag> <command to start api>

1
  • 1
    In my case, I just have to restart docker Daemon, and it works
    – okante
    Aug 18, 2020 at 0:38
2

I've tried most answers in here, but the only thing that worked was re-creating the network:

$ docker network rm the-network
$ docker network create --driver=bridge the-network

I also needed to re-create the docker container that used it:

$ sudo docker create --name the-name --network the-network

Then it started with internet access.

1
2

I am on Arch Linux and after trying all the above answers I realized that I had a firewall enabled in my machine, nftables, and disabling it did the trick. I did :

sudo systemctl disable nftables
sudo systemctl stop nftables
sudo reboot

My network cards:

➜  ~ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp1s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
    link/ether 68:f7:28:84:e7:fe brd ff:ff:ff:ff:ff:ff
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DORMANT group default qlen 1000
    link/ether d0:7e:35:d2:42:6d brd ff:ff:ff:ff:ff:ff
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default 
    link/ether 02:42:43:3f:ff:94 brd ff:ff:ff:ff:ff:ff
5: br-c51881f83e32: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default 
    link/ether 02:42:ae:34:49:c3 brd ff:ff:ff:ff:ff:ff
6: br-c5b2a1d25a86: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default 
    link/ether 02:42:72:d3:6f:4d brd ff:ff:ff:ff:ff:ff
8: veth56f42a2@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default 
    link/ether 8e:70:36:10:4e:83 brd ff:ff:ff:ff:ff:ff link-netnsid 0

and my firewal configuration, /etc/nftables.conf, which I now disabled and will futurely try to improve so I can have the docker0 network card rules setup correctly:

#!/usr/bin/nft -f
# vim:set ts=2 sw=2 et:

# IPv4/IPv6 Simple & Safe firewall ruleset.
# More examples in /usr/share/nftables/ and /usr/share/doc/nftables/examples/.

table inet filter
delete table inet filter
table inet filter {
  chain input {
    type filter hook input priority filter
    policy drop

    ct state invalid drop comment "early drop of invalid connections"
    ct state {established, related} accept comment "allow tracked connections"
    iifname lo accept comment "allow from loopback"
    ip protocol icmp accept comment "allow icmp"
    meta l4proto ipv6-icmp accept comment "allow icmp v6"
    #tcp dport ssh accept comment "allow sshd"
    pkttype host limit rate 5/second counter reject with icmpx type admin-prohibited
    counter
  }
  chain forward {
    type filter hook forward priority filter
    policy drop
  }
1
  • Thanks for posting this. I'm also on arch and using nftables, and this was all I needed to do. I had already stopped and disabled nftables, but the changes didn't affect docker until I rebooted in full.
    – covert
    Jun 29 at 23:12
1

Just adding this here in case someone runs into this issue within a virtualbox container running docker. I reconfigured the virtualbox network to bridged instead of nat, and the problem went away.

1

There are lot of good answer already. I faced similar problem in my orange pi pc running armbian recently. Docker container was blocked to internet. This command solve the problem in my case. So I like to share it

docker run --security-opt seccomp=unconfined imageName
1

The only thing that works on my computer (which 8.8.8.8 didn't work)

  1. find out the dns used in your local machine:
$ netstat -ntpl | grep :53
tcp        0      0 10.194.128.1:53         0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      -
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.2.1:53            0.0.0.0:*               LISTEN      -
  1. modify your docker config sudo vim /etc/docker/daemon.json based on the information above on your own computer
{
  "dns": ["192.168.122.1","10.194.128.1"]
}
  1. restart docker
sudo ip link delete docker0
sudo systemctl restart docker
1
  • XD this one the only answer works for me, apparently I was the one that answer it in the past XD
    – Kokizzu
    Apr 14 at 18:40
1

If you're running Docker rootless and facing this issue, during its installation, iptables may not be configured properly, mainly because of using --skip-iptables option when Docker complained about iptables:

[ERROR] Missing system requirements. Run the following commands to
[ERROR] install the requirements and run this tool again.
[ERROR] Alternatively iptables checks can be disabled with --skip-iptables .

########## BEGIN ##########
sudo sh -eux <<EOF
# Load ip_tables module
modprobe ip_tables
EOF
########## END ##########

Let's check if this is the issue: is ip_tables kernel module loaded?

sudo modprobe ip_tables

If there is no output, this answer may not help you (you could try anyway). Otherwise, the output's something like this:

modprobe: FATAL: Module ip_tables not found in directory /lib/modules/5.18.9-200.fc36.x86_64

Let's fix it!

First, uninstall Docker rootless (no need to stop the service via systemctl, the script handles that):

dockerd-rootless-setuptool.sh uninstall --skip-iptables

Ensure iptables package is installed, although it's shipped by default by major distributions.

Now, make ip_tables module visible to modprobe and install it (thanks to this):

sudo depmod
sudo modprobe ip_tables

Now, re-install Docker rootless:

dockerd-rootless-setuptool.sh install

If it doesn't bother about iptables, you're done and problem should be fixed. Don't forget to enable the service (i.e. systemctl enable --user --now docker)!

1

For me iwas facing the same issue as user of redhat/centos/fedora using podman

firewall-cmd --zone=public --add-masquerade
firewall-cmd --permanent --zone=public --add-masquerade

for more firewalld and podman (or docker) – no internet in the container and could not resolve host

Not the answer you're looking for? Browse other questions tagged or ask your own question.