81

My team uses Docker (with ubuntu:14.04 base image) for local development and we often have to rebuild some or all of our images. But we often get failures downloading packages with apt-get install, even immediately after running apt-get -y update. For instance, today I see

Err http://archive.ubuntu.com/ubuntu/ trusty-security/main libxml2 amd64 2.9.1+dfsg1-3ubuntu4.7
  404  Not Found [IP: 91.189.88.161 80]
Err http://archive.ubuntu.com/ubuntu/ trusty-security/main libxml2-dev amd64 2.9.1+dfsg1-3ubuntu4.7
  404  Not Found [IP: 91.189.88.161 80]
Fetched 84.7 MB in 1min 6s (1281 kB/s)
Unable to correct missing packages.
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libx/libxml2/libxml2_2.9.1+dfsg1-3ubuntu4.7_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libx/libxml2/libxml2-dev_2.9.1+dfsg1-3ubuntu4.7_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

E: Aborting install.

Apparently the specific version of a particular package has been deleted from the archive and replaced with a slightly differently named patch version. For instance, the above error is looking for libxml2_2.9.1+dfsg1-3ubuntu4.7_amd64.deb but the version on the server is libxml2_2.9.1+dfsg1-3ubuntu4.8_amd64.deb.

Often this is solvable by removing the base image (docker rmi ubuntu:14.04) and rebuilding; the newly downloaded ubuntu image has the correct patch number and finds the right archive file. But even this doesn't always work -- probably due to a delay between a new minor upgrade to Ubuntu's dependency db and the deployment of that new ubuntu:14.04 image onto Docker Hub.

We've tried using apt-get flags --fix-missing and --fix-broken and those don't consistently work either.

Any other ideas?

apt-get install fails with Not Found error because package removed from repository is a similar problem but the accepted answer is unacceptable because it's not possible to be automated. Our daily development process, including automatic build and deploy, is all scripted and using Docker and it's not practical to hack around inside a Dockerfile every time a particular archive goes missing (then remove the hack after a few hours or days).


In response to @prateek05, here's the /etc/apt/sources.list from the official ubuntu:14.04 docker image:

root@72daa1942714:/# cat /etc/apt/sources.list
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted

## Uncomment the following two lines to add software from the 'universe'
## repository.
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ trusty universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty universe
deb http://archive.ubuntu.com/ubuntu/ trusty-updates universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates universe

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted

deb http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-security universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security universe
# deb http://archive.ubuntu.com/ubuntu/ trusty-security multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-security multiverse
6
  • 2
    Do you always do apt-get update before any apt-get install? Jun 8, 2016 at 16:24
  • yes, our base dockerfile starts with RUN apt-get -y update Jun 8, 2016 at 17:29
  • Is this normally in a docker build or on the command line? What do you use to bust the docker build cache when rebuilding?
    – Matt
    Jun 8, 2016 at 17:47
  • When this happens I use docker rmi ubuntu:14.04 and then docker build downloads a fresh image and builds without cache from then on. Jun 8, 2016 at 17:50
  • 1
    Could you post a sample Dockerfile? The ordering could contribute
    – Matt
    Jun 8, 2016 at 18:01

7 Answers 7

202

You have stated that your Dockerfile contains RUN apt-get -y update as its own RUN instruction. However, due to build caching, if all changes to the Dockerfile occur later in the file, when docker build is run, Docker will reuse the intermediate image created the last time RUN apt-get -y update executed instead of running the command again, and so any recently-added or -edited apt-get install lines will be using old data, leading to the errors you've observed.

There are two ways to fix this:

  1. Pass the --no-cache option to docker build, forcing every statement in the Dockerfile to be run every time the image is built.

  2. Rewrite the Dockerfile to combine the apt-get commands in a single RUN instruction: RUN apt-get update && apt-get install foo bar .... This way, whenever the list of packages to install is edited, docker build will be forced to re-execute the entire RUN instruction and thus rerun apt-get update before installing.

The Dockerfile best practices page actually has an entire section on apt-get commands in Dockerfiles. I suggest you read it.

4
  • 2
    NOTE - you only need to use the flag --no-cache ONCE to force a purge then that flag can be removed for subsequent docker build for same image ... handy/faster when repeating docker build for other reasons Nov 7, 2016 at 17:33
  • 3
    Interestingly --no-cache worked but I already had RUN apt-get update && apt-get install ... and that hadn't been enough. Nov 18, 2016 at 13:50
  • I have pass --no-cache parameter with any frequency? Aug 8, 2019 at 11:55
  • docker build -t docker/test:latest . --no-cache This is what to use. Place this flag at end of your build command. Apr 14, 2021 at 16:52
17

The issue could be potentially with the ubuntu sources

Check /etc/apt/sources.list

If you see deb http://archive.ubuntu.com/ubuntu main universe restricted multiverse , that could be the potential issue.

Fix that by replacing it with deb http://archive.ubuntu.com/ubuntu/ trusty main universe restricted multiverse

Alternatively it could be the mirror itself doesn't respond. archive.ubuntu.com

Name:   archive.ubuntu.com
Address: 91.189.88.152
Name:   archive.ubuntu.com
Address: 91.189.88.161
Name:   archive.ubuntu.com
Address: 91.189.88.149

Replace archive.ubuntu.com with a more trusted mirror say us.archive.ubuntu.com

Name:   us.archive.ubuntu.com
Address: 91.189.91.23
Name:   us.archive.ubuntu.com
Address: 91.189.91.26

(edit by the oiriginal asker):

Thanks, prateek05! My Dockerfile now starts:

FROM ubuntu:14.04
RUN sed -i'' 's/archive\.ubuntu\.com/us\.archive\.ubuntu\.com/' /etc/apt/sources.list
RUN apt-get -y update

and it seems to be working. But since this is a sporadic issue, only time will tell...

3
  • in sources.list i see lots of settings I don't undersand :-) -- trusty main restricted, trusty universe, trusty-security main restricted, trusty-security universe, but no multiverse (except a commented-out one)... what should I change, exactly? Jun 8, 2016 at 17:40
  • the contents look fine. It may be due to the mirror resolution. You get a mirror which isn't responding. To resolve that specify a mirror directly e.g. change everyhwere from archive.ubuntu.com to us.archive.ubuntu.com
    – prateek05
    Jun 8, 2016 at 17:41
  • this line RUN sed -i'' 's/archive\.ubuntu\.com/us\.archive\.ubuntu\.com/' /etc/apt/sources.list is all that was needed to fix it.. thanks!
    – supersan
    Aug 13, 2021 at 9:31
5

In my case the problem was caused by the parent image since it had not cleared the apt cache properly.

I solve the problem including cleaning commands before the first apt update ...

RUN apt clean && \
    rm -rf /var/lib/apt/lists/* && \
    apt update && \
    ...

Hope this helps

0
1

Found something that works!

So I found this: https://www.mail-archive.com/[email protected]/msg5682159.html

The solution is to create a pinning_file with the contents

# Written by ubuntu-advantage-tools
Package: *
Pin: release o=UbuntuESM, n=trusty
Pin-Priority: never

Then add

COPY pinning_file /etc/apt/preferences.d/ubuntu-esm-infra-trusty

To you Dockerfile before you run the sudo apt-get -y update

0

I was able to fix this error only after adding some extra arguments to apt-get to deal with http issues:

sudo apt-get \
  -o Acquire::BrokenProxy="true" \
  -o Acquire::http::No-Cache="true" \ 
  -o Acquire::http::Pipeline-Depth="0" install \
    ignition
0

What worked for me ...

I had the statements in two lines, but when i merged to one it worked (don't know if it was cache related ...)

it was

RUN apt-get -y update && apt-get upgrade -y

# Install tools && libraries
RUN apt-get -y install --fix-missing apt-utils iputils-ping nano wget dialog \
    build-essential git zip  \

And then i changed to

RUN apt-get -y update && apt-get upgrade -y \
    && apt-get -y install --fix-missing apt-utils nano wget \
    git zip  \
    mysql-client \
-3

Using FTP sources works 100% of the time.

RUN echo \
   'deb ftp://ftp.us.debian.org/debian/ jessie main\n \
    deb ftp://ftp.us.debian.org/debian/ jessie-updates main\n \
    deb http://security.debian.org jessie/updates main\n' \
    > /etc/apt/sources.list

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.