Is it possible to cache docker images on Travis CI? Attempting to cache the /var/lib/docker/aufs/diff
folder and /var/lib/docker/repositories-aufs
file with cache.directories in the travis.yml doesn't seem to work since they require root.
-
2This is a duplicate, see stackoverflow.com/questions/32866599/… – Jakub Holý Feb 1 '17 at 9:16
-
1Possible duplicate of Can Travis CI cache docker images? – Deniz Apr 1 '19 at 11:19
From a Docker perspective, I think the best way you could do this (without the possibility of running a network local registry) is save
the Docker image and cache the exported tar ball. You would need to load
that at the start rather than pull
an image. This way you're not messing with docker storage implementations.
install:
- docker pull busybox
- docker save busybox | gzip > docker/busybox.tar.gz
cache:
directories:
- docker
You would then need to load
the cached image before your Travis run.
before_script:
- gzip -dc docker/busybox.tar.gz | docker load
The bit I'm not clear on for Travis, is if you need to stop it from running the install
step after the first time. You don't want Travis pulling and exporting the image each time once it's cached. I'm not sure if having the cache
directive automatically does that for you?
The main question then is whether this is actually going to be any quicker than pulling the image or not:
The caching tars up all the directories listed in the configuration and uploads them to S3, using a secure and protected URL, ensuring security and privacy of the uploaded archives.
Note that this makes our cache not network-local, it’s still bound to network bandwidth and DNS resolutions for S3. That impacts what you can and should store in the cache. If you store archives larger than a few hundred megabytes in the cache, it’s unlikely that you’ll see a big speed improvement.
You might just be adding overhead. As the Docker registry is backed by Cloudfront, Travis is already pulling compressed images from local, or at least close Amazon infrastructure. Maybe ask them for the feature to cache Docker images natively, similar to what they do for apt
packages, although it doesn't sound hopeful.
-
I'm not sure if this is exactly what I want, as it exports containers, and I'm looking just for images. Also wouldn't update as you might have noted. That is a good snippet though, maybe there really is no point in trying to cache docker images. – kag0 Feb 13 '16 at 0:57
-
You would need to manually bust the cache via the gui when you want to update. The image names would need to be different too, which is a bit of a pain. Also a container filesystem is just an image with changes overlayed on top. If you don't make any changes you still have the image. In fact a
docker build
creates image by committing change's made in a running container. With usingexport
you do lose track of image layers though. – Matt Feb 13 '16 at 2:15 -
rundef.com/fast-travis-ci-docker-build details a way of conditionally caching artefacts into pre-defined
$CACHE_DIR
based on the existence of the latter. If it does not exist, assume that there is no cache and populate it, otherwise, use the archives from theCACHE_DIR
. – Ashutosh Jindal May 8 '19 at 9:37
Have a look at what circleci recommends: https://circleci.com/docs/docker/#caching-docker-layers.
It should be easy to combine docker save
/docker load
with the directory caching provided by travis.
-
CircleCI 2.0 automatically caches docker images. They cache it on each build host, so at the beginning, one might think, there is no caching. The more builds, the more often the cache will be hit. – koppor Aug 16 '17 at 20:42
-
3
-
It appears that docker layer caching on CircleCI is a premium feature, however there are DIY workarounds as well as seen here – Ashutosh Jindal May 8 '19 at 9:41
-
Additionally, it seems docker layer caching has been a premium feature on CircleCI at least since September 2017 – Ashutosh Jindal May 8 '19 at 10:00