I have a private repo that contains packages I want to pip install. I've spent quite a bit of time reading over various forums and articles about different ways to securely do this. There doesn't seem to be a a consensus on how to best do this (if at all possible). I obviously don't want to expose any ssh keys/secrets in my dockerfile -- I want to be careful about making them available via docker history.
6 Answers
As explained in "Securely build small python docker image from private git repos", you would need to use, with Docker 18.09+
--ssh
You can use the--ssh
flag to forward your existing SSH agent key to the builder. Instead of transferring the key data, docker will just notify the builder that such capability is available.
Now when the builder needs access to a remote server through SSH, it will dial back to the client and ask it to sign the specific request needed for this connection.
The key itself never leaves the client, and as soon as the command that requested the access has completed there is no information on the builder side to reestablish that remote connection later.Secrets:
Provides a mount option during the build at/var/run/secrets
available only for the command that used it and is not included in the created layer.
That is:
docker build --ssh github_ssh_key=/path/to/.ssh/git_ssh_id_rsa .
only the agent connection is shared with that command, and not the actual content of the private key.
no other commands/steps in the Dockerfile will have access to it.
The Dockerfile, in a multistage first step, would give a key name github_ssh_key
so we can use it when we invoke docker build
:
RUN --mount=type=ssh,id=github_ssh_key pip wheel \
--no-cache \
--requirement requirements.txt \
--wheel-dir=/app/wheels
The OP Jesus Garcia did report (in the comments) making it work:
I had to use 2 separate
RUN
commands.I'm not sure if it's a limitation of this new feature, or the way I was trying to string together multiple commands in my
RUN
but I kept getting a publickey permission denied error when I added it asother commands && /bin/sh -c "mount=type=ssh,id=github_ssh_key pip install private-repo"
vsRUN --mount=type=ssh,id=github_ssh_key pip install private-repo && more commands ...
-
I'm running into an issue when running the
RUN --mount-type=ssh ...
portion. I get/bin/sh: 1: --mount=type=ssh,: not found
I did some googling, but I wasn't able to find anything useful. Commented May 1, 2019 at 18:16 -
@JesusGarcia This looks like: medium.com/@tonistiigi/…. You are not using BuildKit backend. Check your Docker version and the "Using secrets" section of the article, where you need to set
DOCKER_BUILDKIT=1
.– VonCCommented May 1, 2019 at 19:08 -
I did set DOCKER_BUILDKIT=1 and I am also on the correct docker version. I managed to get rid of the error by running it as
/bin/sh -c "mount=type=ssh,id=github_ssh_key pip install private-repo"
but I am now getting anPermission denied (publickey). #12 35.87 fatal: Could not read from remote repository.
Commented May 1, 2019 at 20:40 -
I did get it to work. I had to use 2 separate
RUN
commands. I'm not sure if it's a limitation of this new feature, or the way I was trying to string together multiple commands in myRUN
but I kept getting a publickey permission denied error when I added it asother commands && /bin/sh -c "mount=type=ssh,id=github_ssh_key pip install private-repo"
vsRUN --mount=type=ssh,id=github_ssh_key pip install private-repo && more commands ...
Thanks for pointing me to the --ssh flag feature Commented May 7, 2019 at 17:37 -
1@dallonsi For a private key passphrase protected, you would need buildkit: stackoverflow.com/a/64036342/6309– VonCCommented Feb 15, 2021 at 15:47
This worked for me with docker 19.03.5 (but should work with 18.09+):
Say your private repo is on Github, github.com/user/repo.git
and you want to pip install
it as part of a docker build. The ssh key to the repo is located on the host at ~/.ssh/my_key
.
In the Dockerfile:
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh pip install git+ssh://[email protected]/user/repo.git
Then, on the host:
export DOCKER_BUILDKIT=1
eval `ssh-agent`
ssh-add ~/.ssh/my_key
docker build --ssh default=$SSH_AUTH_SOCK .
-
works for me. Thanks for notify that ssh-agent in host should be working even we are passing the ssh key to the docker container to build.– Luk AronCommented Jan 30, 2021 at 16:55
Despite using modern (19.03.8) Docker, neither of the above solutions worked for me (even after prepending DOCKER_BUILDKIT=1
) without an extra line I found in Alexandra Ulsh's blog post, which I think is significant enough to deserve an extra answer rather than being buried in the comments.
Though the
--secret
flag appears in a production version of Docker, there are a few hints that support is still somewhat experimental.For example, the first line of a Dockerfile using build secrets must be
# syntax = docker/dockerfile:1.0-experimental
. Without this line you’ll get the errorfailed to create LLB definition: Dockerfile parse error line 6: Unknown flag: mount
. This line enables the Docker CLI to use the “experimental Dockerfile frontend” for Moby BuildKit.
TL;DR add the commment # syntax = docker/dockerfile:1.0-experimental
at the top of your Dockerfile, then follow Dave Rejkher's answer.
I am on Docker version 19.03.13 (build 4484c46d9d), and using the --ssh
feature as described in VonC's and Dave Reikher's answers did not work for me. I still keep getting the error that:
ERROR: Command errored out with exit status 128: git clone -q 'ssh://git@hostname:port/group/repository.git' /tmp/pip-install-8eeeaipo/baseapi Check the logs for full command output.
Note that I am not pip install
-ing from a Github repo, but from some really private/internal repository. The examples also from the Docker docs on --ssh
also show a "github.com" repo URL. I am not sure if that's the case (it shouldn't be), but I've traced it up to pip
/git
not recognizing the hostname of the URL, while if using "github.com" it works as advertised.
I made it work instead by using just the plain --secret
function (still part of BuildKit), by passing-in the SSH key (~/.ssh/id_rsa
) and the ~/.ssh/known_hosts
file as secrets.
Dockerfile
# syntax=docker/dockerfile:1.0-experimental
FROM python:3.8-alpine3.11
RUN apk add openssh-client git
RUN mkdir -p -m 0600 /root/.ssh
RUN --mount=type=secret,id=known_hosts,dst=/root/.ssh/known_hosts \
--mount=type=secret,id=ssh_key,dst=/root/.ssh/id_rsa \
ssh-keyscan -H -t rsa hostname.of.private.repo \
&& pip install --no-cache-dir git+ssh://git@hostname:port/group/repository.git@12345678#egg=baseapi
Build Command
DOCKER_BUILDKIT=1 docker build \
--secret id=known_hosts,src=~/.ssh/known_hosts \
--secret id=ssh_key,src=~/.ssh/id_rsa \
-t name:tag \
-f Dockerfile \
.
I checked the ~/.ssh
folder on the built image and it seems it's still correct that it does not persist the SSH keys. The mounted files remain but they are empty:
/app # ls -l ~/.ssh/
total 0
-rwxr-xr-x 1 root root 0 Nov 19 11:29 id_rsa
-rwxr-xr-x 1 root root 0 Nov 19 11:29 known_hosts
/app # cat ~/.ssh/id_rsa
/app # cat ~/.ssh/known_hosts
And trying to pip install
the same private package should fail.
For reference:
* Docker: 19.03.13 (build 4484c46d9d)
* Image
- pip 20.2.4 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)
- git 2.22.4
- OpenSSH_8.1p1, OpenSSL 1.1.1g 21 Apr 2020
-
Does this works if that private repo has also some other private dependency on some private/internal repository.– Luk AronCommented Jan 30, 2021 at 14:05
-
@LukAron
apk
is the package manager for Alpine. If you are getting command not found, then your Docker image is probably not on Alpine. Use whichever is available on your container. Commented Jan 30, 2021 at 16:43 -
@LukAron As for your 1st question, it's not clear what you mean, and I think you will have to post your own separate question to describe your own separate problem. Commented Jan 30, 2021 at 16:47
Use RUN --mount=type=ssh
ci.yml
build_docker:
stage: build
image: docker:latest
variables:
DOCKER_HOST: tcp://docker:2375
# This instructs Docker not to start over TLS.
DOCKER_TLS_CERTDIR: ""
before_script:
- apk add --no-cache curl jq python3 py3-pip git bind-tools openssh-client
- echo $DOCKERPASS | docker login --username $DOCKERUSER --password-stdin
- mkdir -p -m 0700 ~/.ssh
- ssh-keyscan private-url >> ~/.ssh/known_hosts
- chmod 600 $SSH_KEY
services:
- docker:dind
script:
- export DOCKER_BUILDKIT=1
- eval $(ssh-agent)
- ssh-add $SSH_KEY
- docker build -t tag_name --ssh default=$SSH_AUTH_SOCK .
dockerfile
FROM puckel/docker-airflow:1.10.1
ARG SSH_PRIVATE_KEY
USER root
RUN apt-get update && apt-get install -y git openssh-client
RUN ssh-keyscan private-url >> ~/.ssh/known_hosts
COPY requirements.txt .
RUN --mount=type=ssh pip install git+ssh://private-url/project/repo.git
RUN pip install -r requirements.txt
see the picture https://feiyang233.club/img/vm/23.png
This is hacky, but it sure is simple: You can make a personal access token (PAT) on github, and then add some git config magic to just find-replace that string into the URL.
In the Dockerfile:
ARG GH_PAT
RUN git config --global url."https://$GH_PAT@github".insteadOf https://github
RUN pip install -r docker-requirements.txt
Then, assuming GH_PAT is an environment variable, you can pass it to the build command as
docker build -t some-tag:latest --build-arg GH_PAT=$GH_PAT .