23

I have a private repo that contains packages I want to pip install. I've spent quite a bit of time reading over various forums and articles about different ways to securely do this. There doesn't seem to be a a consensus on how to best do this (if at all possible). I obviously don't want to expose any ssh keys/secrets in my dockerfile -- I want to be careful about making them available via docker history.

6 Answers 6

26

As explained in "Securely build small python docker image from private git repos", you would need to use, with Docker 18.09+

  • --ssh
    You can use the --ssh flag to forward your existing SSH agent key to the builder. Instead of transferring the key data, docker will just notify the builder that such capability is available.
    Now when the builder needs access to a remote server through SSH, it will dial back to the client and ask it to sign the specific request needed for this connection.
    The key itself never leaves the client, and as soon as the command that requested the access has completed there is no information on the builder side to reestablish that remote connection later.

  • Secrets:
    Provides a mount option during the build at /var/run/secrets available only for the command that used it and is not included in the created layer.

That is:

docker build --ssh github_ssh_key=/path/to/.ssh/git_ssh_id_rsa .

only the agent connection is shared with that command, and not the actual content of the private key.
no other commands/steps in the Dockerfile will have access to it.

The Dockerfile, in a multistage first step, would give a key name github_ssh_key so we can use it when we invoke docker build:

RUN --mount=type=ssh,id=github_ssh_key pip wheel \
    --no-cache \
    --requirement requirements.txt \
--wheel-dir=/app/wheels

The OP Jesus Garcia did report (in the comments) making it work:

I had to use 2 separate RUN commands.

I'm not sure if it's a limitation of this new feature, or the way I was trying to string together multiple commands in my RUN but I kept getting a publickey permission denied error when I added it as other commands && /bin/sh -c "mount=type=ssh,id=github_ssh_key pip install private-repo" vs RUN --mount=type=ssh,id=github_ssh_key pip install private-repo && more commands ...

11
  • I'm running into an issue when running the RUN --mount-type=ssh ... portion. I get /bin/sh: 1: --mount=type=ssh,: not found I did some googling, but I wasn't able to find anything useful. Commented May 1, 2019 at 18:16
  • @JesusGarcia This looks like: medium.com/@tonistiigi/…. You are not using BuildKit backend. Check your Docker version and the "Using secrets" section of the article, where you need to set DOCKER_BUILDKIT=1.
    – VonC
    Commented May 1, 2019 at 19:08
  • I did set DOCKER_BUILDKIT=1 and I am also on the correct docker version. I managed to get rid of the error by running it as /bin/sh -c "mount=type=ssh,id=github_ssh_key pip install private-repo" but I am now getting an Permission denied (publickey). #12 35.87 fatal: Could not read from remote repository. Commented May 1, 2019 at 20:40
  • I did get it to work. I had to use 2 separate RUN commands. I'm not sure if it's a limitation of this new feature, or the way I was trying to string together multiple commands in my RUN but I kept getting a publickey permission denied error when I added it as other commands && /bin/sh -c "mount=type=ssh,id=github_ssh_key pip install private-repo" vs RUN --mount=type=ssh,id=github_ssh_key pip install private-repo && more commands ... Thanks for pointing me to the --ssh flag feature Commented May 7, 2019 at 17:37
  • 1
    @dallonsi For a private key passphrase protected, you would need buildkit: stackoverflow.com/a/64036342/6309
    – VonC
    Commented Feb 15, 2021 at 15:47
19

This worked for me with docker 19.03.5 (but should work with 18.09+):

Say your private repo is on Github, github.com/user/repo.git and you want to pip install it as part of a docker build. The ssh key to the repo is located on the host at ~/.ssh/my_key.

In the Dockerfile:

RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh pip install git+ssh://[email protected]/user/repo.git

Then, on the host:

export DOCKER_BUILDKIT=1
eval `ssh-agent`
ssh-add ~/.ssh/my_key
docker build --ssh default=$SSH_AUTH_SOCK .
1
  • works for me. Thanks for notify that ssh-agent in host should be working even we are passing the ssh key to the docker container to build.
    – Luk Aron
    Commented Jan 30, 2021 at 16:55
7

Despite using modern (19.03.8) Docker, neither of the above solutions worked for me (even after prepending DOCKER_BUILDKIT=1) without an extra line I found in Alexandra Ulsh's blog post, which I think is significant enough to deserve an extra answer rather than being buried in the comments.

Though the --secret flag appears in a production version of Docker, there are a few hints that support is still somewhat experimental.

For example, the first line of a Dockerfile using build secrets must be # syntax = docker/dockerfile:1.0-experimental. Without this line you’ll get the error failed to create LLB definition: Dockerfile parse error line 6: Unknown flag: mount. This line enables the Docker CLI to use the “experimental Dockerfile frontend” for Moby BuildKit.

TL;DR add the commment # syntax = docker/dockerfile:1.0-experimental at the top of your Dockerfile, then follow Dave Rejkher's answer.

0
1

I am on Docker version 19.03.13 (build 4484c46d9d), and using the --ssh feature as described in VonC's and Dave Reikher's answers did not work for me. I still keep getting the error that:

ERROR: Command errored out with exit status 128: git clone -q 'ssh://git@hostname:port/group/repository.git' /tmp/pip-install-8eeeaipo/baseapi Check the logs for full command output.

Note that I am not pip install-ing from a Github repo, but from some really private/internal repository. The examples also from the Docker docs on --ssh also show a "github.com" repo URL. I am not sure if that's the case (it shouldn't be), but I've traced it up to pip/git not recognizing the hostname of the URL, while if using "github.com" it works as advertised.

I made it work instead by using just the plain --secret function (still part of BuildKit), by passing-in the SSH key (~/.ssh/id_rsa) and the ~/.ssh/known_hosts file as secrets.

Dockerfile

# syntax=docker/dockerfile:1.0-experimental

FROM python:3.8-alpine3.11

RUN apk add openssh-client git
RUN mkdir -p -m 0600 /root/.ssh
RUN --mount=type=secret,id=known_hosts,dst=/root/.ssh/known_hosts \
    --mount=type=secret,id=ssh_key,dst=/root/.ssh/id_rsa \
    ssh-keyscan -H -t rsa hostname.of.private.repo \
    && pip install --no-cache-dir git+ssh://git@hostname:port/group/repository.git@12345678#egg=baseapi

Build Command

DOCKER_BUILDKIT=1 docker build \
    --secret id=known_hosts,src=~/.ssh/known_hosts \
    --secret id=ssh_key,src=~/.ssh/id_rsa \
    -t name:tag \
    -f Dockerfile \
    .

I checked the ~/.ssh folder on the built image and it seems it's still correct that it does not persist the SSH keys. The mounted files remain but they are empty:

/app # ls -l ~/.ssh/
total 0
-rwxr-xr-x    1 root     root             0 Nov 19 11:29 id_rsa
-rwxr-xr-x    1 root     root             0 Nov 19 11:29 known_hosts
/app # cat ~/.ssh/id_rsa 
/app # cat ~/.ssh/known_hosts

And trying to pip install the same private package should fail.

For reference:

* Docker: 19.03.13 (build 4484c46d9d)
* Image
    - pip 20.2.4 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)
    - git 2.22.4
    - OpenSSH_8.1p1, OpenSSL 1.1.1g  21 Apr 2020
3
  • Does this works if that private repo has also some other private dependency on some private/internal repository.
    – Luk Aron
    Commented Jan 30, 2021 at 14:05
  • @LukAron apk is the package manager for Alpine. If you are getting command not found, then your Docker image is probably not on Alpine. Use whichever is available on your container. Commented Jan 30, 2021 at 16:43
  • @LukAron As for your 1st question, it's not clear what you mean, and I think you will have to post your own separate question to describe your own separate problem. Commented Jan 30, 2021 at 16:47
0

Use RUN --mount=type=ssh

ci.yml

build_docker:
  stage: build
  image: docker:latest
  variables:
    DOCKER_HOST: tcp://docker:2375
    # This instructs Docker not to start over TLS.
    DOCKER_TLS_CERTDIR: ""

  before_script:
    - apk add --no-cache curl jq python3 py3-pip git bind-tools openssh-client
    - echo $DOCKERPASS | docker login --username $DOCKERUSER --password-stdin
    - mkdir -p -m 0700 ~/.ssh
    - ssh-keyscan private-url >> ~/.ssh/known_hosts 
    - chmod 600 $SSH_KEY

  services:
    - docker:dind

  script:
    - export DOCKER_BUILDKIT=1
    - eval $(ssh-agent)
    - ssh-add $SSH_KEY
    - docker build -t tag_name  --ssh default=$SSH_AUTH_SOCK .

dockerfile

FROM puckel/docker-airflow:1.10.1

ARG SSH_PRIVATE_KEY

USER root
RUN apt-get update && apt-get install -y git openssh-client


RUN ssh-keyscan private-url >> ~/.ssh/known_hosts

COPY requirements.txt .

RUN --mount=type=ssh pip install git+ssh://private-url/project/repo.git
RUN pip install -r requirements.txt

see the picture https://feiyang233.club/img/vm/23.png

1
  • will it work with USER airflow ?
    – Niv Cohen
    Commented Apr 19, 2022 at 21:10
0

This is hacky, but it sure is simple: You can make a personal access token (PAT) on github, and then add some git config magic to just find-replace that string into the URL.

In the Dockerfile:

ARG GH_PAT
RUN git config --global url."https://$GH_PAT@github".insteadOf https://github
RUN pip install -r docker-requirements.txt

Then, assuming GH_PAT is an environment variable, you can pass it to the build command as

docker build -t some-tag:latest --build-arg GH_PAT=$GH_PAT .

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.