52

I have hello world web project in Rust + Actix-web. I have several problems. First is every change in code causes recompiling whole project including downloading and compiling every crate. I'd like to work like in normal development - it means cache compiled crates and only recompile my codebase. Second problem is it doesn't expose my my app. It's unreachable via web browser

Dockerfile:

FROM rust

WORKDIR /var/www/app

COPY . .

EXPOSE 8080

RUN cargo run

docker-compose.yml:

version: "3"
services:
  app:
    container_name: hello-world
    build: .
    ports:
      - '8080:8080'
    volumes:
      - .:/var/www/app
      - registry:/root/.cargo/registry

volumes:
  registry:
    driver: local

main.rs:

extern crate actix_web;

use actix_web::{web, App, HttpServer, Responder};

fn index() -> impl Responder {
    "Hello world"
}

fn main() -> std::io::Result<()> {
    HttpServer::new(|| App::new().service(web::resource("/").to(index)))
        .bind("0.0.0.0:8080")?
        .run()
}

Cargo.toml:

[package]
name = "hello-world"
version = "0.1.0"
authors = []
edition = "2018"

[dependencies]
actix-web = "1.0"
3
  • please provide your cargo.toml as well
    – ckaserer
    Oct 20, 2019 at 14:59
  • Your Dockerfile is missing a CMD line; what's the container supposed to run when it starts up? (Is the build sequence actually completing?)
    – David Maze
    Oct 20, 2019 at 15:07
  • If the answer solved your question please accept it or provide additional information to further debug the issue at hand
    – ckaserer
    Nov 18, 2019 at 12:37

10 Answers 10

49

Seems like you are not alone in your endeavor to cache rust dependencies via the docker build process. Here is a great article that helps you along the way: https://blog.mgattozzi.dev/caching-rust-docker-builds/

The gist of it is you need a dummy.rs and your Cargo.toml first, then build it to cache the dependencies and then copy your application source later in order to not invalidate the cache with every build.

Dockerfile

FROM rust
WORKDIR /var/www/app
COPY dummy.rs .
COPY Cargo.toml .
RUN sed -i 's#src/main.rs#dummy.rs#' Cargo.toml
RUN cargo build --release
RUN sed -i 's#dummy.rs#src/main.rs#' Cargo.toml
COPY . .
RUN cargo build --release
CMD ["target/release/app"]

CMD application name "app" is based on what you have specified in your Cargo.toml for your binary.

dummy.rs

fn main() {}

Cargo.toml

[package]
name = "app"
version = "0.1.0"
authors = ["..."]
[[bin]]
name = "app"
path = "src/main.rs"

[dependencies]
actix-web = "1.0.0"

src/main.rs

extern crate actix_web;

use actix_web::{web, App, HttpServer, Responder};

fn index() -> impl Responder {
    "Hello world"
}

fn main() -> std::io::Result<()> {
    HttpServer::new(|| App::new().service(web::resource("/").to(index)))
        .bind("0.0.0.0:8080")?
        .run()
}
11
  • 23
    Great answer. For reference, COPY dummy.rs can be replaced by RUN echo "fn main() {}" > dummy.rs, which avoids having a file dummy.rs on the repository. Mar 22, 2020 at 7:11
  • 1
    Instead of manipulating the Cargo.toml you could add a 2nd [[bin]] target to it. Make sure to copy src/dummy.rs together with your Cargo.toml. Then add a 2nd bin target (like lines 5-7 fro your example). Set name= "download-only", path = "src/dummy.rs" and call cargo build --bin download-only. Mar 14, 2021 at 15:59
  • 5
    If you're doing RUN echo "fn main() {}" > ./src/main.rs, then COPY ./src ./src, cargo may not rebuild your app. You'll need to update the last modified of the main.rs file to inform cargo to rebuild it. Before your 2nd cargo build, you need to run RUN touch -a -m ./src/main.rs to update the last modified of the file.
    – Ari Seyhun
    Mar 20, 2021 at 11:34
  • 2
    My cargo.toml didn't have a main.rs in it, and incorporating some of the other comments, I came up with this version that does not need sed or dummy.rs. Docker caches by content so I don't think you need any special touch command. RUN mkdir src RUN echo "fn main() {}" > ./src/main.rs COPY ["Cargo.toml", "Cargo.lock", "./"] RUN cargo build --release COPY src src RUN cargo build --release
    – casret
    Oct 7, 2022 at 23:02
  • 1
    The original blog.mgatozzi.dev article is now 404. The Wayback Machine is your friend: web.archive.org/web/20221028051630/https://blog.mgattozzi.dev/… Jan 7 at 1:19
30

With the (still experimental) Docker Buildkit you can finally properly cache build folders during a docker build step:

Dockerfile:

# syntax=docker/dockerfile:experimental
from rust
ENV HOME=/home/root
WORKDIR $HOME/app
[...]
RUN --mount=type=cache,target=/usr/local/cargo/registry \
    --mount=type=cache,target=/home/root/app/target \
    cargo build --release

Then run:

DOCKER_BUILDKIT=1 docker build . --progress=plain

Subsequent docker builds will reuse the cargo and target folders from cache, hence massively speeding up your builds.

To clear the docker cache mount: docker builder prune --filter type=exec.cachemount

If you don't see proper caching: Make sure to confirm the location of your cargo/registry and target folders in the docker image, if you don't see proper caching.

Minimal working Example: https://github.com/benmarten/sccache-docker-test/tree/no-sccache

2
  • Won't using --mount=type=cache,target=/home/rust/src/target mean that other simultaneous builds access the target directory? Should sharing=private or sharing=locked be added to that cache line? Perhaps in combination with a separate layer to populate a build local target/ from the cache and another layer to push the new stuff back into the cache? Jan 5, 2021 at 23:44
  • If you're using multi-stage build and need copy something from cache (for example built executable binary) you have to copy it out of the cache. For exmaple please see this answer stackoverflow.com/a/64141061/4183352
    – cymruu
    Jun 21 at 15:53
13

You can use cargo-chef to leverage Docker layer caching using a multi-stage build.

FROM rust as planner
WORKDIR app
# We only pay the installation cost once, 
# it will be cached from the second build onwards
RUN cargo install cargo-chef 
COPY . .
RUN cargo chef prepare  --recipe-path recipe.json

FROM rust as cacher
WORKDIR app
RUN cargo install cargo-chef
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json

FROM rust as builder
WORKDIR app
COPY . .
# Copy over the cached dependencies
COPY --from=cacher /app/target target
RUN cargo build --release --bin app

FROM rust as runtime
WORKDIR app
COPY --from=builder /app/target/release/app /usr/local/bin
ENTRYPOINT ["./usr/local/bin/app"]

It does not require Buildkit and works for both simple projects and workspaces. You can find more details in the release announcement.

4
  • 1
    Won't using --mount=type=cache,target=/home/rust/src/target mean that other simultaneous builds access the target directory? Should sharing=private or sharing=locked be added to that cache line? Perhaps in combination with a separate layer to populate a build local target/ from the cache and another layer to push the new stuff back into the cache? Jan 5, 2021 at 23:44
  • I am getting this error: docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "./usr/local/bin/app": stat ./usr/local/bin/app: no such file or directory: unknown. Feb 2, 2021 at 13:45
  • 1
    Update: I fixed the error by deleting the dot in the array value of "ENTRYPOINT". Feb 2, 2021 at 13:56
  • This is nice answer. But it lack caching index of crates.io and also it takes long to copy big target directory
    – S.R
    Apr 22, 2022 at 10:26
6

While electronix384128 answer is excellent. I would like to expand on it by adding cache for .cargo/git which is needed for any dependency using git and by adding a multistage docker example.

Using rust-musl-builder and Docker Buildkit feature, which is now default in Docker Desktop 2.4. On other versions you may still need to enable it via: DOCKER_BUILDKIT=1 docker build .

rusl-musl-builder's working directory is /home/rust/src
Tried setting uid/gid on --mount but failed to compile rust due to permission issue in target.

# syntax=docker/dockerfile:1.2
FROM ekidd/rust-musl-builder:stable AS builder

COPY . .
RUN --mount=type=cache,target=/home/rust/.cargo/git \
    --mount=type=cache,target=/home/rust/.cargo/registry \
    --mount=type=cache,sharing=private,target=/home/rust/src/target \
    sudo chown -R rust: target /home/rust/.cargo && \
    cargo build --release && \
    # Copy executable out of the cache so it is available in the final image.
    cp target/x86_64-unknown-linux-musl/release/my-executable ./my-executable

FROM alpine
COPY --from=builder /home/rust/src/my-executable .
USER 1000
CMD ["./my-executable"]
3
  • Won't using --mount=type=cache,target=/home/rust/src/target mean that other simultaneous builds access the target directory? Should sharing=private or sharing=locked be added to that cache line? Perhaps in combination with a separate layer to populate a build local target/ from the cache and another layer to push the new stuff back into the cache? Jan 5, 2021 at 23:44
  • 1
    Thanks for the hint, I have updated the example and dockerfile syntax is now no longer experimental 👏
    – jetersen
    Feb 7, 2021 at 18:03
  • Thank you for the comment you made about copying executable out of the cache. It is very helpful!
    – cymruu
    Jun 21 at 15:51
5

Based on @ckaserer's answer, you can RUN echo "fn main() {}" > ./src/main.rs to build the dependencies before building your app.

First copy just your Cargo.toml and Cargo.lock files and build the dummy main.rs file:

FROM rust as rust-builder
WORKDIR /usr/src/app

# Copy Cargo files
COPY ./Cargo.toml .
COPY ./Cargo.lock .

# Create fake main.rs file in src and build
RUN mkdir ./src && echo 'fn main() { println!("Dummy!"); }' > ./src/main.rs
RUN cargo build --release

Then you can copy over your real src directory and run build again:

# Copy source files over
RUN rm -rf ./src
COPY ./src ./src

# The last modified attribute of main.rs needs to be updated manually,
# otherwise cargo won't rebuild it.
RUN touch -a -m ./src/main.rs

RUN cargo build --release

Then we can copy our file to a slim version of debain. Here's the full docker file:

FROM rust as rust-builder
WORKDIR /usr/src/app
COPY ./Cargo.toml .
COPY ./Cargo.lock .
RUN mkdir ./src && echo 'fn main() { println!("Dummy!"); }' > ./src/main.rs
RUN cargo build --release
RUN rm -rf ./src
COPY ./src ./src
RUN touch -a -m ./src/main.rs
RUN cargo build --release

FROM debian:buster-slim
COPY --from=rust-builder /usr/src/app/target/release/app /usr/local/bin/
WORKDIR /usr/local/bin
CMD ["app"]
1
  • in my case, after RUN rm -rf ./src i need to add RUN rm -rf ./target/release also Apr 30, 2022 at 8:20
3

This is a refinement of the @ckaserer's answer incorporating the comments and some personal experiences. It does not require you create a dummy file in your repo, or edit the Cargo.toml file at build time.

RUN echo 'fn main() { panic!("Dummy Image Called!")}' > ./src/main.rs
COPY ["Cargo.toml", "Cargo.lock",  "./"]
RUN cargo build --release
COPY src src
#need to break the cargo cache
RUN touch ./src/main.rs
RUN cargo build --release
1

I think the problem is your volumes definition is not doing a bind mount. I believe your current config is copying HOST ./registry/ into DOCKER /root/.cargo/registry/, writing to DOCKER /root/.cargo/registry/, and discarding the contents when the container is shut down.

Instead, you'll want to specify the bind type on the volume:

version: "3"
services:
  app:
    container_name: hello-world
    build: .
    environment:
      - CARGO_HOME=/var/www/
    ports:
      - '8080:8080'
    volumes:
      - .:/var/www/app
      - type: bind
        source: ./registry
        target: /root/.cargo/registry

However, keep in mind a /root/.cargo/.package-cache file is also created, but not kept here. Instead, you could change the source to ./.cargo and target to /root/.cargo.


For my own (mostly cli) rust projects, I like to use a drop-in replacement I've written for cargo that I've confirmed caches packages between builds, greatly reducing build time. This can be copied to /usr/local/bin to be used globally, or ran as ./cargo build within a single project. But do keep in mind this specific script assumes the app is located at /usr/src/app within the container, so it'd likely need adjusting for your use.

1
  • Can you share what your Dockerfile would look like here? Nov 30, 2022 at 15:54
1

This is what i do, and it's compatible with build scripts. It's a multi-stage build so it results in a small image, but caches the built dependencies in the first image.

FROM rust:1.43 AS builder

RUN apt-get update
RUN cd /tmp && USER=root cargo new --bin <projectname>
WORKDIR /tmp/<projectname>

# cache rust dependencies in docker layer
COPY Cargo.toml Cargo.lock ./
RUN touch build.rs && echo "fn main() {println!(\"cargo:rerun-if-changed=\\\"/tmp/<projectname>/build.rs\\\"\");}" >> build.rs
RUN cargo build --release

# build the real stuff and disable cache via the ADD
ADD "https://www.random.org/cgi-bin/randbyte?nbytes=10&format=h" skipcache
COPY ./build.rs ./build.rs

# force the build.rs script to run by modifying it
RUN echo " " >> build.rs
COPY ./src ./src
RUN cargo build --release

FROM rust:1.43
WORKDIR /bin
COPY --from=builder /tmp/<projectname>/target/release/server /bin/<project binary>
RUN chmod +x ./<project binary>
CMD ./<project binary>
1

I had the exact same problem as you did, and tried multiple ways at shortening the build time by caching dependencies.

1. @ckaserer's answer

It gets the job done, and with the easy-to-understand explanation of why it works, it's a great solution. However, this boils down to preference, but if you don't cache the dependencies this way, you can follow #2.

2. Use cargo-chef

@LukeMathWalker, the creator himself, goes through the steps necessary to use cargo-chef, but here's a *slightly adjusted example from the github page.

Dockerfile

FROM lukemathwalker/cargo-chef:latest-rust-1.60.0 AS chef
WORKDIR /app

FROM chef as planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef as builder
COPY --from=planner /app/recipe.json recipe.json
# Build the dependencies (and add to docker's caching layer)
# This caches the dependency files similar to how @ckaserer's solution
# does, but is handled solely through the `cargo-chef` library.
RUN cargo chef cook --release --recipe-path recipe.json
# Build the application
COPY . .
RUN cargo build --release --bin emailer

FROM debian:buster-slim AS runtime
WORKDIR /app
COPY --from=builder /app/target/release/<Name of Rust Application> /usr/local/bin
ENTRYPOINT ["/usr/local/bin/<Name of Rust Application>"]

You should notice a significant decrease in build times with the above changes!


Side note, as far as I know, this blog entry, though not on dockerized builds, holds the best info on compiling rust apps faster on your local machine. You might find it helpful so I suggest checking it out if you're interested.

0
0

I realise this answer is a little late to the party, however I believe I have found a solution that is slightly different (although along the same basic idea) but that will create the build dependencies in a single Docker layer, meaning they will be cached - you can simply copy the RUN command layer from the following:

...
COPY Cargo.toml /app/

RUN mkdir src && \
    echo 'fn main() {\nprintln!("Hello, world!");\n}' > src/main.rs && \
    cargo build && \ 
    cargo clean --package $(awk '/name/ {gsub(/"/,""); print $3}' Cargo.toml | sed ':a;N;$!ba;s/\n//g' | tr -d '\r') && \
    rm -rf src 

COPY src /app/src
...

First copy your Cargo.toml file with it's dependencies and then insert the RUN layer below, then afterwards copy in your actual code.

This first runs a dummy application (copied straight from cargo init) and this package is named the same as your project, then it runs cargo clean on just that package, meaning if you add your code and run the build again, all of the dependencies are cached already, and it just rebuilds your code. By combining the whole thing in a single layer it will save time should anything change further down the pipeline.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.