Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: use buildkit cache for dockerfile #860

Merged
merged 5 commits into from
Apr 3, 2024
Merged

ci: use buildkit cache for dockerfile #860

Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c2f3a6c
ci: use buildx cache for dockerfile
joroshiba Mar 25, 2024
5fec2a9
make a fake change to test second build time
joroshiba Mar 25, 2024
229ef00
attempt fix ci builds
joroshiba Mar 25, 2024
a5bc1b3
not really a change
joroshiba Mar 25, 2024
74ab81b
Merge branch 'main' into joroshiba/faster-docker
joroshiba Apr 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 10 additions & 36 deletions containerfiles/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
# build stage
FROM --platform=$BUILDPLATFORM lukemathwalker/cargo-chef:0.1.66-rust-1.76.0-bookworm AS chef
FROM --platform=$BUILDPLATFORM rust:1.76-bookworm AS rust

WORKDIR /build/

Expand Down Expand Up @@ -40,40 +39,17 @@ RUN \
install ./protoc/include/google/protobuf/* -Dt /usr/local/include/google/protobuf; \
install ./protoc/include/google/protobuf/compiler/* -Dt /usr/local/include/google/protobuf/compiler;

# install targets
FROM chef AS planner
ARG TARGETBINARY
COPY . .
RUN cargo chef prepare --bin $TARGETBINARY --recipe-path recipe.json

FROM chef as builder
COPY --from=planner /build/recipe.json recipe.json
FROM rust as builder

ARG BUILDPLATFORM
ARG TARGETPLATFORM
ARG TARGETBINARY
RUN \
if [ "$TARGETPLATFORM" = "linux/arm64" ] && [ "$BUILDPLATFORM" != "linux/arm64" ]; then \
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++ \
PKG_CONFIG_SYSROOT_DIR=/usr/aarch64-linux-gnu; \
TARGET_TRIPLE=aarch64-unknown-linux-gnu; \
elif [ "$TARGETPLATFORM" = "linux/amd64" ] && [ "$BUILDPLATFORM" != "linux/amd64" ]; then \
export CARGO_TARGET_x86_64_UNKNOWN_LINUX_GNU_LINKER=x86_64-linux-gnu-gcc \
CC_x86_64_unknown_linux_gnu=x86_64-linux-gnu-gcc \
CXX_x86_64_unknown_linux_gnu=x86_64-linux-gnu-g++ \
PKG_CONFIG_SYSROOT_DIR=/usr/x86_64-linux-gnu; \
TARGET_TRIPLE=x86_64-unknown-linux-gnu; \
else \
TARGET_TRIPLE=$(uname -m)-unknown-linux-gnu; \
fi; \
export PROTOC=/usr/local/bin/protoc; \
cargo chef cook --release --bin $TARGETBINARY --target $TARGET_TRIPLE --recipe-path recipe.json;

COPY . .

RUN mkdir -p release
RUN \
--mount=type=cache,target=/usr/local/cargo/registry,id=${TARGETPLATFORM}-${TARGETBINARY} \
--mount=type=cache,target=/build/target,id=${TARGETPLATFORM}-${TARGETBINARY} \
if [ "$TARGETPLATFORM" = "linux/arm64" ] && [ "$BUILDPLATFORM" != "linux/arm64" ]; then \
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
Expand All @@ -90,13 +66,11 @@ RUN \
TARGET_TRIPLE=$(uname -m)-unknown-linux-gnu; \
fi; \
export PROTOC=/usr/local/bin/protoc; \
cargo build --release --target $TARGET_TRIPLE --bin $TARGETBINARY;

# replace this with `--out` or `--out-dir` once stable
RUN mkdir -p target/release
RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then ARCH=aarch64; \
cargo build --release --target $TARGET_TRIPLE --bin $TARGETBINARY;\
# Copies the binary from out of the cache directory
if [ "$TARGETPLATFORM" = "linux/arm64" ]; then ARCH=aarch64; \
elif [ "$TARGETPLATFORM" = "linux/amd64" ]; then ARCH=x86_64; fi; \
cp target/$ARCH-unknown-linux-gnu/release/$TARGETBINARY target/release/
cp target/$ARCH-unknown-linux-gnu/release/$TARGETBINARY release/;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this used to be a seperate step, but now that it is in the cache, made a single step. Copies the executable out of the cache directory into a seperate one. We might be able to replace with out_dir if/when that is stabilized.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for doing this? Is moving from target/$ARCH-unknown-linux-gnu/release to target/release invalidating the cache?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the cache is all of target/, if you run in a different command the file doesn't exist because you need to mount the cache. By moving the artifact out of the cache it persists.

FROM debian:bookworm-slim
ARG TARGETBINARY
Expand All @@ -107,6 +81,6 @@ RUN \
apt install -y wget ca-certificates; \
apt-get clean; \
rm -rf /var/lib/apt/lists/*;
COPY --from=builder /build/target/release/$TARGETBINARY /usr/local/bin/$TARGETBINARY
COPY --from=builder /build/release/$TARGETBINARY /usr/local/bin/$TARGETBINARY
RUN ln -s /usr/local/bin/$TARGETBINARY /usr/local/bin/entrypoint
ENTRYPOINT ["/usr/local/bin/entrypoint"]
Loading