ci: use buildkit cache for dockerfile

[joroshiba]

Summary

Using new buildx cache instead of cargo chef for image updates, makes subsequent updates much faster locally.

Background

Cargo chef has not created huge speed up in build times due to nature of our dependencies. In local tests it improves second build by ~1 minute while slowing first build by about 3 minutes. This makes the first local build take ~4 min and then subsequent minor updates can be built in less than 30s.

Changes

• Remove cargo chef
• Utilize buildkit caches

Testing

Running locally, testing image built + CI tests on here to see how it impacts CI build time.

[joroshiba]

mounting a buildkit cache to cover the cargo registry + target dir

[joroshiba]

this used to be a seperate step, but now that it is in the cache, made a single step. Copies the executable out of the cache directory into a seperate one. We might be able to replace with out_dir if/when that is stabilized.

[SuperFluffy]

What's the reason for doing this? Is moving from target/$ARCH-unknown-linux-gnu/release to target/release invalidating the cache?

[joroshiba]

the cache is all of target/, if you run in a different command the file doesn't exist because you need to mount the cache. By moving the artifact out of the cache it persists.

[SuperFluffy]

This is nice. Shame that cargo-chef didn't work out for our purposes. We can consider bringing it back if our dependency structure changes.