Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate caching docker build images in build-ci repo #20

Closed
CodeGat opened this issue Jul 19, 2023 · 2 comments
Closed

Investigate caching docker build images in build-ci repo #20

CodeGat opened this issue Jul 19, 2023 · 2 comments
Assignees

Comments

@CodeGat
Copy link
Member

CodeGat commented Jul 19, 2023

Docker Images don't seem to be caching very well when running GitHub actions. Potentially this could be because the base image (spack_packages) is constantly changing and therefore any images on top of it must be rebuilt form scratch.

Investigate possible solutions:

  • Check if there are any caching options in the Github Actions syntax itself - https://docs.docker.com/build/cache/backends/gha/ and https://github.com/marketplace/actions/docker-layer-caching
  • Make the environment setup (apt-get) it's own image, meaning it will be snapshotted and won't need to be rebuilt.
  • Determine if there is a need for multiple image layers at all - could we just have an image with all of the models dependencies installed, and use spack activate <model_environment> as a build arg?
    • Pros: amalgamation into a single image and easy switching between dependencies with spack activate, potential for a bit more build caching
    • Cons: Every time a model (dependency) is changed, this large one will have to be rebuilt.

The ramblings of a madman...
image

@CodeGat CodeGat self-assigned this Jul 23, 2023
@aidanheerdegen
Copy link
Member

See also this issue #8

@CodeGat
Copy link
Member Author

CodeGat commented Jul 26, 2023

Closing as this issue is better described in #9 - the info here will be useful, however

@CodeGat CodeGat closed this as completed Jul 26, 2023
@CodeGat CodeGat closed this as not planned Won't fix, can't repro, duplicate, stale Jul 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants