Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

download_strategy: compare cached file size to Content-Length #19460

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

EricFromCanada
Copy link
Member

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew style with your changes locally?
  • Have you successfully run brew typecheck with your changes locally?
  • Have you successfully run brew tests with your changes locally?

Before using a cached download, also check that the Content-Length header returned from the URL matches the size of the file. This should catch instances where a newer version of a cask isn't downloaded because a cask's URL doesn't change between updates, i.e. Homebrew/homebrew-cask#204165.

Fixes Homebrew/homebrew-cask#204166, but there's bound to be edge cases.

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Some style tweaks and should be good to go.

@EricFromCanada EricFromCanada force-pushed the cache-consider-file-size branch from 71f5944 to 14a541f Compare March 11, 2025 13:36
Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again!

@MikeMcQuaid MikeMcQuaid enabled auto-merge March 11, 2025 13:39
# - Content-Length value is different than the file's size
newer_last_modified = last_modified && last_modified > cached_location.mtime
different_file_size = file_size && file_size != cached_location.size
use_cached_location = false if cached_location.exist? && (newer_last_modified || different_file_size)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be inscrutable, but if you want to short-circuit:

Suggested change
use_cached_location = false if cached_location.exist? && (newer_last_modified || different_file_size)
use_cached_location &&= cached_location.exist? && (newer_last_modified || different_file_size)

@@ -410,7 +410,7 @@ def fetch(timeout: nil)
use_cached_location = cached_location.exist?
use_cached_location = false if version.respond_to?(:latest?) && version.latest?

resolved_url, _, last_modified, _, is_redirection = begin
resolved_url, _, last_modified, file_size, is_redirection = begin
resolve_url_basename_time_file_size(url, timeout: Utils::Timer.remaining!(end_time))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe not in this PR, but both the name and return type strongly suggest using a URLMetadata (or such) struct here instead.

@EricFromCanada
Copy link
Member Author

Not sure how to get the tests to pass:

$ brew tests --only download_strategies/curl --fail-fast
Randomized with seed 42865
1 process for 1 spec, ~ 1 spec per process
.......F

Failures:

  1) CurlDownloadStrategy#fetch calls curl with default arguments
     Failure/Error: strategy.fetch
     
     Errno::ENOENT:
       No such file or directory @ rb_file_s_size - /private/tmp/homebrew-tests-20250311-35401-g5955c/cache/downloads/3d1c0ae7da22be9d83fb1eb774df96b7c4da71d3cf07e1cb28555cf9a5e5af70--foo.tar.gz
     # ./download_strategy.rb:426:in `size'
     # ./download_strategy.rb:426:in `size'
     # ./download_strategy.rb:426:in `fetch'
     # ./test/download_strategies/curl_spec.rb:48:in `block (3 levels) in <top (required)>'

@Homebrew Homebrew deleted a comment from EricFromCanada Mar 11, 2025
@dduugg
Copy link
Member

dduugg commented Mar 11, 2025

Not sure how to get the tests to pass:

$ brew tests --only download_strategies/curl --fail-fast
Randomized with seed 42865
1 process for 1 spec, ~ 1 spec per process
.......F

Failures:

  1) CurlDownloadStrategy#fetch calls curl with default arguments
     Failure/Error: strategy.fetch
     
     Errno::ENOENT:
       No such file or directory @ rb_file_s_size - /private/tmp/homebrew-tests-20250311-35401-g5955c/cache/downloads/3d1c0ae7da22be9d83fb1eb774df96b7c4da71d3cf07e1cb28555cf9a5e5af70--foo.tar.gz
     # ./download_strategy.rb:426:in `size'
     # ./download_strategy.rb:426:in `size'
     # ./download_strategy.rb:426:in `fetch'
     # ./test/download_strategies/curl_spec.rb:48:in `block (3 levels) in <top (required)>'

Ah, I think line 426 needs a cached_location.exist? guard

@EricFromCanada
Copy link
Member Author

EricFromCanada commented Mar 11, 2025

Ah, I think line 426 needs a cached_location.exist? guard

Odd that cached_location.mtime doesn't error out on the previous line though.

@dduugg
Copy link
Member

dduugg commented Mar 11, 2025

Ah, I think line 426 needs a cached_location.exist? guard

Odd that cached_location.mtime doesn't error out on the previous line though.

I'm guessing that there's no last_modified within the test case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

roboform cask fails to upgrade due to unversioned URL and caching behavior
4 participants