Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: source URL can be a list (meta.yaml) #5218

Merged
merged 3 commits into from
Mar 22, 2024
Merged

Conversation

ytausch
Copy link
Contributor

@ytausch ytausch commented Mar 6, 2024

Description

meta.yaml actually supports a list of URLs in the url field of a single source but this feature is currently undocumented. This is the relevant source code snippet:

source_urls = source_dict["url"]
if not isinstance(source_urls, list):
source_urls = [source_urls]
unhashed_fn = fn = (
source_dict["fn"] if "fn" in source_dict else basename(source_urls[0])
)
hash_added = False
for hash_type in ("md5", "sha1", "sha256"):
if hash_type in source_dict:
if source_dict[hash_type] in (None, ""):
raise ValueError(f"Empty {hash_type} hash provided for {fn}")
fn = append_hash_to_fn(fn, source_dict[hash_type])
hash_added = True
break
else:
log.warn(
f"No hash (md5, sha1, sha256) provided for {unhashed_fn}. Source download forced. "
"Add hash to recipe to use source cache."
)
path = join(cache_folder, fn)
if isfile(path):
if verbose:
log.info("Found source in cache: %s" % fn)
else:
if verbose:
log.info("Downloading source to cache: %s" % fn)
for url in source_urls:
if "://" not in url:
if url.startswith("~"):
url = expanduser(url)
if not os.path.isabs(url):
url = os.path.normpath(os.path.join(recipe_path, url))
url = url_path(url)
else:
if url.startswith("file:///~"):
url = "file:///" + expanduser(url[8:]).replace("\\", "/")
try:
if verbose:
log.info("Downloading %s" % url)
with LoggingContext():
download(url, path)
except CondaHTTPError as e:
log.warn("Error: %s" % str(e).strip())
rm_rf(path)
except RuntimeError as e:
log.warn("Error: %s" % str(e).strip())
rm_rf(path)
else:
if verbose:
log.info("Success")
break

Checklist - did you ...

  • Add a file to the news directory (using the template) for the next release's release notes?
  • Add / update necessary tests?
  • Add / update outdated documentation?

@ytausch ytausch requested a review from a team as a code owner March 6, 2024 18:39
@conda-bot conda-bot added the cla-signed [bot] added once the contributor has signed the CLA label Mar 6, 2024
@beeankha beeankha added type::documentation request for improved documentation source::community catch-all for issues filed by community members labels Mar 7, 2024
ryanskeith
ryanskeith previously approved these changes Mar 19, 2024
Copy link
Contributor

@ryanskeith ryanskeith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As stated in your PR, this isn't a well tested feature. I would suggest adding a warning to the docs about that for now.

@jaimergp
Copy link
Contributor

This is used routinely in the R ecosystem in conda-forge:

https://github.com/search?q=org%3Aconda-forge+%2Furl%3A%5Cn%5Cs%2B-%2F+path%3Ameta.yaml&type=code

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>
@ytausch
Copy link
Contributor Author

ytausch commented Mar 22, 2024

@beeankha thank you for fixing!

@beeankha beeankha merged commit ab137d2 into conda:main Mar 22, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed [bot] added once the contributor has signed the CLA source::community catch-all for issues filed by community members type::documentation request for improved documentation
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants