Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hard-code .zmetadata in FSStore._normalize_key #1467

Closed
wants to merge 3 commits into from

Conversation

joshmoore
Copy link
Member

The normalization code in FSStore uses a list of well-known files to prevent incorrectly re-writing keys when dimension separator is "/" rather than ".". This was initialized only with the names specified in the spec, which left out ".zmetadata" from consolidated metadata.

see: #1121

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/tutorial.rst
  • Changes documented in docs/release.rst
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

The normalization code in FSStore uses a list of well-known
files to prevent incorrectly re-writing keys when dimension
separator is "/" rather than ".". This was initialized only
with the names specified in the spec, which left out
".zmetadata" from consolidated metadata.

see: zarr-developers#1121
@github-actions github-actions bot added the needs release notes Automatically applied to PRs which haven't added release notes label Jul 17, 2023
@codecov
Copy link

codecov bot commented Jul 17, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.99%. Comparing base (0b0ac88) to head (0350a24).
Report is 521 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1467   +/-   ##
=======================================
  Coverage   99.99%   99.99%           
=======================================
  Files          38       38           
  Lines       14574    14574           
=======================================
  Hits        14573    14573           
  Misses          1        1           
Files with missing lines Coverage Δ
zarr/storage.py 100.00% <100.00%> (ø)

@joshmoore
Copy link
Member Author

It would be good to have a failing test in the suite before this gets merged.

self._array_meta_key,
self._group_meta_key,
self._attrs_key,
".zmetadata", # see: #1121
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we do this via a class attribute like _consolidated_meta_key?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and that would definitely be better than my quick fix here (intended to test the CI) but there's really no good place for it to live. It should naturally be in a Consolidated* class, but FSStore should be unaware of that. This is simply a problem of there being no spec for consolidated.

@github-actions github-actions bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Feb 14, 2024
@joshmoore
Copy link
Member Author

Any thoughts about getting this into the 2.17.x line?

@jhamman
Copy link
Member

jhamman commented Oct 11, 2024

I'm going to close this as stale. Folks should feel free to reopen if there is interest in continuing this work.

@jhamman jhamman closed this Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants