Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split optional dependencies in pyproject.toml #309

Merged
merged 18 commits into from
Feb 3, 2025
Merged

Split optional dependencies in pyproject.toml #309

Changes from 2 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
6d87930
split optional deps
TomNicholas Nov 19, 2024
5b14477
comment out un-implemented readers
TomNicholas Nov 19, 2024
850c38a
remove _reader suffix
TomNicholas Nov 19, 2024
aeecf8a
add kerchunk as writer to cover fastparquet dependency
TomNicholas Nov 20, 2024
e1df506
Merge branch 'main' into split_optional_deps
TomNicholas Nov 20, 2024
a633a0c
embed the pyproject.toml in the installation docs
TomNicholas Nov 20, 2024
4d77fa1
notes on test and docs deps
TomNicholas Nov 20, 2024
32966e5
wording
TomNicholas Nov 20, 2024
aa34761
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 20, 2024
cc8a218
release note
TomNicholas Nov 20, 2024
b4cfe5c
Merge branch 'main' into split_optional_deps
TomNicholas Dec 3, 2024
28cbb03
Merge branch 'main' into split_optional_deps
TomNicholas Feb 3, 2025
2cbcd2f
update kerchunk version
TomNicholas Feb 3, 2025
af52c4c
separate remote options
TomNicholas Feb 3, 2025
65c4af1
remove sneaky git thing
TomNicholas Feb 3, 2025
a81a845
Update pyproject.toml
TomNicholas Feb 3, 2025
a34843b
update kerchunk verion
TomNicholas Feb 3, 2025
d79e348
remove commented-out readers
TomNicholas Feb 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 38 additions & 11 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,30 +23,59 @@ dynamic = ["version"]
dependencies = [
"xarray>=2024.10.0",
"numpy>=2.0.0",
"packaging",
"universal-pathlib",
"numcodecs",
"ujson",
"packaging",
]

[project.optional-dependencies]
# non-kerchunk readers
hdf_reader = [
"fsspec",
"h5py",
"hdf5plugin",
"imagecodecs",
"imagecodecs-numcodecs==2024.6.1",
"numcodecs"
]
# kerchunk-based readers
hdf5_reader = [
"kerchunk>=0.25.0",
"h5py",
]
netcdf3_reader = [
"kerchunk>=0.25.0",
"scipy",
]
fits_reader = [
"kerchunk>=0.25.0",
"astropy",
]
# un-implemented readers
# tiff_reader = [
# "kerchunk>=0.25.0",
# "tifffile",
# ]
# grib_reader = ["kerchunk>=0.25.0"]
# zarr_reader = ["zarr==3.0.0b2"]
all_readers = [
"virtualizarr[hdf]",
"virtualizarr[hdf5]",
"virtualizarr[netcdf3]",
"virtualizarr[fits]",
# "virtualizarr[tiff]",
# "virtualizarr[grib]",
# "virtualizarr[zarr]",
]
# writers
icechunk = ["icechunk"]
all = [
"virtualizarr[all_readers]",
"virtualizarr[icechunk]",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now I don't think this can possibly work, because icechunk requires zarr-python v3 but kerchunk requires <3.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not without truly living on the bleeding edge 😅

https://github.com/developmentseed/virtualize-nex-gddp-cmip6/blob/f331421ee87adcfb42cb4a1904c05ec1214eb932/pyproject.toml#L24

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we should change the public pip install to use some random branch...

Aside: Is there a reason you are using virtualizarr main there? Would it be helpful to issue another release?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we should change the public pip install to use some random branch...

sorry, I wasn't suggesting you should use that approach, but just was sharing for information about what is/isn't possible. I should've been more clear.

Aside: Is there a reason you are using virtualizarr main there? Would it be helpful to issue another release?

I wanted strict type hinting (#306)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now I don't think this can possibly work, because icechunk requires zarr-python v3 but kerchunk requires <3.

Now that #412 is in this can work again, so we should push this PR forward too. But I'm not sure whether we should do this one before or after your Pixi PR #407 @maxrjones ?

]
test = [
"codecov",
"fastparquet",
"fsspec",
"h5py",
"kerchunk>=0.2.5",
"mypy",
"netcdf4",
"numcodecs",
"pandas-stubs",
"pooch",
"pre-commit",
Expand All @@ -55,11 +84,9 @@ test = [
"pytest",
"ruff",
"s3fs",
"scipy",
"virtualizarr[hdf_reader]"
"virtualizarr[all]"
]


[project.urls]
Home = "https://github.com/TomNicholas/VirtualiZarr"
Documentation = "https://github.com/TomNicholas/VirtualiZarr/blob/main/README.md"
Expand Down
Loading