Skip to content

Commit

Permalink
more mods
Browse files Browse the repository at this point in the history
  • Loading branch information
@jhamman
jhamman committed Dec 20, 2024
1 parent dd3805b commit 22a5705
Show file tree
Hide file tree
Showing 13 changed files with 754 additions and 680 deletions.
4 changes: 2 additions & 2 deletions docs/developers/index.rst
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Developer Guide
---------------
Developer's Guide
-----------------

.. toctree::
:maxdepth: 1
Expand Down
34 changes: 17 additions & 17 deletions docs/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,14 @@ Zarr-Python
:hidden:

quickstart
about
user-guide/index
api/index
developers/index
about
developers/release

**Version**: |version|

**Download documentation**: `PDF/Zipped HTML <https://readthedocs.org/projects/zarr/downloads/>`_

**Useful links**:
`Source Repository <https://github.com/zarr-developers/zarr-python>`_ |
`Issue Tracker <https://github.com/zarr-developers/zarr-python/issues>`_ |
`Zulip Chat <https://ossci.zulipchat.com/>`_ |
`Zarr specifications <https://zarr-specs.readthedocs.io>`_

Zarr is a storage format for chunked, compressed, N-dimensional arrays based on an open-source specification. Highlights include:

* Create N-dimensional arrays with any NumPy dtype.
Expand All @@ -34,9 +27,6 @@ Zarr is a storage format for chunked, compressed, N-dimensional arrays based on
* Write to an array concurrently from multiple threads or processes.
* Organize arrays into hierarchies via groups.

Zarr-Python Documentation
-------------------------

.. grid:: 2

.. grid-item-card::
Expand Down Expand Up @@ -93,23 +83,33 @@ Zarr-Python Documentation
:color: dark
:click-parent:

To the api reference guide
To the API reference guide

.. grid-item-card::
:img-top: _static/index_contribute.svg

Contributor's Guide
^^^^^^^^^^^^^^^^^^^
Developer's Guide
^^^^^^^^^^^^^^^^^

Want to contribute to Zarr? We welcome contributions in the form of bug reports,
bug fixes, documentation, enhancement proposals and more. The contributing
guidelines will guide you through the process of improving Zarr.

+++

.. button-ref:: developers/contributing
.. button-ref:: developers/index
:expand:
:color: dark
:click-parent:

To the contributor's guide
To the developers's guide

**Useful links**:
`Source Repository <https://github.com/zarr-developers/zarr-python>`_ |
`Issue Tracker <https://github.com/zarr-developers/zarr-python/issues>`_ |
`Zulip Chat <https://ossci.zulipchat.com/>`_ |
`Zarr specifications <https://zarr-specs.readthedocs.io>`_

**Download documentation**: `PDF/Zipped HTML <https://readthedocs.org/projects/zarr/downloads/>`_

.. _NumCodecs: https://numcodecs.readthedocs.io
121 changes: 79 additions & 42 deletions docs/quickstart.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,12 @@
Quickstart
==========

Welcome to the Zarr-Python Quickstart guide! This page will help you get up and running with the Zarr library in Python to efficiently manage and analyze multi-dimensional arrays.
Welcome to the Zarr-Python Quickstart guide! This page will help you get up and running with
the Zarr library in Python to efficiently manage and analyze multi-dimensional arrays.

Zarr is a powerful library for storage of n-dimensional arrays, supporting chunking, compression, and various backends, making it a versatile choice for scientific and large-scale data.
Zarr is a powerful library for storage of n-dimensional arrays, supporting chunking,
compression, and various backends, making it a versatile choice for scientific and
large-scale data.

Installation
------------
Expand All @@ -32,102 +35,136 @@ or `conda`:
Creating an Array
-----------------

To get started, you can create a simple Zarr array using the in-memory store:
To get started, you can create a simple Zarr array:

.. ipython:: python
import zarr
import numpy as np
# Create a 2D Zarr array
z = zarr.zeros((100, 100), chunks=(10, 10), dtype='f4')
z = zarr.zeros(
store="data/example-1.zarr",
shape=(100, 100),
chunks=(10, 10),
dtype="f4"
)
# Assign data to the array
z[:, :] = np.random.random((100, 100))
z.info
print(z.info)
Here, we created a 2D array of shape ``(100, 100)``, chunked into blocks of ``(10, 10)``, and filled it with random floating-point data.
Here, we created a 2D array of shape ``(100, 100)``, chunked into blocks of
``(10, 10)``, and filled it with random floating-point data. This array was
written to a ``LocalStore`` in the ``data/example-1.zarr`` directory.

Persistent Storage
------------------
Compression and Filters
~~~~~~~~~~~~~~~~~~~~~~~

Zarr supports persistent storage to disk or cloud-compatible backends. To store arrays on the filesystem:
Zarr supports data compression and filters. For example, to use Blosc compression:

.. ipython:: python
# Store the array in a directory on disk
from numcodecs import Blosc
z = zarr.open(
'data/example-1.zarr',
mode='w', shape=(100, 100), chunks=(10, 10), dtype='f4'
"data/example-3.zarr",
mode="w", shape=(100, 100),
chunks=(10, 10), dtype="f4",
compressor=Blosc(cname="zstd", clevel=3, shuffle=Blosc.SHUFFLE),
zarr_format=2
)
z[:, :] = np.random.random((100, 100))
print("Array stored at 'data/example-1.zarr'")
z.info
To open an existing array:

.. ipython:: python
z = zarr.open('data/example-1.zarr', mode='r')
print(z[:])
This compresses the data using the Zstandard codec with shuffle enabled for better compression.

Hierarchical Groups
-------------------

Zarr allows creating hierarchical groups, similar to directories:
Zarr allows you to create hierarchical groups, similar to directories:

.. ipython:: python
# Create a group and add arrays
root = zarr.group('data/example-2.zarr')
foo = root.create_array(name='foo', shape=(1000, 100), chunks=(10, 10), dtype='f4')
bar = root.create_array(name='bar', shape=(100,), dtype='i4')
# Create nested groups and add arrays
root = zarr.group("data/example-2.zarr")
foo = root.create_group(name="foo")
bar = root.create_array(
name="bar", shape=(100, 10), chunks=(10, 10)
)
spam = foo.create_array(name="spam", shape=(10,), dtype="i4")
# Assign values
foo[:, :] = np.random.random((1000, 100))
bar[:] = np.arange(100)
bar[:, :] = np.random.random((100, 10))
spam[:] = np.arange(10)
# print the hierarchy
root.tree()
This creates a group with two datasets: ``foo`` and ``bar``.

.. Compression and Filters
.. -----------------------
Persistent Storage
------------------

Zarr supports persistent storage to disk or cloud-compatible backends. While examples above
utilized a :class:`zarr.storage.LocalStore`, a number of other storage options are available,
including the :class:`zarr.storage.ZipStore` and :class:`zarr.storage.FsspecStore`.

.. Zarr supports data compression and filters. For example, to use Blosc compression:
.. ipython:: python
.. .. ipython:: python
.. :verbatim:
# Store the array in a ZIP file
store = zarr.storage.ZipStore("data/example-3.zip", mode='w')
.. z = zarr.open('data/example-3.zarr', mode='w', shape=(100, 100), chunks=(10, 10), dtype='f4',
.. compressor=zarr.Blosc(cname='zstd', clevel=3, shuffle=zarr.Blosc.SHUFFLE))
.. z[:, :] = np.random.random((100, 100))
z = zarr.open(
store=store,
mode="w",
shape=(100, 100),
chunks=(10, 10),
dtype="f4"
)
.. print(z.info)
# write to the array
z[:, :] = np.random.random((100, 100))
.. This compresses the data using the Zstandard codec with shuffle enabled for better compression.
# the ZipStore must be explicitly closed
store.close()
To open an existing array:

.. ipython:: python
# Open the ZipStore in read-only mode
store = zarr.storage.ZipStore("data/example-3.zip", read_only=True)
z = zarr.open(store, mode='r')
# read the data as a NumPy Array
z[:]
Cloud Storage Backends
----------------------
~~~~~~~~~~~~~~~~~~~~~~

Zarr integrates seamlessly with cloud storage such as Amazon S3 and Google Cloud Storage using external libraries like `s3fs` or `gcsfs`.
Zarr integrates seamlessly with cloud storage such as Amazon S3 and Google Cloud Storage
using external libraries like `s3fs <https://s3fs.readthedocs.io>`_ or
`gcsfs <https://gcsfs.readthedocs.io>`_.

For example, to use S3:

.. ipython:: python
:verbatim:
import s3fs
import zarr
z = zarr.open("s3://example-bucket/foo", mode='w', shape=(100, 100), chunks=(10, 10))
z = zarr.open("s3://example-bucket/foo", mode="w", shape=(100, 100), chunks=(10, 10))
z[:, :] = np.random.random((100, 100))
Read more about Zarr's storage options in the `User Guide <user-guide/storage.html>`_.

Next Steps
----------

Now that you're familiar with the basics, explore the following resources:

- `User Guide <guide>`_
- `User Guide <user-guide>`_
- `API Reference <api>`_
Loading

0 comments on commit 22a5705

Please sign in to comment.