diff --git a/docs/articles.md b/docs/articles.md index b33e79992..42921fe56 100644 --- a/docs/articles.md +++ b/docs/articles.md @@ -1,5 +1,5 @@ -# Articles +# Articles and Presentations -[Cubed: Bounded-memory serverless array processing in xarray](https://xarray.dev/blog/cubed-xarray) - -[Optimizing Cubed](https://medium.com/pangeo/optimizing-cubed-7a0b8f65f5b7) +- [Cubed: an introduction](https://cubed-dev.github.io/cubed/cubed-intro.slides.html) by Tom White, November 2024 +- [Optimizing Cubed](https://medium.com/pangeo/optimizing-cubed-7a0b8f65f5b7) by Tom White, April 3, 2024 +- [Cubed: Bounded-memory serverless array processing in xarray](https://xarray.dev/blog/cubed-xarray) by Tom Nicholas and Tom White, June 28th, 2023 diff --git a/docs/getting-started.md b/docs/getting-started.md new file mode 100644 index 000000000..47732b68b --- /dev/null +++ b/docs/getting-started.md @@ -0,0 +1,61 @@ +# Getting Started + +We'll start with a simple example that runs locally. + +## Installation + +Install Cubed with pip: + +```shell +pip install cubed +``` + +This installs a minimal set of dependencies for running Cubed, which is sufficient for the demo below. You can also install the `diagnostics` extra package, which is needed for later examples to provide things like progress bars and visualizations of the computation: + +```shell +pip install "cubed[diagnostics]" +``` + +Alternatively, you can install Cubed with Conda (note that this doesn't include the packages for diagnostics): + +```shell +conda install -c conda-forge cubed +``` + +## Demo + +First, we'll create a small array `a`: + +```python +import cubed +import cubed.array_api as xp +spec = cubed.Spec(work_dir="tmp", allowed_mem="100kB") +a = xp.asarray([[1, 2, 3], [4, 5, 6], [7, 8, 9]], chunks=(2, 2), spec=spec) +``` + +Cubed implements the [Python Array API standard](https://data-apis.org/array-api/latest/), which is essentially a subset of NumPy, and is imported as `xp` by convention. + +Notice that we also specify chunks, just like in Dask Array, and a {py:class}`Spec ` object that describes the resources available to run computations. + +Next we create another array `b` and add to two array together to get `c`. + +```python +b = xp.asarray([[1, 1, 1], [1, 1, 1], [1, 1, 1]], chunks=(2, 2), spec=spec) +c = xp.add(a, b) +``` + +Cubed uses lazy evaluation, so nothing has been computed yet. + +```python +c.compute() +``` + +This runs the computation using the (default) local Python executor and prints the result (if run interactively): + +``` +array([[ 2, 3, 4], + [ 5, 6, 7], + [ 8, 9, 10]]) +``` + +That's it! For your next step you can read the [user guide](user-guide/index.md), have a look at [configuration](configuration.md) options, or see more [examples](https://github.com/cubed-dev/cubed/blob/main/examples/README.md) to run locally or in the cloud. diff --git a/docs/getting-started/demo.md b/docs/getting-started/demo.md deleted file mode 100644 index 38c2369a2..000000000 --- a/docs/getting-started/demo.md +++ /dev/null @@ -1,35 +0,0 @@ -# Demo - -We'll start with a simple example that runs locally. - -```python -import cubed -import cubed.array_api as xp -spec = cubed.Spec(work_dir="tmp", allowed_mem="100kB") -a = xp.asarray([[1, 2, 3], [4, 5, 6], [7, 8, 9]], chunks=(2, 2), spec=spec) -``` - -Cubed implements the [Python Array API standard](https://data-apis.org/array-api/latest/), which is essentially a subset of NumPy, and is imported as `xp` by convention. - -Notice that we also specify chunks, just like in Dask Array, and a {py:class}`Spec ` object that describes the resources available to run the computation. - -```python -b = xp.asarray([[1, 1, 1], [1, 1, 1], [1, 1, 1]], chunks=(2, 2), spec=spec) -c = xp.add(a, b) -``` - -Cubed uses lazy evaluation, so nothing has been computed yet. - -```python -c.compute() -``` - -This runs the computation using the (default) local Python executor and prints the result (if run interactively): - -``` -array([[ 2, 3, 4], - [ 5, 6, 7], - [ 8, 9, 10]]) -``` - -See the [examples README](https://github.com/cubed-dev/cubed/blob/main/examples/README.md) for more examples that run on a single multi-core machine, or in the cloud. diff --git a/docs/getting-started/index.md b/docs/getting-started/index.md deleted file mode 100644 index 6776b7b83..000000000 --- a/docs/getting-started/index.md +++ /dev/null @@ -1,13 +0,0 @@ -# Getting Started - -This guide aims to get you started using Cubed as quickly as possible. - -Have a look at [Cubed: an introduction](https://cubed-dev.github.io/cubed/cubed-intro.slides.html) for a high-level overview. - -```{toctree} ---- -maxdepth: 2 ---- -installation -demo -``` diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md deleted file mode 100644 index 30ece875c..000000000 --- a/docs/getting-started/installation.md +++ /dev/null @@ -1,28 +0,0 @@ -# Installation - -## Conda - -You can install Cubed with a minimal set of dependencies using conda: - -```shell -conda install -c conda-forge cubed -``` - -## Pip - -You can also install Cubed with pip: - -```shell -python -m pip install cubed -``` - -## Optional dependencies - -Cubed has many optional dependencies, which can be installed in sets for different functionality (especially for running on different executors): - - $ python -m pip install "cubed[diagnostics]" # Install optional dependencies for cubed diagnostics - $ python -m pip install "cubed[beam]" # Install optional dependencies for the beam executor - $ python -m pip install "cubed[lithops]" # Install optional dependencies for the lithops executor - $ python -m pip install "cubed[modal]" # Install optional dependencies for the modal executor - -See the [examples](https://github.com/cubed-dev/cubed/blob/main/examples/README.md) for details on installing Cubed to run on different executors. diff --git a/docs/index.md b/docs/index.md index dbbf0c252..84a4101e0 100644 --- a/docs/index.md +++ b/docs/index.md @@ -29,9 +29,8 @@ Cubed is horizontally scalable and stateless, and can scale to multi-TB datasets :hidden: :maxdepth: 2 :caption: For users -getting-started/index +getting-started user-guide/index -Intro slides Examples api array-api diff --git a/docs/user-guide/diagnostics.md b/docs/user-guide/diagnostics.md index 22a8e82b7..095c538fc 100644 --- a/docs/user-guide/diagnostics.md +++ b/docs/user-guide/diagnostics.md @@ -5,7 +5,7 @@ Cubed provides a variety of tools to understand a computation before running it, To use these features ensure that the optional dependencies for diagnostics have been installed: ```shell -python -m pip install "cubed[diagnostics]" +pip install "cubed[diagnostics]" ``` ## Visualize the computation plan