diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 170d34a4..80002c02 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -9,7 +9,7 @@ repos: # hooks: # - id: markdownlint-fix - repo: https://github.com/executablebooks/mdformat - rev: 0.7.18 # Use the ref you want to point at + rev: 0.7.21 # Use the ref you want to point at hooks: - id: mdformat # Optionally add plugins diff --git a/book/explanation/architecture.md b/book/explanation/architecture.md index 4b3d5b04..02544fff 100644 --- a/book/explanation/architecture.md +++ b/book/explanation/architecture.md @@ -174,7 +174,7 @@ It’s useful to understand the recent history and related efforts in this space - **[Google Colab](https://research.google.com/colaboratory/faq.html)** is a free notebook-in-the-cloud service run by Google. It is built around the open source Jupyter project, but with advanced notebook sharing capabilities (like Google Docs). - **[Google Earth Engine](https://earthengine.google.org/)** is a reference point for all cloud geospatial analytics platforms. - It’s actually a standalone application that is separate from Google Cloud, the single instance of a highly customized, black box (i.e. not open source) application that enables parallel computing on distributed data. + It’s actually a standalone application that is separate from Google Cloud, the single instance of a highly customized, black box (i.e. not open source) application that enables parallel computing on distributed data. It’s very good at what it was designed for (analyzing satellite images), but isn’t easily adapted to other applications, such as machine learning. - **[Columbia IRI Data Library](https://iridl.ldeo.columbia.edu/index.html)** is a powerful and freely accessible online data repository and analysis tool that allows a user to view, analyze, and download hundreds of terabytes of climate-related data through a standard web browser. Due to its somewhat outdated architecture, IRI data library cannot easily be updated or adapted to new projects. diff --git a/book/explanation/data_policy.md b/book/explanation/data_policy.md index 622b06ed..c359a123 100644 --- a/book/explanation/data_policy.md +++ b/book/explanation/data_policy.md @@ -37,7 +37,7 @@ Data that is already publically available but has been ingested into cloud stora :::\{admonition} LEAP curated :class: dropdown -Data that is already available in {abbr}`ARCO` formats in publically accessible object storage. Adding this data to the LEAP-Pangeo Catalog enables us to visualize it with the Data Viewer, and collect all datasets of importance in one single location, but none of the data itself is modified. +Data that is already available in {abbr}`ARCO` formats in publically accessible object storage. Adding this data to the LEAP-Pangeo Catalog enables us to visualize it with the Data Viewer, and collect all datasets of importance in one single location, but none of the data itself is modified. ::: ## Roles diff --git a/book/guides/faq.md b/book/guides/faq.md index c35d9002..df803a0e 100644 --- a/book/guides/faq.md +++ b/book/guides/faq.md @@ -24,20 +24,20 @@ You can follow that and accept the invitation there aswell. If you are unable to log into the hub, please check the following steps: -- \[ \] Check if you are member of the [appropriate github teams](reference.membership.tiers). +- [ ] Check if you are member of the [appropriate github teams](reference.membership.tiers). If you **are not** follow these steps: -- \[ \] Did you [sign up for LEAP membership](users.membership.apply)? This will be done for you if you sign up for an event like the Momentum bootcamp! -- \[ \] Did you receive a github invite? [Here](faq.where_is_my_invite) is how to check for that. -- \[ \] Check again if they are part of the [appropriate github teams](reference.membership.tiers). +- [ ] Did you [sign up for LEAP membership](users.membership.apply)? This will be done for you if you sign up for an event like the Momentum bootcamp! +- [ ] Did you receive a github invite? [Here](faq.where_is_my_invite) is how to check for that. +- [ ] Check again if they are part of the [appropriate github teams](reference.membership.tiers). - If these steps do not work, please reach out to the [](support.data_compute_team). If you **are** member of one of the github teams, ask them to try the following steps: -- \[ \] Refresh the browser cache -- \[ \] Try a different browser -- \[ \] Restart the computer +- [ ] Refresh the browser cache +- [ ] Try a different browser +- [ ] Restart the computer - If these steps do not work, please reach out to the [](support.data_compute_team). (faq.usr_dir_usage_warning)= @@ -48,11 +48,11 @@ If you get a Hub Usage Alert email, this means you are violating the User Direct **Troubleshooting** -- To see which files and directories are taking up the bulk of your storage, run `du -h --max-depth=1 ~/ | sort -h` in Terminal. It will likely reveal cached files and small/medium size data files that can be removed without disrupting typical usage. +- To see which files and directories are taking up the bulk of your storage, run `du -h --max-depth=1 ~/ | sort -h` in Terminal. It will likely reveal cached files and small/medium size data files that can be removed without disrupting typical usage. - Delete cached files, ipython checkpoints, and any other unwanted files. - If you still require more storage, it is likely that you are storing downloaded data in your user directory. We recommend storing data in a LEAP [cloud bucket](https://leap-stc.github.io/leap-pangeo/jupyterhub.html#leap-pangeo-cloud-storage-buckets) or data catalog. For more information, please consult our [](guide.data) and [](explanation.data-policy). -Our goal is to accomodate all community members and thus we are happy to assist users in relocating data. If you have any concerns, please reach out to the [](support.data_compute_team). +Our goal is to accomodate all community members and thus we are happy to assist users in relocating data. If you have any concerns, please reach out to the [](support.data_compute_team). ## Dask "Killed Workers" diff --git a/book/guides/team_guide.md b/book/guides/team_guide.md index bc68b5a4..248951f3 100644 --- a/book/guides/team_guide.md +++ b/book/guides/team_guide.md @@ -6,13 +6,13 @@ This is a short write up facilitate the spin up of new team members for the Data ### Checklist for new members -- \[ \] Ask to be added to the [Data and Computation Github team](https://github.com/orgs/leap-stc/teams/data-team/members) -- \[ \] Ask to be added to the `@data-and-compute` Slack user group -- \[ \] Subscribe to [](onboarding.slack) -- \[ \] Consider enabling notifications for [](onboarding.github) -- \[ \] Make a PR to the `_config.yaml` file [here](https://github.com/leap-stc/leap-stc.github.io/blob/fd69890ffc2f1871968e39b1c460370a0b3f98b3/book/_config.yml#L40-L51) in a PR. to add a picture and your personal data to the webpage. -- \[ \] Get access to the [Grafana Dashboard](https://grafana.leap.2i2c.cloud) -- \[ \] Request access to a service account to monitor Google Dataflow and Storage from the [Google Cloud Console](https://console.cloud.google.com/welcome?project=leap-pangeo) by raising an issue [here](https://github.com/leap-stc/data-and-compute-team/issues) +- [ ] Ask to be added to the [Data and Computation Github team](https://github.com/orgs/leap-stc/teams/data-team/members) +- [ ] Ask to be added to the `@data-and-compute` Slack user group +- [ ] Subscribe to [](onboarding.slack) +- [ ] Consider enabling notifications for [](onboarding.github) +- [ ] Make a PR to the `_config.yaml` file [here](https://github.com/leap-stc/leap-stc.github.io/blob/fd69890ffc2f1871968e39b1c460370a0b3f98b3/book/_config.yml#L40-L51) in a PR. to add a picture and your personal data to the webpage. +- [ ] Get access to the [Grafana Dashboard](https://grafana.leap.2i2c.cloud) +- [ ] Request access to a service account to monitor Google Dataflow and Storage from the [Google Cloud Console](https://console.cloud.google.com/welcome?project=leap-pangeo) by raising an issue [here](https://github.com/leap-stc/data-and-compute-team/issues) - Instructions for admin: - Go to the Google Cloud Console > IAM > Grant Access - Add the following permissions: @@ -59,7 +59,7 @@ We aim to provide users with [up-to-date default software environments](referenc ## Offboarding members -- \[\] Delete personal `dct-team-` service account in IAM (needs admin priviliges). +- [] Delete personal `dct-team-` service account in IAM (needs admin priviliges). ## Admin Tasks diff --git a/book/reference/infrastructure.md b/book/reference/infrastructure.md index 3c509ee0..c4ea3347 100644 --- a/book/reference/infrastructure.md +++ b/book/reference/infrastructure.md @@ -57,7 +57,7 @@ As shown in the picture above, every user will see `'/home/jovyan'` as their roo The primary purpose of this directory is to store small files, like github repositories and other code. :::\{warning} -To accommodate the expanding LEAP community, the data and compute team has instituted a storage quota on individual user directories `/home/jovyan`. Your home directory is intended only for notebooks, analysis scripts, and small datasets (\< 1 GB). It is not an appropriate place to store large datasets. Unlike the cloud buckets, these directories use an underlying storage with a rigid limit. If a single user fills up the space, the Hub crashes for everyone. We recommend users use less than 25GB and enforce a hard limit of 50GB. **Users who persistently violate the limit may temporarily get reduced cloud access**. +To accommodate the expanding LEAP community, the data and compute team has instituted a storage quota on individual user directories `/home/jovyan`. Your home directory is intended only for notebooks, analysis scripts, and small datasets (< 1 GB). It is not an appropriate place to store large datasets. Unlike the cloud buckets, these directories use an underlying storage with a rigid limit. If a single user fills up the space, the Hub crashes for everyone. We recommend users use less than 25GB and enforce a hard limit of 50GB. **Users who persistently violate the limit may temporarily get reduced cloud access**. To check how much space you are using in your home directory open a terminal window on the hub and run `du -h --max-depth=1 ~/ | sort -h`. diff --git a/book/reference/membership.md b/book/reference/membership.md index 935a5a48..5353e57b 100644 --- a/book/reference/membership.md +++ b/book/reference/membership.md @@ -14,8 +14,8 @@ All users of LEAP-Pangeo must abide by the [LEAP Code of Conduct](https://leap.c The membership tiers are listed in ascending order of access/privileges. -- **PUBLIC MEMBERSHIP(Please note: Public Membership will be live soon.)** Receive weekly LEAP newsletter and invitations to LEAP events. Access to LEAP Pangeo that includes limited performance notebooks, read only access to all LEAP storage, and write access to our scratch bucket, where everything is deleted after 7 days. -- **EDUCATION MEMBERSHIP** Open to participants in LEAP Education programs (e.g., Bootcamp, academic courses, and summer programs). Receive weekly LEAP newsletter and invitations to LEAP events. Access to LEAP Pangeo compute and storage for duration of education program. +- **PUBLIC MEMBERSHIP(Please note: Public Membership will be live soon.)** Receive weekly LEAP newsletter and invitations to LEAP events. Access to LEAP Pangeo that includes limited performance notebooks, read only access to all LEAP storage, and write access to our scratch bucket, where everything is deleted after 7 days. +- **EDUCATION MEMBERSHIP** Open to participants in LEAP Education programs (e.g., Bootcamp, academic courses, and summer programs). Receive weekly LEAP newsletter and invitations to LEAP events. Access to LEAP Pangeo compute and storage for duration of education program. - **RESEARCH MEMBERSHIP** Open to researchers who have been referred by a LEAP scientist. Receive weekly LEAP newsletter and invitations to LEAP events. Access to LEAP Pangeo compute and storage for six (6) months (renewal applications may be submitted at the conclusion of six months). - **LEAP-FUNDED RESEARCHE MEMBERSHIP**. Open to researchers who receive LEAP funding. Receive weekly LEAP newsletter and invitations to LEAP events. Access to LEAP Pangeo compute and storage for duration of time with LEAP. diff --git a/book/tutorials/getting_started.md b/book/tutorials/getting_started.md index d94b54e6..e8020ddd 100644 --- a/book/tutorials/getting_started.md +++ b/book/tutorials/getting_started.md @@ -25,7 +25,7 @@ Feel free to [edit it yourself](https://github.com/leap-stc/leap-stc.github.io/b 1. 👀 Navigate to and click the big orange button that says "Log in to continue" 1. 🔐 You will be prompted to authorize a GitHub application. Say "yes" to everything. Note you must belong to the appropriate GitHub team in order to access the hub. - See [](reference.membership.team-resources) for more information. + See [](reference.membership.team-resources) for more information. 1. 📠 You will redirect to a screen with the following options. image