Skip to content

Commit

Permalink
Merge branch 'main' into chgres-cube-output
Browse files Browse the repository at this point in the history
Merge with main.
  • Loading branch information
Byrnetp committed Dec 12, 2024
2 parents c25ffef + 05dea64 commit bee331f
Show file tree
Hide file tree
Showing 70 changed files with 981 additions and 663 deletions.
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# See https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners

* @NaureenBharwaniNOAA @christinaholtNOAA @elcarpenterNOAA @fgabelmannjr @maddenp-noaa @weirae @Byrnetp
* @Byrnetp @NaureenBharwaniNOAA @christinaholtNOAA @elcarpenterNOAA @maddenp-noaa
2 changes: 1 addition & 1 deletion .github/scripts/install-conda.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ wget --no-verbose -O $installer $url
bash $installer -bfp $CI_CONDA_DIR
set +ux
ci_conda_activate
conda install --quiet --yes --channel maddenp --repodata-fn repodata.json anaconda-client condev
conda install --quiet --yes --channel maddenp --repodata-fn repodata.json anaconda-client condev jq
8 changes: 8 additions & 0 deletions .github/scripts/publish.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,11 @@ glob="$(jq -r .name $f)-$(jq -r .version $f)-*_$(jq -r .buildnum $f).tar.bz2"
for x in $(find $CI_CONDA_DIR/conda-bld -type f -name "$glob"); do
anaconda -t $ANACONDA_TOKEN upload $x
done
# Refresh Anaconda badges on GitHub.
ids=(
6c9ed105fbe1d674e82460d5e7fa6c7eb8e2eb6eb3640c763a8189f407e2a9a2/68747470733a2f2f616e61636f6e64612e6f72672f7566732d636f6d6d756e6974792f7577746f6f6c732f6261646765732f76657273696f6e2e737667
a6f1f3ab481647dc492ab577cb7e60522efded549caf0544ba863d0a72958179/68747470733a2f2f616e61636f6e64612e6f72672f7566732d636f6d6d756e6974792f7577746f6f6c732f6261646765732f6c61746573745f72656c656173655f646174652e737667
)
for id in ${ids[*]}; do
curl -s -X PURGE https://camo.githubusercontent.com/$id | jq .
done
4 changes: 2 additions & 2 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ name: Test
on:
pull_request:
branches:
- main
- '**'
push:
branches:
- main
- '**'
workflow_dispatch:
branches:
- '**'
Expand Down
2 changes: 1 addition & 1 deletion docs/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ dependencies:
- python >=3.9,<3.13 # keep in sync with meta.yaml run req
- sphinx_rtd_theme 3.0.*
- sphinxcontrib-bibtex 2.6.*
- tree
- tree 2.2.*
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/cdeps/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/esg_grid/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/fv3/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/ioda/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/jedi/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/make_hgrid/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/mpas/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/mpas_init/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/orog/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/orog_gsl/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/shave/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/ungrib/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/drivers/upp/run-help.out
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--schema-file PATH
Path to schema file to use for validation
--quiet, -q
Expand Down
56 changes: 28 additions & 28 deletions docs/sections/user_guide/cli/tools/config/realize-verbose.out
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
[2024-05-23T19:39:16] DEBUG Command: uw config realize --input-format yaml --output-format yaml --verbose
[2024-05-23T19:39:16] DEBUG Reading input from stdin
[2024-05-23T19:39:16] DEBUG Dereferencing, current value:
[2024-05-23T19:39:16] DEBUG hello: '{{ recipient }}'
[2024-05-23T19:39:16] DEBUG recipient: world
[2024-05-23T19:39:16] DEBUG [dereference] Rendering: {{ recipient }}
[2024-05-23T19:39:16] DEBUG [dereference] Rendered: world
[2024-05-23T19:39:16] DEBUG [dereference] Rendering: hello
[2024-05-23T19:39:16] DEBUG [dereference] Rendered: hello
[2024-05-23T19:39:16] DEBUG [dereference] Rendering: world
[2024-05-23T19:39:16] DEBUG [dereference] Rendered: world
[2024-05-23T19:39:16] DEBUG [dereference] Rendering: recipient
[2024-05-23T19:39:16] DEBUG [dereference] Rendered: recipient
[2024-05-23T19:39:16] DEBUG Dereferencing, current value:
[2024-05-23T19:39:16] DEBUG hello: world
[2024-05-23T19:39:16] DEBUG recipient: world
[2024-05-23T19:39:16] DEBUG [dereference] Rendering: world
[2024-05-23T19:39:16] DEBUG [dereference] Rendered: world
[2024-05-23T19:39:16] DEBUG [dereference] Rendering: hello
[2024-05-23T19:39:16] DEBUG [dereference] Rendered: hello
[2024-05-23T19:39:16] DEBUG [dereference] Rendering: world
[2024-05-23T19:39:16] DEBUG [dereference] Rendered: world
[2024-05-23T19:39:16] DEBUG [dereference] Rendering: recipient
[2024-05-23T19:39:16] DEBUG [dereference] Rendered: recipient
[2024-05-23T19:39:16] DEBUG Dereferencing, final value:
[2024-05-23T19:39:16] DEBUG hello: world
[2024-05-23T19:39:16] DEBUG recipient: world
[2024-05-23T19:39:16] DEBUG Writing output to stdout
[2024-11-27T05:24:34] DEBUG Command: uw config realize --input-format yaml --output-format yaml --verbose
[2024-11-27T05:24:34] DEBUG Reading input from stdin
[2024-11-27T05:24:34] DEBUG Dereferencing, current value:
[2024-11-27T05:24:34] DEBUG hello: '{{ recipient }}'
[2024-11-27T05:24:34] DEBUG recipient: world
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: hello
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: hello
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: {{ recipient }}
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: world
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: recipient
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: recipient
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: world
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: world
[2024-11-27T05:24:34] DEBUG Dereferencing, current value:
[2024-11-27T05:24:34] DEBUG hello: world
[2024-11-27T05:24:34] DEBUG recipient: world
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: hello
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: hello
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: world
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: world
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: recipient
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: recipient
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: world
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: world
[2024-11-27T05:24:34] DEBUG Dereferencing, final value:
[2024-11-27T05:24:34] DEBUG hello: world
[2024-11-27T05:24:34] DEBUG recipient: world
[2024-11-27T05:24:34] DEBUG Writing output to stdout
hello: world
recipient: world
42 changes: 21 additions & 21 deletions docs/sections/user_guide/cli/tools/config/validate-verbose.out
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
[2024-08-26T22:54:28] DEBUG Command: uw config validate --schema-file schema.jsonschema --input-file values.yaml --verbose
[2024-08-26T22:54:28] DEBUG Using schema file: schema.jsonschema
[2024-08-26T22:54:28] DEBUG Dereferencing, current value:
[2024-08-26T22:54:28] DEBUG values:
[2024-08-26T22:54:28] DEBUG greeting: Hello
[2024-08-26T22:54:28] DEBUG recipient: World
[2024-08-26T22:54:28] DEBUG [dereference] Rendering: Hello
[2024-08-26T22:54:28] DEBUG [dereference] Rendered: Hello
[2024-08-26T22:54:28] DEBUG [dereference] Rendering: greeting
[2024-08-26T22:54:28] DEBUG [dereference] Rendered: greeting
[2024-08-26T22:54:28] DEBUG [dereference] Rendering: World
[2024-08-26T22:54:28] DEBUG [dereference] Rendered: World
[2024-08-26T22:54:28] DEBUG [dereference] Rendering: recipient
[2024-08-26T22:54:28] DEBUG [dereference] Rendered: recipient
[2024-08-26T22:54:28] DEBUG [dereference] Rendering: values
[2024-08-26T22:54:28] DEBUG [dereference] Rendered: values
[2024-08-26T22:54:28] DEBUG Dereferencing, final value:
[2024-08-26T22:54:28] DEBUG values:
[2024-08-26T22:54:28] DEBUG greeting: Hello
[2024-08-26T22:54:28] DEBUG recipient: World
[2024-08-26T22:54:29] INFO 0 UW schema-validation errors found in config
[2024-11-27T05:24:34] DEBUG Command: uw config validate --schema-file schema.jsonschema --input-file values.yaml --verbose
[2024-11-27T05:24:34] DEBUG Using schema file: schema.jsonschema
[2024-11-27T05:24:34] DEBUG Dereferencing, current value:
[2024-11-27T05:24:34] DEBUG values:
[2024-11-27T05:24:34] DEBUG greeting: Hello
[2024-11-27T05:24:34] DEBUG recipient: World
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: values
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: values
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: greeting
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: greeting
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: Hello
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: Hello
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: recipient
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: recipient
[2024-11-27T05:24:34] DEBUG [dereference] Rendering: World
[2024-11-27T05:24:34] DEBUG [dereference] Rendered: World
[2024-11-27T05:24:34] DEBUG Dereferencing, final value:
[2024-11-27T05:24:34] DEBUG values:
[2024-11-27T05:24:34] DEBUG greeting: Hello
[2024-11-27T05:24:34] DEBUG recipient: World
[2024-11-27T05:24:34] INFO 0 UW schema-validation errors found in config
3 changes: 1 addition & 2 deletions docs/sections/user_guide/cli/tools/execute/help.out
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,7 @@ Optional arguments:
--graph-file PATH
Path to Graphviz DOT output [experimental]
--key-path KEY[.KEY...]
Dot-separated path of keys leading through the config to the driver's
configuration block
Dot-separated path of keys to driver config block
--quiet, -q
Print no logging messages
--verbose, -v
Expand Down
6 changes: 4 additions & 2 deletions docs/sections/user_guide/cli/tools/fs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ The ``uw`` mode for handling filesystem items (files and directories).

The ``copy`` action stages files in a target directory by copying files. Any ``KEY`` positional arguments are used to navigate, in the order given, from the top of the config to the :ref:`file block <files_yaml>`.

Source paths prefixed with ``http://`` or ``https://`` will be copied from their upstream network locations to the local filesystem.

.. literalinclude:: fs/copy-help.cmd
:emphasize-lines: 1
.. literalinclude:: fs/copy-help.out
Expand All @@ -23,7 +25,7 @@ The ``copy`` action stages files in a target directory by copying files. Any ``K
Examples
^^^^^^^^

Given ``copy-config.yaml`` containing
Given ``copy-config.yaml`` containing a mapping from local-filesystem destination paths to source paths

.. literalinclude:: fs/copy-config.yaml
:language: yaml
Expand All @@ -32,7 +34,7 @@ Given ``copy-config.yaml`` containing
.. literalinclude:: fs/copy-exec.out
:language: text

Here, ``foo`` and ``bar`` are copies of their respective source files.
Here, ``foo`` and ``bar`` are copies of their respective local-filesystem source files, and ``gpl`` is a copy of the upstream network source.

The ``--cycle`` and ``--leadtime`` options can be used to make Python ``datetime`` and ``timedelta`` objects, respectively, available for use in Jinja2 expression in the config. For example:

Expand Down
1 change: 1 addition & 0 deletions docs/sections/user_guide/cli/tools/fs/copy-config.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
config:
files:
foo: src/foo
licenses/gpl: https://www.gnu.org/licenses/gpl-3.0.txt
subdir/bar: src/bar
Original file line number Diff line number Diff line change
@@ -1 +1 @@
uw fs copy --config-file copy-config.yaml config files
uw fs copy --config-file copy-config.yaml --key-path config.files
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[2024-08-26T23:03:40] INFO Validating config against internal schema: files-to-stage
[2024-08-26T23:03:40] INFO 0 UW schema-validation errors found in fs config
[2024-08-26T23:03:40] ERROR Relative path 'foo' requires the target directory to be specified
[2024-12-07T01:01:51] INFO Validating config against internal schema: files-to-stage
[2024-12-07T01:01:53] INFO 0 UW schema-validation errors found in fs config
[2024-12-07T01:01:53] ERROR Relative path 'foo' requires target directory to be specified
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
rm -rf copy-dst-timedep
uw fs copy --target-dir copy-dst-timedep --config-file copy-config-timedep.yaml --cycle 2024-05-29T12 --leadtime 6 config files
uw fs copy --target-dir copy-dst-timedep --config-file copy-config-timedep.yaml --cycle 2024-05-29T12 --leadtime 6 --key-path config.files
echo
tree copy-dst-timedep
2 changes: 1 addition & 1 deletion docs/sections/user_guide/cli/tools/fs/copy-exec.cmd
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
rm -rf copy-dst
uw fs copy --target-dir copy-dst --config-file copy-config.yaml config files
uw fs copy --target-dir copy-dst --config-file copy-config.yaml --key-path config.files
echo
tree copy-dst
Loading

0 comments on commit bee331f

Please sign in to comment.