Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UW-509 docs for sfc_climo_gen driver #412

Merged
merged 18 commits into from
Feb 23, 2024
1 change: 1 addition & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@
"rocoto": ("https://christopherwharrop.github.io/rocoto/%s", "%s"),
"rst": ("https://www.sphinx-doc.org/en/master/usage/restructuredtext/%s", "%s"),
"rtd": ("https://readthedocs.org/projects/uwtools/%s", "%s"),
"sfc-climo-gen": ("https://ufs-community.github.io/UFS_UTILS/sfc_climo_gen/%s", "%s"),
"ufs": ("https://ufscommunity.org/%s", "%s"),
"ufs-weather-model": ("https://github.com/ufs-community/ufs-weather-model/%s", "%s"),
"uwtools": ("https://github.com/ufs-community/uwtools/%s", "%s"),
Expand Down
1 change: 1 addition & 0 deletions docs/sections/user_guide/cli/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ CLI
mode_config
mode_fv3
mode_rocoto
mode_sfc_climo_gen
mode_template
30 changes: 15 additions & 15 deletions docs/sections/user_guide/cli/mode_fv3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,27 +17,27 @@ The ``uw`` mode for configuring and running FV3.
Positional arguments:
TASK
boundary_files
The FV3 lateral boundary-condition files
Lateral boundary-condition files
diag_table
The FV3 diag_table file
The diag_table file
field_table
The FV3 field_table file
The field_table file
files_copied
Files copied for FV3 run
Files copied for run
files_linked
Files linked for FV3 run
Files linked for run
model_configure
The FV3 model_configure file
The model_configure file
namelist_file
The FV3 namelist file
The namelist file
provisioned_run_directory
The run directory provisioned with all required content
Run directory provisioned with all required content
restart_directory
The FV3 RESTART directory
The RESTART directory
run
FV3 run execution
A run
runscript
A runscript suitable for submission to the scheduler
The runscript

All tasks take the same arguments. For example:

Expand All @@ -47,7 +47,7 @@ All tasks take the same arguments. For example:
usage: uw fv3 run --config-file PATH --cycle CYCLE [-h] [--batch] [--dry-run] [--debug] [--quiet]
[--verbose]

FV3 run execution
A run

Required arguments:
--config-file PATH, -c PATH
Expand Down Expand Up @@ -80,15 +80,15 @@ The examples use a configuration file named ``config.yaml``. Its contents are de

$ uw fv3 run --config-file config.yaml --cycle 2024-02-11T12

The driver creates a ``runscript`` file in the directory specified by ``run_dir`` in the config and runs it, executing FV3.
The driver creates a ``runscript`` file in the directory specified by ``run_dir:`` in the config and runs it, executing FV3.

* Run FV3 via a batch job

.. code-block:: text

$ uw fv3 run --config-file config.yaml --cycle 2024-02-11T12 --batch

The driver creates a ``runscript`` file in the directory specified by ``run_dir`` in the config and submits it to the batch system. Running with ``--batch`` requires a correctly configured ``platform`` block in ``config,yaml``, as well as appropriate settings in the ``execution`` block under ``fv3``.
The driver creates a ``runscript`` file in the directory specified by ``run_dir:`` in the config and submits it to the batch system. Running with ``--batch`` requires a correctly configured ``platform:`` block in ``config.yaml``, as well as appropriate settings in the ``execution:`` block under ``fv3:``.

* Specifying the ``--dry-run`` flag results in the driver logging messages about actions it would have taken, without actually taking any.

Expand All @@ -98,6 +98,6 @@ The examples use a configuration file named ``config.yaml``. Its contents are de

* The ``run`` task depends on the other available tasks and executes them as prerequisites. It is possible to execute any task directly, which entails execution of any of *its* dependencies. For example, to create an FV3 run directory provisioned with all the files, directories, symlinks, etc. required per the configuration file:

.. code-block: text
.. code-block:: text

$ uw fv3 provisioned_run_directory --config-file config.yaml --cycle 2024-02-11T12 --batch
93 changes: 93 additions & 0 deletions docs/sections/user_guide/cli/mode_sfc_climo_gen.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
Mode ``sfc_climo_gen``
======================

The ``uw`` mode for configuring and running the :sfc-climo-gen:`sfc_climo_gen<>` component.

.. code-block:: text

$ uw sfc_climo_gen --help
usage: uw sfc_climo_gen [-h] TASK ...

Execute sfc_climo_gen tasks

Optional arguments:
-h, --help
Show help and exit

Positional arguments:
TASK
namelist_file
The namelist file
provisioned_run_directory
Run directory provisioned with all required content
run
Run execution
runscript
The runscript

All tasks take the same arguments. For example:

.. code-block:: text

$ uw sfc_climo_gen run --help
usage: uw sfc_climo_gen run --config-file PATH [-h] [--batch] [--dry-run] [--debug] [--quiet]
[--verbose]

Run execution

Required arguments:
--config-file PATH, -c PATH
Path to config file

Optional arguments:
-h, --help
Show help and exit
--batch
Submit run to batch scheduler
--dry-run
Only log info, making no changes
--debug
Print all log messages, plus any unhandled exception's stack trace (implies --verbose)
--quiet, -q
Print no logging messages
--verbose, -v
Print all logging messages


Examples
^^^^^^^^

The examples use a configuration file named ``config.yaml`` with content similar to:

.. highlight:: yaml
.. literalinclude:: ../../../shared/sfc_climo_gen.yaml

Its contents are described in depth in section :ref:`sfc_climo_gen_yaml`.

* Run ``sfc_climo_gen`` on an interactive node

.. code-block:: text

$ uw sfc_climo_gen run --config-file config.yaml

The driver creates a ``runscript`` file in the directory specified by ``run_dir:`` in the config and runs it, executing ``sfc_climo_gen``.

* Run ``sfc_climo_gen`` via a batch job

.. code-block:: text

$ uw sfc_climo_gen run --config-file config.yaml --batch

The driver creates a ``runscript`` file in the directory specified by ``run_dir:`` in the config and submits it to the batch system. Running with ``--batch`` requires a correctly configured ``platform:`` block in ``config,yaml``, as well as appropriate settings in the ``execution:`` block under ``sfc_climo_gen:``.

* Specifying the ``--dry-run`` flag results in the driver logging messages about actions it would have taken, without actually taking any.

.. code-block:: text

$ uw sfc_climo_gen run --config-file config.yaml --batch --dry-run

* The ``run`` task depends on the other available tasks and executes them as prerequisites. It is possible to execute any task directly, which entails execution of any of *its* dependencies. For example, to create an ``sfc_climo_gen`` run directory provisioned with all the files, directories, symlinks, etc. required per the configuration file:

.. code-block:: text

$ uw sfc_climo_gen provisioned_run_directory --config-file config.yaml --batch
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
.. _fv3_yaml:

FV3 YAML
========
fv3
===

The structured YAML to run FV3 is described below. It is validated by JSON Schema. The configuration files required by the UFS Weather Model are documented :weather-model-io:`here<model-configuration-files>`.
Structured YAML to run FV3 is validated by JSON Schema and requires the ``fv3:`` block, described below. If FV3 is to be run via a batch system, the ``platform:`` block, described :ref:`here <platform_yaml>`, is also required. The configuration files required by the UFS Weather Model are documented :weather-model-io:`here<model-configuration-files>`.

The ``fv3:`` and ``platform:`` Blocks
-------------------------------------
The ``fv3:`` Block
------------------

Here are prototype UW YAML ``fv3:`` and ``platform:`` blocks, explained in detail below:
Here is a prototype UW YAML ``fv3:`` block, explained in detail below:

.. code-block:: yaml

Expand Down Expand Up @@ -57,51 +57,6 @@ Here are prototype UW YAML ``fv3:`` and ``platform:`` blocks, explained in detai
k_split: 2
n_split: 6
run_dir: /path/to/runs/{{ cycle.strftime('%Y%m%d%H') }}
platform:
account: user_account
scheduler: slurm

.. _updating_values:

Updating Values
---------------

Some blocks support ``base_file:`` and ``update_values:`` entries. A ``base_file:`` entry specifies the path to a file to use as a basis for a particular runtime file. An ``update_values:`` entry specifies changes/additions to the base file. At least one of ``base_file:`` or ``update_values:`` must be provided. If only ``base_file:`` is provided, the file will be used as-is. If only ``update_values:`` is provided, it will completely define the runtime file in question. If both are provided, ``update_values:`` is used to modify the contents of ``base_file:``. The hierarchy of entries in the ``update_values:`` block must mirror that in the ``base_file:``. For example, a ``base_file:`` named ``people.yaml`` might contain:

.. code-block:: yaml

person:
age: 19
address:
city: Boston
number: 12
state: MA
street: Acorn St
name: Jane

A compatible YAML block updating the person's street address might then contain:

.. code-block:: yaml

base_file: people.yaml
update_values:
person:
address:
street: Main St
number: 99

The result would be:

.. code-block:: yaml

person:
age: 19
address:
city: Boston
number: 99
state: MA
street: Main St
name: Jane

UW YAML for the ``fv3:`` Block
------------------------------
Expand All @@ -119,50 +74,7 @@ Accepted values are ``global`` and ``regional``.
execution:
^^^^^^^^^^

batchargs:
""""""""""

These entries map to job-scheduler directives sent to e.g. Slurm when a batch job is submitted via the ``--batch`` CLI switch or the ``batch=True`` API argument. The only **required** entry is ``walltime``.

Shorthand names are provided for certain directives for each scheduler, and can be specified as-so along with appropriate values. Recognized names for each scheduler are:

* LSF: ``jobname``, ``memory``, ``nodes``, ``queue``, ``shell``, ``stdout``, ``tasks_per_node``, ``threads``, ``walltime``
* PBS: ``debug``, ``jobname``, ``memory``, ``nodes``, ``queue``, ``shell``, ``stdout``, ``tasks_per_node``, ``threads``, ``walltime``
* Slurm: ``cores``, ``exclusive``, ``export``, ``jobname``, ``memory``, ``nodes``, ``partition``, ``queue``, ``rundir``, ``stderr``, ``stdout``, ``tasks_per_node``, ``threads``, ``walltime``

Other, arbitrary directive key-value pairs can be provided exactly as they should appear in the batch runscript. For example

.. code-block:: yaml

--nice: 100

could be specified to have the Slurm directive

.. code-block: text

#SBATCH --nice=100

included in the batch runscript.

executable:
"""""""""""

The name of or path to the FV3 executable binary.

mpiargs:
""""""""

An **array** of string arguments that should follow the MPI launch program (``mpiexec``, ``srun``, et al.) on the command line.

mpicmd:
"""""""

The MPI launch program (``mpiexec``, ``srun``, et al.)

threads:
""""""""

The number of OpenMP threads to use when running FV3.
See :ref:`here <execution_yaml>` for details.

field_table:
^^^^^^^^^^^^
Expand Down Expand Up @@ -213,34 +125,21 @@ path:
An absolute-path template to the lateral boundary condition files prepared for the forecast. The Python ``int`` variable ``forecast_hour`` will be interpolated into, e.g., ``/path/to/srw.t00z.gfs_bndy.tile7.f{forecast_hour:03d}.nc``. Note that this is a Python string template rather than a Jinja2 template.

length:
"""""""
^^^^^^^

The length of the forecast in integer hours.

model_configure:
""""""""""""""""
^^^^^^^^^^^^^^^^

Supports ``base_file:`` and ``update_values:`` blocks (see the :ref:`updating_values` for details). See FV3 ``model_configure`` documentation :weather-model-io:`here<model-configure-file>`.

namelist:
"""""""""
^^^^^^^^^

Supports ``base_file:`` and ``update_values:`` blocks (see the :ref:`updating_values` for details). See FV3 ``model_configure`` documentation :weather-model-io:`here<namelist-file-input-nml>`.

run_dir:
""""""""

The path to the directory where FV3 will find its inputs, configuration files, etc., and where it will write its output.

UW YAML for the ``platform:`` Block
-----------------------------------

account:
^^^^^^^^

The account name to use when requesting resources from the batch job scheduler.

scheduler:
^^^^^^^^^^

One of ``lsf``, ``pbs``, or ``slurm``.
The path to the directory where FV3 will find its inputs, configuration files, etc., and where it will write its output.
8 changes: 8 additions & 0 deletions docs/sections/user_guide/yaml/components/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
UW YAML for Components
======================

.. toctree::
:maxdepth: 1

fv3
sfc_climo_gen
31 changes: 31 additions & 0 deletions docs/sections/user_guide/yaml/components/sfc_climo_gen.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
.. _sfc_climo_gen_yaml:

sfc_climo_gen
=============

Structured YAML to run :sfc-climo-gen:`sfc_climo_gen<>` is validated by JSON Schema and requires the ``sfc_climo_gen:`` block, described below. If ``sfc_climo_gen`` is to be run via a batch system, the ``platform:`` block, described :ref:`here <platform_yaml>`, is also required.

Here is a prototype UW YAML ``sfc_climo_gen:`` block, explained in detail below:

.. highlight:: yaml
.. literalinclude:: ../../../../shared/sfc_climo_gen.yaml


UW YAML for the ``sfc_climo_gen:`` Block
----------------------------------------

execution:
^^^^^^^^^^

See :ref:`here <execution_yaml>` for details.


namelist:
^^^^^^^^^

Supports ``base_file:`` and ``update_values:`` blocks (see the :ref:`updating_values` for details). Namelist options are described `here <https://noaa-emcufs-utils.readthedocs.io/en/latest/ufs_utils.html#id57>`_.

run_dir:
^^^^^^^^

The path to the directory where ``sfc_climo_gen`` will find its namelist and write its outputs.
Loading
Loading