-
Notifications
You must be signed in to change notification settings - Fork 47
spack‐stack‐1.9.1 release documentation (placeholder)
This is the documentation at the time the release was rolled out. For post-release updates, see here.
Ready-to-use spack-stack 1.9.1 installations are available on the following, fully supported platforms. This version supports JEDI-Skylab, various UFS and related applications (UFS Weather Model, EMC Global Workflow, GSI, UFS Short Range Weather Application), and several NEPTUNE applications.
In column Compilers, GCC refers to combinations of gcc/g++/gfortran, Intel refers to icc/icpc/ifort, oneAPI refers to icx/icpx/ifort, and oneAPI-ifx refers to icx/icpx/ifx.
| Organization | System | Compilers | Maintainers |
|---|---|---|---|
| HPC platforms | |||
| MSU | Hercules | TBD | EPIC / JCSDA |
| MSU | Orion | TBD | EPIC / JCSDA |
| NASA | Discover SCU17 | GCC, oneAPI, intel | JCSDA |
| NCAR-Wyoming | Derecho | TBD | EPIC / JCSDA |
| NOAA (NCEP) | Acorn | TBD | NOAA-EMC |
| NOAA (RDHPCS) | Gaea C5 | TBD | EPIC / NOAA-EMC |
| NOAA (RDHPCS) | Gaea C6 | TBD | EPIC / NOAA-EMC |
| NOAA (RDHPCS) | Hera | TBD | EPIC / NOAA-EMC |
| NOAA (RDHPCS) | Jet | TBD | EPIC / NOAA-EMC |
| U.S. Navy (HPCMP) | Narwhal | GCC, oneAPI | NRL |
| U.S. Navy (HPCMP) | Nautilus | GCC, oneAPI, oneAPI-ifx | NRL |
| Univ. of Wisconsin | S4 | TBD | JCSDA |
| Cloud platforms | |||
| Amazon Web Services | AMI Red Hat 8 | TBD | JCSDA |
| Amazon Web Services | Parallelcluster JCSDA | TBD | JCSDA |
| NOAA (RDHPCS) | RDHPCS Parallel Works | TBD | EPIC / JCSDA |
To use one of the above installations via the system default environment module system, follow the instructions in the individual sections below. For questions or problems, please consult the known issues, the currently open GitHub issues, and the GitHub discussions first.
The following is a list of supplemental or "add-on" environments that are maintained through spack-stack. These environments are provided on a subset of the tier 1 platforms. To use them, replace ue-COMPILER-VERSION in the instructions below with the add-on environment name.
| Environment name | Description | Platforms |
|---|---|---|
| ce-* | Supports CYLC workflow (standalone environment) | Nautilus, Narwhal |
| gsi-addon-* | Supports GSI and related applications | |
| ne-* | Supports NRL's NEPTUNE atmospheric model with xNRL | Nautilus, Narwhal |
| ufswm-* | Supports UFS Weather Model with WCOSS2 package versions |
For general instructions for installing add-on environments that make use of the installations available on tier-1 platforms, see https://spack-stack.readthedocs.io/en/latest/AddingTestPackages.html.
The CYLC standalone environment is not a chained environment and is installed independently from the standard unified environment. To install this environment, simply replace unified-dev in the spack stack create env call with cylc-dev, and use gcc (this environment will not compile with any other compiler at this time). For example:
spack stack create env --name=ce-gcc-11.2.1 --site=nautilus --compiler=gcc --template=cylc-dev
There are important differences in using the CYLC environment compared to all the other environments: For this environment, we do not use the spack-stack generated modulefiles, but instead take advantage of a spack environment view. This view is generated in /full/path/to/ce-COMPILER-VERSION/view. The recommended way of using cylc and encapsulate it entirely from any environment loaded in the cylc tasks is to generate a cylc-wrapper and either call the wrapper directly or alias it with alias cylc=/path/to/cylc-wrapper. An example cylc-wrapper is provided here, where it is assumed that CYLC_INSTALL_PREFIX points to the environment view.
#!/bin/bash
[ -z ${CYLC_INSTALL_PREFIX} ] && echo "Environment variable CYLC_INSTALL_PREFIX not set!" && exit 1
PATH=${CYLC_INSTALL_PREFIX}/bin:$PATH
cylc "$@"
The add-on environment for GSI is maintained under the gsi-addon-dev template.
The NEPTUNE standalone environment is not a chained environment and is installed independently from the standard unified environment. To install this environment, simply replace unified-dev in the spack stack create env call with neptune-dev, for example:
spack stack create env --name=ne-oneapi-2024.2.1 --site=nautilus --compiler=oneapi --template=neptune-dev
The ufs-weather-model-static template is installed on several systems as a standalone (non-chained) environment that attempts to follow the package versions available on WCOSS2.
NEEDS UPDATING
The following is required for using spack to build and run software with any of the compilers below.
module purge
For Intel (classic):
module use /apps/contrib/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.9.0/install/modulefiles/Core
module load stack-intel/2021.9.0
module load stack-intel-oneapi-mpi/2021.9.0
module load stack-python/3.11.7
# Needed to ensure the spack-stack built udunits package gets loaded
# instead of the spack-managed udunits package.
export LMOD_TMOD_FIND_FIRST=yes
module use /apps/contrib/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.9.0/install/modulefiles/intel/2021.9.0
For GNU:
module use /apps/contrib/spack-stack/spack-stack-1.8.0/envs/ue-gcc-12.2.0/install/modulefiles/Core
module load stack-gcc/12.2.0
module load stack-openmpi/4.1.4
module load stack-python/3.11.7
# Needed to ensure the spack-stack built udunits package gets loaded
# instead of the spack-managed udunits package.
export LMOD_TMOD_FIND_FIRST=yes
module use /apps/contrib/spack-stack/spack-stack-1.8.0/envs/ue-gcc-12.2.0/install/modulefiles/gcc/12.2.0
NEEDS UPDATING
The following is required for using spack to build and run software with any of the compilers below.
module purge
For Intel (classic):
module use /apps/contrib/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.9.0/install/modulefiles/Core
module load stack-intel/2021.9.0
module load stack-intel-oneapi-mpi/2021.9.0
module load stack-python/3.11.7
For GNU:
module use /apps/contrib/spack-stack/spack-stack-1.8.0/envs/ue-gcc-12.2.0/install/modulefiles/Core
module load stack-gcc/12.2.0
module load stack-openmpi/4.1.4
module load stack-python/3.11.7
The following is required for using spack to build and run software with any of the compilers below.
module purge
module use /discover/swdev/gmao_SIteam/modulefiles-SLES15
module use /discover/swdev/jcsda/spack-stack/scu17/modulefiles
For Intel (classic):
module use /gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-1.9.0/envs/ue-intel-2021.10.0/install/modulefiles/Core
module load stack-intel/2021.10.0
module load stack-intel-oneapi-mpi/2021.10.0
module load stack-python/3.11.7
For GNU:
module use /gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-1.9.0/envs/ue-gcc-12.3.0/install/modulefiles/Core
module load stack-gcc/12.3.0
module load stack-openmpi/4.1.6
module load stack-python/3.11.7
For Intel oneAPI (icx, icpx, ifort):
module use /gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-1.9.0/envs/ue-oneapi-2024.2.0/install/modulefiles/Core
module load stack-oneapi/2024.2.0
module load stack-intel-oneapi-mpi/2021.13
module load stack-python/3.11.7
The following is required for using spack to build and run software with Intel oneAPI. Don't use module purge on Narwhal!
module unload PrgEnv-cray
module load PrgEnv-intel/8.4.0
module unload intel
module load intel/2024.2
module unload craype-network-ofi
module load craype-network-ucx
module unload cray-mpich
module load cray-mpich-ucx/8.1.26
module load libfabric/1.12.1.2.2.1
module unload cray-libsci
module load cray-libsci/23.05.1.4
prepend-path MODULEPATH /p/app/projects/NEPTUNE/spack-stack/spack-stack-1.9.1/envs/ue-oneapi-2024.2.0/install/modulefiles/Core
module load stack-oneapi/2024.2.0
module load stack-cray-mpich/8.1.26
module load stack-python/3.11.7
For the NEPTUNE standalone environment (ne-oneapi-2024.2.0), substitute ue-oneapi-2024.2.0 with ne-oneapi-2024.2.0.
The following is required for using spack to build and run software with GNU. Don't use module purge on Narwhal!
module unload PrgEnv-cray
module load PrgEnv-gnu/8.4.0
module unload gcc
module load gcc/10.3.0
module unload craype-network-ofi
module load craype-network-ucx
module unload cray-mpich
module load cray-mpich-ucx/8.1.26
module load libfabric/1.12.1.2.2.1
module unload cray-libsci
module load cray-libsci/23.05.1.4
module use /p/app/projects/NEPTUNE/spack-stack/spack-stack-1.9.1/envs/ue-gcc-10.3.0/install/modulefiles/Core
module load stack-gcc/10.3.0
module load stack-cray-mpich/8.1.26
module load stack-python/3.11.7
For the NEPTUNE standalone environment (ne-gcc-10.3.0), substitute ue-gcc-10.3.0 with ne-gcc-10.3.0.
A CYLC standalone environment is available with gcc-10.3.0 and can be used as described earlier on this page. The environment view is:
/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.9.1/envs/ce-gcc-10.3.0/view
The following is required for using spack to build and run software with any of the compilers below.
module purge
For Intel oneAPI (icx, icpx, ifort):
module use /p/app/projects/NEPTUNE/spack-stack/oneapi-2024.2.1/modulefiles
module use /p/app/projects/NEPTUNE/spack-stack/spack-stack-1.9.1/envs/ue-oneapi-2024.2.1/install/modulefiles/Core
module load stack-oneapi/2024.2.1
module load stack-intel-oneapi-mpi/2021.13
module load stack-python/3.11.7
For the NEPTUNE standalone environment (ne-oneapi-2024.2.1), substitute ue-oneapi-2024.2.1 with ne-oneapi-2024.2.1.
For GNU:
module use /p/app/projects/NEPTUNE/spack-stack/spack-stack-1.9.1/envs/ue-gcc-11.2.1/install/modulefiles/Core
module load stack-gcc/11.2.1
module load stack-openmpi/5.0.1
module load stack-python/3.11.7
For the NEPTUNE standalone environment (ne-gcc-11.2.1), substitute ue-gcc-11.2.1 with ne-gcc-11.2.1.
For Intel oneAPI with ifx, only the NEPTUNE standalone environment is available:
module use /p/app/projects/NEPTUNE/spack-stack/oneapi-2025.0.0/modulefiles
module use /p/app/projects/NEPTUNE/spack-stack/spack-stack-1.9.1/envs/ne-oneapi-2025.0.0/install/modulefiles/Core
module load stack-oneapi/2025.0.0
module load stack-intel-oneapi-mpi/2021.14
module load stack-python/3.11.7
A CYLC standalone environment is available with gcc-11.2.1 and can be used as described earlier on this page. The environment view is:
/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.9.1/envs/ce-gcc-11.2.1/view
NEEDS UPDATING
The following is required for using spack to build and run software with any of the compilers below.
module purge
# ignore that the sticky module ncarenv/... is not unloaded
export LMOD_TMOD_FIND_FIRST=yes
module load ncarenv/23.09
module use /glade/work/epicufsrt/contrib/spack-stack/derecho/modulefiles
For Intel (classic):
module use /glade/work/epicufsrt/contrib/spack-stack/derecho/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install/modulefiles/Core
module load stack-intel/2021.10.0
module load stack-cray-mpich/8.1.25
module load stack-python/3.11.7
For GNU:
module use /glade/work/epicufsrt/contrib/spack-stack/derecho/spack-stack-1.8.0/envs/ue-gcc-12.2.0/install/modulefiles/Core
module load stack-gcc/12.2.0
module load stack-cray-mpich/8.1.25
module load stack-python/3.11.7
NEEDS UPDATING
The following is required for using spack to build and run software with the compiler below.
module purge
For Intel (classic):
module use /contrib/spack-stack-rocky8/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install/modulefiles/Core
module load stack-intel/2021.10.0
module load stack-intel-oneapi-mpi/2021.10.0
module load stack-python/3.11.7
NEEDS UPDATING
The following is required for using spack to build and run software with the compiler below. Don't use module purge on Gaea - C5!
For Intel classic:
module use /ncrc/proj/epic/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install/modulefiles/Core
module load stack-intel/2023.2.0
module load stack-cray-mpich/8.1.28
module load stack-python/3.11.7
NEEDS UPDATING
The following is required for using spack to build and run software with the compiler below. Don't use module purge on Gaea - C6!
For Intel classic:
module use /ncrc/proj/epic/spack-stack/c6/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install/modulefiles/Core
module load stack-intel/2023.2.0
module load stack-cray-mpich/8.1.29
module load stack-python/3.11.7
NEEDS UPDATING
The following is required for using spack to build and run software with any of the compilers below.
module purge
For Intel (classic):
module use /contrib/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.5.0/install/modulefiles/Core
module load stack-intel/2021.5.0
module load stack-intel-oneapi-mpi/2021.5.1
module load stack-python/3.11.7
For GNU:
module use /contrib/spack-stack/spack-stack-1.8.0/envs/ue-gcc-9.2.0/install/modulefiles/Core
module load stack-gcc/9.2.0
module load stack-openmpi/4.1.6
module load stack-python/3.11.7
NEEDS UPDATING
The following is required for using spack to build and run software with any of the compilers below.
module purge
For Intel (classic):
module use /contrib/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.5.0/install/modulefiles/Core
module load stack-intel/2021.5.0
module load stack-intel-oneapi-mpi/2021.5.1
module load stack-python/3.11.7
For GNU:
module use /contrib/spack-stack/spack-stack-1.8.0/envs/ue-gcc-9.2.0/install/modulefiles/Core
module load stack-gcc/9.2.0
module load stack-openmpi/4.1.6
module load stack-python/3.11.7
NEEDS UPDATING
When installing packages with spack-stack, to avoid a bug with Spack resource caching, avoid the system Python (3.6) and use:
module load gcc/10.3.0 python/3.11.7
before running setup.sh ($SPACK_PYTHON should point to a Python executable other than the one in /usr/bin).
NEEDS UPDATING
The following is required for using spack to build and run software with any of the compilers below. For Intel (classic):
module purge
module use /data/prod/jedi/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install/modulefiles/Core
module load stack-intel/2021.10.0
module load stack-intel-oneapi-mpi/2021.10.0
module load stack-python/3.11.7
MISSING
MISSING
None at this time.