NOAA/NASA Domain Specific Language middleware

NDSL is a middleware for climate and weather modelling developed jointly by NOAA and NASA. The middleware brings together GT4Py (the cartesian flavor), ETH CSCS's stencil DSL, and DaCe, ETH SPCL's data flow framework, both developed for high-performance and portability. On top of those pillars, NDSL deploys a series of optimized APIs for common operations (Halo exchange, domain decomposition, MPI, ...), a set of bespoke optimizations for the models targeted by the middleware and tools to port existing models.

Batteries-included for FV-based models

Historically, NDSL was developed to port the FV3 dynamical core on the cubed-sphere. Therefore, the middleware ships with ready-to-execute specialization for models based on cubed-sphere grids and FV-based models in particular.

Quickstart

Currently, NDSL requires python (versions 3.11,3.12, and 3.13 are supported), a C/C++ compiler and MPI installed. All other dependencies installed during package installation. We recommend using a virtual (or conda) environment.

# We have submodules for GT4Py and DaCe. Don't forget to pull them
git clone --recurse-submodules [email protected]:NOAA-GFDL/NDSL.git

cd NDSL/

# We strongly recommend using a virtual environment (or conda)
python -m venv .venv/
source ./venv/bin/activate

# Choose pip install -e .[dev] if you'd like to contribute
pip install .[demos]

Now, checkout examples/NDSL and run through the Jupyter notebooks. Note that you have to install NDSL from GitHub, as it is not available on pypi.

The slightly longer version

NDSL is under active development and may only work with specific setups. This is what we know works for us.

Requirements and supported compilers

The run the CPU backends you will need:

• Python: 3.11, 3.12, 3.13
• CXX compiler: GNU 11.2+
• Libraries: MPI

To run the GPU backends, you'll need:

• Python: 3.11, 3.12, 3.13
• CXX compiler: GNU 11.2+
• Libraries: MPI compiled with CUDA support
• CUDA 11.2+
• Python package:
  • cupy (latest with proper driver support see install notes)

A simple way to install MPI is using pre-built wheels, e.g.

# See "quickstart" above how to setup a virtual environment
cd NDSL/
source ./venv/bin/activate

# Install MPI into your virtual environment
pip install openmpi

Porting workflows might depend on serialbox, e.g. the script ndsl-serialbox_to_netcdf depends on it. Serialbox is an optional dependency. If you install the serialbox extra, you'll need the Boost library and development headers.

Installation options

See quickstart above on how to pull and setup a virtual environment. The packages has a few options:

• ndsl[test]: extra dependencies to run tests (based on pytest)
• ndsl[demos]: extra dependencies to run NDSL examples
• ndsl[docs]: extra dependencies to build the docs
• ndsl[dev]: installs tools for development, docs, and tests.
• ndsl[serialbox]: installs serialbox, which is used in porting workflows
• ndsl[pyfms]: installs pyFMS and allows interaction with its diagnostics manager.

Running tests

Tests are available via pytest (don't forget to install the test or dev extras).

To run serial tests on CPU (GPU tests also run if cupy is available)

pytest -m "not parallel and not gpu and not zarr and not pyfms" tests/

To run parallel tests on CPU (GPU tests also run if cupy is available)

mpirun -np 6 pytest -m "parallel and not gpu" tests/

Development

Code/contribution guidelines

1. Code quality is enforced by pre-commit (which is part of the "dev" extra). Run pre-commit install to install the pre-commit hooks locally or make sure to run pre-commit run -a before submitting a pull request.
2. While we don't strictly enforce type hints, we add them on new code.
3. Pull requests have to merged as "squash merge" to keep the git history clean.

Documentation

Documentation is available online. We are using Zensical, which allows us to write the docs in Markdown files and serve it as a static site.

To view the documentation locally, install NDSL with the docs or dev extras. Then, run the following:

zensical serve

Contributing to the documentation is straight forward:

1. Add and/or change files in the docs/ folder as necessary.
2. [Optional] If you have changes to the navigation, modify zensical.toml.
3. [Optional] Start the development server and look how your changes are rendered.
4. Submit a pull request with your changes.

Points of contact

• NOAA: Rusty Benson: rusty.benson -at- noaa.gov
• NASA: Florian Deconinck florian.g.deconinck -at- nasa.gov