Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
c81621e
Initial python framework for pyjedi utilities. Initial ioda2df functi…
srherbener Jul 17, 2025
2caeeb4
Added buildObsSeqFromObsqDF function to the pyjedi package. This cons…
srherbener Jul 18, 2025
0f241d9
Merge remote-tracking branch 'ncar/python-packages' into feature/py-i…
srherbener Jul 22, 2025
7a990e2
WIP: ioda dateTime to obs_seq (seconds, days, time) conversion is wor…
srherbener Jul 22, 2025
e64d0d8
Merge remote-tracking branch 'ncar/python-packages' into feature/py-i…
srherbener Jul 23, 2025
ea95453
WIP: simple radiosonde example is working! Need to make the code more…
srherbener Jul 23, 2025
d5f4280
WIP: Simple radiosonde example is producing a file. First step is to …
srherbener Jul 23, 2025
30da652
Merge remote-tracking branch 'ncar/main' into feature/py-ioda-reader
srherbener Jul 24, 2025
4726f7a
Added YAML configuration, made ioda dataframe processing code more ge…
srherbener Jul 24, 2025
a9346cc
Put in dataframe filter that removes rows where the observation value…
srherbener Jul 24, 2025
4225e59
Added a simple test for the ioda2obsq converter
srherbener Jul 28, 2025
72a7afb
Updated documentation (README.md file) and added timestamp columns to…
srherbener Jul 28, 2025
e092cfd
Merge remote-tracking branch 'ncar/main' into feature/py-ioda-reader
srherbener Aug 5, 2025
c26c4cf
Repaired python packaging according to review comments.
srherbener Aug 5, 2025
1b935c5
Changed the YAML control "assim variables" to "observation variables".
srherbener Aug 5, 2025
e995ee3
Removed partial python unit test. (will add back in later)
srherbener Aug 7, 2025
817be57
Added documentation for ioda2obsq in the RTD guide.
srherbener Aug 7, 2025
6ba8a95
bump version and changelog for release
hkershaw-brown Aug 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ individual files.

The changes are now listed with the most recent at the top.

**August 13 2025 :: IODA2obsq. Tag v11.13.0**

- Initial pyjedi package for DART JEDI interoperability.
- ioda2obsq tool for conventional observations.

*contributed by Stephen Herbener, DART collaboration with JCSDA*

**July 30 2025 :: MOM6 interpolation bug-fixes. Tag v11.12.1**

- Fixes quad points assignment in model_interpolate
Expand Down
2 changes: 1 addition & 1 deletion conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
author = 'Data Assimilation Research Section'

# The full version, including alpha/beta/rc tags
release = '11.12.1'
release = '11.13.0'
root_doc = 'index'

# -- General configuration ---------------------------------------------------
Expand Down
70 changes: 67 additions & 3 deletions guide/pytools.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,28 +4,92 @@
DART Python tools
=========================

Installation
------------

DART provides a set of Python tools that can be used to work with DART data and models.
These tools are located in the DART/pytools directory.

We recommend creating a `virtual environment <https://docs.python.org/3/library/venv.html>`__
to install DART Python tools. This helps to keep the DART Python tools and their dependencies
isolated from other Python packages you may have installed.

.. code-block:: text
.. code-block:: bash

python3 -m venv py-dart
source py-dart/bin/activate

You can then install the DART Python tools by navigating to the DART/pytools directory and running:

.. code-block:: text
.. code-block:: bash

cd DART/pytools
pip install -r pyDART.txt


When you want to use the DART Python tools in a new session you will need to activate the virtual environment:

.. code-block:: bash

source py-dart/bin/activate

To deactivate the virtual environment when you are done, simply run:

.. code-block:: text

source py-dart/bin/activate
deactivate

Available Tool Packages
-----------------------

pyjedi
^^^^^^

The `pyjedi` package provides utilities for DART JEDI interoperability.
This is a new package that is under development and is not fully functional yet.
At this time, a new converter is available that can transform a JEDI IODA observation file into a DART observation sequence file.
This converter can be run as follows:

.. code-block:: bash

ioda2obsq <yamlConfigFile> <inputIodaFile> <outputObsqFile>

where ``<yamlConfigFile>`` is a YAML configuration file that specifies the mapping between JEDI IODA observation
variables and DART observation variables, ``<inputIodaFile>`` is the input JEDI IODA observation file,
and ``<outputObsqFile>`` is the output DART observation sequence file.

The YAML configuration file contains the mapping between JEDI IODA observation variables and DART observation types along
with the specification of the desired vertical coordinate variable.
Here is an example configuration file for radiosondes:

.. code-block:: yaml

---

ioda to obsq converter:
observation variables:
- name: airTemperature
type: RADIOSONDE_TEMPERATURE
- name: specificHumidity
type: RADIOSONDE_SPECIFIC_HUMIDITY
- name: windEastward
type: RADIOSONDE_U_WIND_COMPONENT
- name: windNorthward
type: RADIOSONDE_V_WIND_COMPONENT

vertical coordinate:
name: MetaData/pressure
units: "pressure (Pa)"

The `name:` entries are the names of the variables in the JEDI IODA observation file, and the `type:` entries
are the corresponding DART observation types.
The `vertical coordinate` section specifies the variable that will be used as the vertical coordinate in
the DART observation sequence file.
Note that in the `vertical coordinate` section, the `name:` entry must specify the full netcdf path to the variabile
in the JEDI IODA observation file, and the `units:` entry specifies the units to be used in the DART observation sequence file.

.. note::

The `ioda2obsq` tool is under active development and has limited functionality at this time.
It is expected that more features will be added soon, including support for satellite radiance observation types.
In its current state, it is primarily intended for use with radiosonde and similar conventional observation types.
1 change: 1 addition & 0 deletions pytools/pyDART.txt
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
-e ./pyqceff
-e ./pyjedi
52 changes: 52 additions & 0 deletions pytools/pyjedi/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# pyjedi

Python package for JEDI utilities.

## Installation

To install `pyjedi` in editable/development mode using pip:

```sh
pip install -e DART/pytools/pyjedi
```

where DART is the path to DART.

> **Note:** If you update DART with `git pull`, using the editable install (`pip install -e DART/pytools/pyjedi`) ensures your installed package always matches the latest source code.


## Usage

After installation, you can use the `ioda2obsq` script to convert a IODA observation file into a DART obs_seq.out file:

```sh
ioda2obsq yamlConfigFile inputIodaFile outputObsqFile
```

To view the help
```sh
ioda2obsq -h
```

The yamConfigFile is used to tell ioda2obsq which variables, and their type, from the input IODA file to convert, as well as define the vertical coordinate.
Here is a sample from the radiosonde test case:
```yaml
---

ioda to obsq converter:
observation variables:
- name: airTemperature
type: RADIOSONDE_TEMPERATURE
- name: specificHumidity
type: RADIOSONDE_SPECIFIC_HUMIDITY
- name: windEastward
type: RADIOSONDE_U_WIND_COMPONENT
- name: windNorthward
type: RADIOSONDE_V_WIND_COMPONENT

vertical coordinate:
name: MetaData/height
units: "height (m)"
```

Note "observation variables" is the list of variables you intend to assimilate in the DA job.
33 changes: 33 additions & 0 deletions pytools/pyjedi/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "pyjedi"
version = "0.1.0"
description = "Python package for JEDI utilities."
authors = [
{ name = "DART", email = "[email protected]" }
]
readme = "README.md"
requires-python = ">=3.8"
license = { text = "Apache-2.0" }
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: Apache Software License",
"Operating System :: OS Independent",
]
dependencies = [
"numpy",
"pandas>=2.2.0",
"PyYAML",
"netCDF4",
"pydartdiags>=0.6.1",
]

[project.scripts]
ioda2obsq = "pyjedi.ioda2obsq:main"

[project.urls]
Homepage = "https://github.com/NCAR/DART"
Repository = "https://github.com/NCAR/DART"
Empty file.
Loading