I built the converters with following commands:

1. Load the JEDI Intel environment with derecho_setup_intel.sh

2. Clone DART

git clone https://github.com/NCAR/DART DART

3. Check out the feature/py-ioda-reader branch from the fork:

git remote add forked https://github.com/srherbener/DART.git
git fetch forked
git checkout feature/py-ioda-reader

4. Create a virtual environment (per DART readthedocs):

  cd DART
  python3 -m venv py-dart
  source py-dart/bin/activate
  cd pytools
  pip install -r pyDART.txt

5. Go to the pytools/pyjedi/tests directory

cd pytools/pyjedi/tests

6. Convert the NCAR RDA prepbufr /glade/campaign/collections/rda/data/d337000/prepnr/2021/prepbufr.gdas.2021021600.nr file for 2021/02/16 00Z into IODA using the attached convert_bufr.sh script.

7. Convert the IODA radiosonde files into an obs sequence file with the command:

ioda2obsq obsq.radiosonde.yaml ioda.radiosonde_obs_2021021600.nc4 obs_seq.radiosonde_obs_2021021600.out

convert_bufr.txt

This is really nice Steve, I've been running it on various conventional obs.
ioda.aircraft_obs_2021021600.nc4, ioda.ascat_obs_2021021600.nc4, ioda.radiosonde_obs_2021021600.nc4 ioda.satwind_obs_2021021600.nc4

@srherbener @fcvdb
I've got couple of questions on the MetaData. Looking at the ioda files MetaData height and pressure are available.

so you could switch out in the obsq.radiosonde.yaml, height for pressure if you wanted the vertical location of the obs in pressure? e.g.

  vertical coordinate:
    name: MetaData/pressure
     units: "pressure (Pa)"

I can do this and get an obs sequence out, but I am I interpreting the ioda file correctly? Some observations have a vertical coordinate in pressure, some in height (some maybe both?)

INFO: Converted 127651 observations -- this is with vertical coordinate pressure
INFO: Converted 35596 observations -- this is with vertical coordinate height

This is really nice Steve, I've been running it on various conventional obs. ioda.aircraft_obs_2021021600.nc4, ioda.ascat_obs_2021021600.nc4, ioda.radiosonde_obs_2021021600.nc4 ioda.satwind_obs_2021021600.nc4

@srherbener @fcvdb I've got couple of questions on the MetaData. Looking at the ioda files MetaData height and pressure are available.

so you could switch out in the obsq.radiosonde.yaml, height for pressure if you wanted the vertical location of the obs in pressure? e.g.

  vertical coordinate:
    name: MetaData/pressure
     units: "pressure (Pa)"

I can do this and get an obs sequence out, but I am I interpreting the ioda file correctly? Some observations have a vertical coordinate in pressure, some in height (some maybe both?)

INFO: Converted 127651 observations -- this is with vertical coordinate pressure INFO: Converted 35596 observations -- this is with vertical coordinate height

I'm not sure why the prepBUFR to IODA converter is writing out both height and pressure. Perhaps this was done because both of those are measurements from an instrument (ie, observed quantities). If it was me, I would place the variable intended to be the vertical coordinate in the MetaData group, and the other one in the ObsValue group to help clarify which one is the vertical coordinate. Perhaps @fcvdb can shed more light on why both height and pressure appear in the MetaData group.

The new IODA to obs_seq converter removes rows from the internal pandas dataframe that have missing values (NaNs) for any of the location, height, timestamp, and observation values. I think this can explain why the obs counts are different between using height vs pressure for the vertical coordinate. My inclination would be to go with the vertical coordinate that yields the most converted obs (ie pressure in this case). @fcvdb would that make sense?

@hkershaw-brown I have removed the partial python test, and updated the documentation. In the pytools.rst I added a section for "Available Tool Packages" and included pyjedi documentation. I was anticipating adding in documentation for pyqceff too, but I was hoping for you to fill that in. I'm okay if you don't like the organization I made on the pytools.rst page, and I'm open to another approach if that's what you want. Thanks!

@hkershaw-brown this is perfect Steve. Sorry I have not got to releasing this yet, various CISL events have been talking up time this week.

@hkershaw-brown no worries. Just wanted to let you know I finished the changes you requested. I'm fine with this PR waiting while you take care of some other things. No need to rush.