Feature/datatree #185

bbakernoaa · 2025-04-28T15:53:50Z

This PR allows for the ability to open a grib file requiring a lot of filters to access multiple data arrays into a single xarray.DataTree. This creates a tree'd structure which users can then go in and access without knowing the filters needed beforehand.

Enhancements to `DataTree` Test Coverage:

tests/test_datatree_backend.py: Added multiple new test cases to validate the structure, optimization, subsetting, and interpolation of DataTree objects created from GRIB2 files. These include tests for basic structure, level organization, PDTN optimizations, perturbation structures, subsetting by levels, and interpolation to new grids.

Minor Code Cleanup:

tests/test_xarray_backend.py: Removed an unnecessary blank line in the test_named_filter function for better readability.

…d of having to supply filters

EricEngle-NOAA · 2025-04-28T17:00:02Z

Looks like GitHub updated their default macOS runner. "macos-latest" is now Apple silicon and only supports Python >= 3.9. I think I am going to update master to reflect this.

EricEngle-NOAA · 2025-04-28T17:09:24Z

Also, can you make a Feature Request issue for this?

bbakernoaa · 2025-04-28T17:30:43Z

#187 Feature request

bbakernoaa · 2025-04-28T17:32:08Z

@EricEngle-NOAA I made a requirement for the datatree pieces that xarray 2024.10.0 need to be installed to be able to enable the datatree register and the test_datatree_backend.

bbakernoaa · 2025-04-28T18:04:15Z

An example on a GFS grib2 file

In [6]: a = xr.open_datatree('../CampaignForecast/data/gfs.t00z.pgrb2.0p25.f024',engine='grib2io', stack_vertical=True)

In [7]: a
Out[26]: 
<xarray.DataTree>
Group: /
├── Group: /mean_sea_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           MSLET                     (y, x) float32 4MB ...
│           PRMSL                     (y, x) float32 4MB ...
├── Group: /hybrid_level
│       Dimensions:                   (y: 721, x: 1440, valueOfFirstFixedSurface: 2)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│         * valueOfFirstFixedSurface  (valueOfFirstFixedSurface) float64 16B 1.0 2.0
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           CLMR                      (y, x) float32 4MB ...
│           GRLE                      (y, x) float32 4MB ...
│           ICMR                      (y, x) float32 4MB ...
│           REFD                      (valueOfFirstFixedSurface, y, x) float32 8MB ...
│           RWMR                      (y, x) float32 4MB ...
│           SNMR                      (y, x) float32 4MB ...
├── Group: /entire_atmosphere
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           CWAT                      (y, x) float32 4MB ...
├── Group: /surface
│   ├── Group: /surface/pdtn_0
│   │       Dimensions:                   (y: 721, x: 1440)
│   │       Coordinates:
│   │           refDate                   datetime64[ns] 8B ...
│   │           leadTime                  timedelta64[ns] 8B ...
│   │           valueOfFirstFixedSurface  float64 8B ...
│   │           latitude                  (y, x) float64 8MB ...
│   │           longitude                 (y, x) float64 8MB ...
│   │           validDate                 datetime64[ns] 8B ...
│   │       Dimensions without coordinates: y, x
│   │       Data variables:
│   │           4LFTX                     (y, x) float32 4MB ...
│   └── Group: /surface/pdtn_8
│           Dimensions:                   (y: 721, x: 1440)
│           Coordinates:
│               refDate                   datetime64[ns] 8B ...
│               leadTime                  timedelta64[ns] 8B ...
│               valueOfFirstFixedSurface  float64 8B ...
│               latitude                  (y, x) float64 8MB ...
│               longitude                 (y, x) float64 8MB ...
│               validDate                 datetime64[ns] 8B ...
│           Dimensions without coordinates: y, x
│           Data variables:
│               4LFTX                     (y, x) float32 4MB ...
├── Group: /planetary_boundary_layer
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           UGRD                      (y, x) float32 4MB ...
│           VGRD                      (y, x) float32 4MB ...
│           VRATE                     (y, x) float32 4MB ...
├── Group: /isobaric_surface
│       Dimensions:                   (valueOfFirstFixedSurface: 41, y: 721, x: 1440)
│       Coordinates:
│         * valueOfFirstFixedSurface  (valueOfFirstFixedSurface) float64 328B 1.0 ......
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables: (12/16)
│           ABSV                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           CLMR                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           DZDT                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           GRLE                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           HGT                       (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           ICMR                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           ...                        ...
│           SPFH                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           TCDC                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           TMP                       (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           UGRD                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           VGRD                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
│           VVEL                      (valueOfFirstFixedSurface, y, x) float32 170MB ...
├── Group: /height_above_ground
│   ├── Group: /height_above_ground/pdtn_0
│   │       Dimensions:                   (y: 721, x: 1440)
│   │       Coordinates:
│   │           refDate                   datetime64[ns] 8B ...
│   │           leadTime                  timedelta64[ns] 8B ...
│   │           valueOfFirstFixedSurface  float64 8B ...
│   │           latitude                  (y, x) float64 8MB ...
│   │           longitude                 (y, x) float64 8MB ...
│   │           validDate                 datetime64[ns] 8B ...
│   │       Dimensions without coordinates: y, x
│   │       Data variables:
│   │           APTMP                     (y, x) float32 4MB ...
│   └── Group: /height_above_ground/pdtn_8
│           Dimensions:                   (y: 721, x: 1440)
│           Coordinates:
│               refDate                   datetime64[ns] 8B ...
│               leadTime                  timedelta64[ns] 8B ...
│               valueOfFirstFixedSurface  float64 8B ...
│               latitude                  (y, x) float64 8MB ...
│               longitude                 (y, x) float64 8MB ...
│               validDate                 datetime64[ns] 8B ...
│           Dimensions without coordinates: y, x
│           Data variables:
│               APTMP                     (y, x) float32 4MB ...
├── Group: /depth_below_land
│       Dimensions:                   (valueOfFirstFixedSurface: 4, y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│         * valueOfFirstFixedSurface  (valueOfFirstFixedSurface) float64 32B 0.0 ... 1.0
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           SOILL                     (valueOfFirstFixedSurface, y, x) float32 17MB ...
│           SOILW                     (valueOfFirstFixedSurface, y, x) float32 17MB ...
│           TSOIL                     (valueOfFirstFixedSurface, y, x) float32 17MB ...
├── Group: /height_above_msl
│       Dimensions:                   (y: 721, x: 1440, valueOfFirstFixedSurface: 3)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│         * valueOfFirstFixedSurface  (valueOfFirstFixedSurface) float64 24B 1.829e+0...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           ICEG                      (y, x) float32 4MB ...
│           TMP                       (valueOfFirstFixedSurface, y, x) float32 12MB ...
│           UGRD                      (valueOfFirstFixedSurface, y, x) float32 12MB ...
│           VGRD                      (valueOfFirstFixedSurface, y, x) float32 12MB ...
├── Group: /low_cloud_layer
│       Dimensions:                   (leadTime: 2, y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│         * leadTime                  (leadTime) timedelta64[ns] 16B 18:00:00 1 days
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 (leadTime) datetime64[ns] 16B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           LCDC                      (leadTime, y, x) float32 8MB ...
├── Group: /medium_cloud_layer
│       Dimensions:                   (leadTime: 2, y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│         * leadTime                  (leadTime) timedelta64[ns] 16B 18:00:00 1 days
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 (leadTime) datetime64[ns] 16B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           MCDC                      (leadTime, y, x) float32 8MB ...
├── Group: /high_cloud_layer
│       Dimensions:                   (leadTime: 2, y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│         * leadTime                  (leadTime) timedelta64[ns] 16B 18:00:00 1 days
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 (leadTime) datetime64[ns] 16B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           HCDC                      (leadTime, y, x) float32 8MB ...
├── Group: /cloud_ceiling
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           HGT                       (y, x) float32 4MB ...
├── Group: /convective_cloud_bottom_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           PRES                      (y, x) float32 4MB ...
├── Group: /low_cloud_bottom_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           duration                  timedelta64[ns] 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           PRES                      (y, x) float32 4MB ...
├── Group: /medium_cloud_bottom_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           duration                  timedelta64[ns] 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           PRES                      (y, x) float32 4MB ...
├── Group: /high_cloud_bottom_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           duration                  timedelta64[ns] 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           PRES                      (y, x) float32 4MB ...
├── Group: /convective_cloud_top_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           PRES                      (y, x) float32 4MB ...
├── Group: /low_cloud_top_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           duration                  timedelta64[ns] 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           PRES                      (y, x) float32 4MB ...
│           TMP                       (y, x) float32 4MB ...
├── Group: /medium_cloud_top_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           duration                  timedelta64[ns] 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           PRES                      (y, x) float32 4MB ...
│           TMP                       (y, x) float32 4MB ...
├── Group: /high_cloud_top_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           duration                  timedelta64[ns] 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           PRES                      (y, x) float32 4MB ...
│           TMP                       (y, x) float32 4MB ...
├── Group: /level_244 - ['Convective cloud layer', 'unknown']
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           TCDC                      (y, x) float32 4MB ...
├── Group: /level_211 - ['Boundary layer cloud layer', 'unknown']
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           duration                  timedelta64[ns] 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           TCDC                      (y, x) float32 4MB ...
├── Group: /level_8 - ['Nominal Top of the Atmosphere', 'unknown']
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           duration                  timedelta64[ns] 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           ULWRF                     (y, x) float32 4MB ...
│           USWRF                     (y, x) float32 4MB ...
├── Group: /tropopause
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           HGT                       (y, x) float32 4MB ...
│           ICAHT                     (y, x) float32 4MB ...
│           PRES                      (y, x) float32 4MB ...
│           TMP                       (y, x) float32 4MB ...
│           UGRD                      (y, x) float32 4MB ...
│           VGRD                      (y, x) float32 4MB ...
│           VWSH                      (y, x) float32 4MB ...
├── Group: /max_wind_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           HGT                       (y, x) float32 4MB ...
│           ICAHT                     (y, x) float32 4MB ...
│           PRES                      (y, x) float32 4MB ...
│           TMP                       (y, x) float32 4MB ...
│           UGRD                      (y, x) float32 4MB ...
│           VGRD                      (y, x) float32 4MB ...
├── Group: /level_4 - ['Level of 0o C Isotherm', 'unknown']
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           HGT                       (y, x) float32 4MB ...
│           RH                        (y, x) float32 4MB ...
├── Group: /highest_freezing_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           HGT                       (y, x) float32 4MB ...
│           RH                        (y, x) float32 4MB ...
├── Group: /pressure_diff_from_ground
│       Dimensions:                   (valueOfFirstFixedSurface: 3, y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│         * valueOfFirstFixedSurface  (valueOfFirstFixedSurface) float64 24B 9e+03 .....
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           CAPE                      (valueOfFirstFixedSurface, y, x) float32 12MB ...
│           CIN                       (valueOfFirstFixedSurface, y, x) float32 12MB ...
│           PLPL                      (y, x) float32 4MB ...
│           RH                        (y, x) float32 4MB ...
│           SPFH                      (y, x) float32 4MB ...
│           TMP                       (y, x) float32 4MB ...
│           UGRD                      (y, x) float32 4MB ...
│           VGRD                      (y, x) float32 4MB ...
├── Group: /sigma_level
│       Dimensions:                   (y: 721, x: 1440)
│       Coordinates:
│           refDate                   datetime64[ns] 8B ...
│           leadTime                  timedelta64[ns] 8B ...
│           valueOfFirstFixedSurface  float64 8B ...
│           latitude                  (y, x) float64 8MB ...
│           longitude                 (y, x) float64 8MB ...
│           validDate                 datetime64[ns] 8B ...
│       Dimensions without coordinates: y, x
│       Data variables:
│           POT                       (y, x) float32 4MB ...
│           TMP                       (y, x) float32 4MB ...
│           UGRD                      (y, x) float32 4MB ...
│           VGRD                      (y, x) float32 4MB ...
│           VVEL                      (y, x) float32 4MB ...
└── Group: /pot_vorticity_surface
        Dimensions:                   (valueOfFirstFixedSurface: 2, y: 721, x: 1440)
        Coordinates:
            refDate                   datetime64[ns] 8B ...
            leadTime                  timedelta64[ns] 8B ...
          * valueOfFirstFixedSurface  (valueOfFirstFixedSurface) float64 16B -2e-06 2...
            latitude                  (y, x) float64 8MB ...
            longitude                 (y, x) float64 8MB ...
            validDate                 datetime64[ns] 8B ...
        Dimensions without coordinates: y, x
        Data variables:
            HGT                       (valueOfFirstFixedSurface, y, x) float32 8MB ...
            PRES                      (valueOfFirstFixedSurface, y, x) float32 8MB ...
            TMP                       (valueOfFirstFixedSurface, y, x) float32 8MB ...
            UGRD                      (valueOfFirstFixedSurface, y, x) float32 8MB ...
            VGRD                      (valueOfFirstFixedSurface, y, x) float32 8MB ...
            VWSH                      (valueOfFirstFixedSurface, y, x) float32 8MB ...

and plotting the temperature from the isobaric surfaces can then be done easily

a['isobaric_surface'].TMP.isel(y=0,x=720).plot(y='valueOfFirstFixedSurface', yincrease=True, yscale='log')

src/grib2io/xarray_backend.py

EricEngle-NOAA · 2025-04-29T01:39:25Z

This is an awesome feature! I had have some knowledge of DataTree but have not used them nor did I think to use it as a mechanism to handle the various types of GRIB2 message that can exist in a file. Really good stuff!

I will likely move the level mapping table to inside grib2io.tables. I have a more complete table that I asked a LLM to provide and intial first pass. I will integrate those into what you have.

bbakernoaa · 2025-04-29T13:14:48Z

Thanks @EricEngle-NOAA. I think it's a really interesting use case of it. I was working on another project where I needed to read and provide campaign forecasts for a field experiment.

A better mapping would be fantastic. We may even be able to map some things like typeOfFirstFixedSurface to something simpler like level like cfgrib. Or map to NetCDF-CF standard names for attributes and dimensions.

EricEngle-NOAA · 2025-04-29T15:47:38Z

When I approve the pull request, I am going to go merge it into 187-xarray-datatree branch

EricEngle-NOAA · 2025-04-29T17:30:09Z

Another reason to merge into a feature branch is to build out the table mapping.

* adding a datatree method to be able to open an entire gribfile instead of having to supply filters * Add additional human-readable names to LEVEL_NAME_MAPPING * Add tests for DataTree structure and filtering in GRIB2 files * remove accidental submodule * remove datatree interp test * require xarray >= 2023.4.0 * make datatree optional * Add DataTree support checks and refactor DataTree accessor registration * increase datatree xarray requirement * Add additional level mappings for cloud layers in LEVEL_NAME_MAPPING * Fix syntax error in LEVEL_NAME_MAPPING by adding missing comma

* Feature/datatree (#185) * adding a datatree method to be able to open an entire gribfile instead of having to supply filters * Add additional human-readable names to LEVEL_NAME_MAPPING * Add tests for DataTree structure and filtering in GRIB2 files * remove accidental submodule * remove datatree interp test * require xarray >= 2023.4.0 * make datatree optional * Add DataTree support checks and refactor DataTree accessor registration * increase datatree xarray requirement * Add additional level mappings for cloud layers in LEVEL_NAME_MAPPING * Fix syntax error in LEVEL_NAME_MAPPING by adding missing comma * Adding a custom level naming table. Adding "table_4_5_grib2io_level_name". This is a table that maps code values from table 4.5 to a custom name. The name is either mapped directly to a CF-conventions standard name or is a custom name for grib2io. These names are used to define the DataTree. * Update for xarray backend Level names now being pulled from new custom grib2io table for GRIB2 table 4.5. * Renaming xarray datatree test. [skip ci] * Update _grib2io.py Add to_section3() method for Grib2GridDef class. * Update tests/test_to_grib2.py Remove dtype attr for leadTime before decode. This is a temporary change. * Update Grib2GridDef and some code clean up. Clean up of Grib2GridDef property returns and ProjParameters class. * Update build_linux.yml to gcc-14. * Update license in pyproject.toml Modify setup.py with test prints to diagnose finding omp.h * Update setup.py * Update setup.py Added find_openmp_include() * Update setup.py * Update setup.py Clean up of setup.py * Clean up setup.py [skip ci] * Update macOS testing to gcc-14. * Update setup.py Make sure full omp include path is returned Minor update to GRIB2 tables. --------- Co-authored-by: Barry Baker <[email protected]> Co-authored-by: Eric Engle <[email protected]>

bbakernoaa added 8 commits April 28, 2025 10:36

adding a datatree method to be able to open an entire gribfile instea…

721675d

…d of having to supply filters

Add additional human-readable names to LEVEL_NAME_MAPPING

de76174

Add tests for DataTree structure and filtering in GRIB2 files

9967d4b

remove accidental submodule

cc7dfb1

remove datatree interp test

81b05c6

require xarray >= 2023.4.0

5d65209

make datatree optional

6944b42

Add DataTree support checks and refactor DataTree accessor registration

df8dea0

increase datatree xarray requirement

9e8c04c

bbakernoaa added 2 commits April 28, 2025 13:36

Add additional level mappings for cloud layers in LEVEL_NAME_MAPPING

254855b

Fix syntax error in LEVEL_NAME_MAPPING by adding missing comma

f5b9d6b

bbakernoaa marked this pull request as ready for review April 28, 2025 18:17

EricEngle-NOAA mentioned this pull request Apr 28, 2025

Add ability to read grib as xarray DataTree #187

Closed

EricEngle-NOAA reviewed Apr 28, 2025

View reviewed changes

src/grib2io/xarray_backend.py Show resolved Hide resolved

EricEngle-NOAA self-assigned this Apr 29, 2025

EricEngle-NOAA added the enhancement New feature or request label Apr 29, 2025

EricEngle-NOAA linked an issue Apr 29, 2025 that may be closed by this pull request

Add ability to read grib as xarray DataTree #187

Closed

EricEngle-NOAA changed the base branch from master to 187-xarray-datatree April 29, 2025 17:40

EricEngle-NOAA merged commit 524b2bc into NOAA-MDL:187-xarray-datatree Apr 29, 2025
18 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feature/datatree #185

Feature/datatree #185

Uh oh!

bbakernoaa commented Apr 28, 2025

Uh oh!

EricEngle-NOAA commented Apr 28, 2025 •

edited

Loading

Uh oh!

EricEngle-NOAA commented Apr 28, 2025

Uh oh!

bbakernoaa commented Apr 28, 2025

Uh oh!

bbakernoaa commented Apr 28, 2025

Uh oh!

bbakernoaa commented Apr 28, 2025 •

edited

Loading

Uh oh!

Uh oh!

EricEngle-NOAA commented Apr 29, 2025 •

edited

Loading

Uh oh!

bbakernoaa commented Apr 29, 2025

Uh oh!

EricEngle-NOAA commented Apr 29, 2025

Uh oh!

EricEngle-NOAA commented Apr 29, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Feature/datatree #185

Feature/datatree #185

Uh oh!

Conversation

bbakernoaa commented Apr 28, 2025

Enhancements to DataTree Test Coverage:

Minor Code Cleanup:

Uh oh!

EricEngle-NOAA commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EricEngle-NOAA commented Apr 28, 2025

Uh oh!

bbakernoaa commented Apr 28, 2025

Uh oh!

bbakernoaa commented Apr 28, 2025

Uh oh!

bbakernoaa commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

EricEngle-NOAA commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bbakernoaa commented Apr 29, 2025

Uh oh!

EricEngle-NOAA commented Apr 29, 2025

Uh oh!

EricEngle-NOAA commented Apr 29, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Enhancements to `DataTree` Test Coverage:

EricEngle-NOAA commented Apr 28, 2025 •

edited

Loading

bbakernoaa commented Apr 28, 2025 •

edited

Loading

EricEngle-NOAA commented Apr 29, 2025 •

edited

Loading