Thanks to visit codestin.com
Credit goes to Github.com

Skip to content

Conversation

@karlwx
Copy link
Contributor

@karlwx karlwx commented Jul 2, 2025

I came up with the idea for this feature a few days ago and figured it would be worth a shot. Since the only way to subset datasets before loading them into memory is by downloading certain variables, I wanted some capability to preprocess the datasets before they're loaded, like the preprocess keyword argument in xarray.open_mfdataset(). I think this could be especially useful for loading in multiple files at once using FastHerbie -- this can become problematic if you've got global data for many forecast times. The most obvious usage would be spatial subsetting, but it could also be used for dropping variables and dimensions that could cause dataset merge conflicts. Please have a look and see if you think it accomplishes what I think it does!

@blaylockbk
Copy link
Owner

This is a really neat idea. I'll try to take a look in the next few days.

@blaylockbk blaylockbk self-assigned this Aug 9, 2025
@blaylockbk blaylockbk added the enhancement New feature or request label Aug 9, 2025
@blaylockbk blaylockbk added this to the Next Release milestone Aug 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants