Hi everyone,

Upstream grib2io recently gained the ability to stream data directly from AWS (see PR: NOAA-MDL/grib2io#232). I wanted to open this issue to discuss how we can leverage this for our workflows.

Currently, downloading and temporarily storing GRIB2 files can be a bottleneck. By utilizing this new streaming capability, we could achieve:

• Faster Processing: Read data directly into memory without waiting for full file downloads.
• Reduced Storage footprint: Eliminate the need for temporary local storage volumes.
• Cloud-Native Efficiency: Easier integration with serverless or containerized environments.

Are there specific pipelines or future applications where we could pilot this? If there's interest, I'd be happy to put together a quick proof-of-concept to benchmark the performance difference.