Thanks to visit codestin.com
Credit goes to github.com

Skip to content

1885 write cdf default compression to 6#1886

Merged
tmplummer merged 3 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:1885-write_cdf---default-compression-to-6
Jul 2, 2025
Merged

1885 write cdf default compression to 6#1886
tmplummer merged 3 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:1885-write_cdf---default-compression-to-6

Conversation

@tmplummer

Copy link
Copy Markdown
Contributor

Change Summary

Overview

This PR:

  • Sets the default gzip compression for CDF variables to 6.
  • Fixes a bug with how extra kwargs were being passed to cdf_to_xarray.

Updated Files

  • imap_processing/cdf/utils.py
    • Fix bug in how kwargs was being passed through to cdf_to_xarray
    • Set default compression in write_cdf to 6.
  • imap_processing/tests/cdf/test_utils.py
    • Add test coverage for kwargs passed to cdf_to_xarray and xarray_to_cdf

Closes: #1885

@tmplummer tmplummer self-assigned this Jul 1, 2025
@tmplummer tmplummer added this to IMAP Jul 1, 2025
@tmplummer tmplummer added the CDF Related to CDF files label Jul 1, 2025

@greglucas greglucas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you test this on your datasets at all to see if it helped?

file_path = file_path.construct_path()

dataset = cdf_to_xarray(file_path, kwargs)
dataset = cdf_to_xarray(file_path, **kwargs)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!

@tmplummer

Copy link
Copy Markdown
Contributor Author

@greglucas

Did you test this on your datasets at all to see if it helped?

Good idea... haha. Yes for a 34 MB file that I read in and wrote back out, the new file is 17 MB.

Comment on lines +49 to +50
if "to_datetime" not in kwargs:
kwargs["to_datetime"] = False # type: ignore

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@greglucas, do you have any opinions about this implementation (mirrors what is also in the write_cdf function) of setting defaults in the kwargs dict vs. just exposing those keywords at the interface to the load and write functions?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean having a signature like the following instead?

def func(args, to_datetime=False, **kwargs):
    cdf_to_xarray(file_path, to_datetime, **kwargs)

I don't have a strong opinion here if that is what you are asking about. I would suggest doing that as a separate PR though.

@tmplummer tmplummer merged commit de97a26 into IMAP-Science-Operations-Center:dev Jul 2, 2025
14 checks passed
@github-project-automation github-project-automation Bot moved this to Done in IMAP Jul 2, 2025
@tmplummer tmplummer deleted the 1885-write_cdf---default-compression-to-6 branch July 2, 2025 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CDF Related to CDF files

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

write_cdf - default compression to 6

2 participants