Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Relax tolerance for HCOMPRESS_1 tests with quantization #18476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Relax tolerance for HCOMPRESS_1 tests with quantization #18476

wants to merge 1 commit into from

Conversation

mohsinm-dev
Copy link
Contributor

Description

This pull request addresses test failures in test_fitsio.py::test_compress for HCOMPRESS_1 compression with SUBTRACTIVE_DITHER_2 quantization method by relaxing the comparison tolerances to account for the expected precision differences in lossy compression.

The HCOMPRESS_1 algorithm with quantization method 2 (SUBTRACTIVE_DITHER_2) is inherently lossy, which leads to small precision differences when comparing data compressed by different implementations (fitsio vs astropy). The tests were using NumPy's default tolerances (rtol=1e-7, atol=0), which were too strict for this lossy compression method.

Changes Made

  • Modified test_compress() in astropy/io/fits/hdu/compressed/tests/test_fitsio.py
  • Added conditional logic to detect when HCOMPRESS_1 with quantization is being tested (by checking for ZQUANTIZ header)
  • Applied relaxed tolerances (rtol=1e-5, atol=1e-5) for these specific cases only
  • Maintained default tolerances for all other compression types to ensure test strictness elsewhere

Test Results

All 15 previously failing tests now pass:

  • HCOMPRESS_1 {'qlevel': 20, 'qmethod': 2} with various float types (<f4, <f8, >f4, >f8)
  • Various array shapes: (12, 12), (15, 15), (15, 15, 15), (5, 5, 5)
  • Various tile dimensions

The reported maximum differences were:

  • Max absolute difference: 1.5258789e-05
  • Max relative difference: 7.06999977e-06

Our chosen tolerances provide appropriate coverage with a safety margin.

Impact

This is a test-only change with no impact on the actual compression/decompression functionality. It only affects how strictly we compare results for this specific lossy compression method.

Fixes #18468

Checklist

  • No changelog entry needed (minor test update per contributing guidelines)
  • Tests pass locally with the fix
  • No user-visible changes
  • No documentation changes needed

@mohsinm-dev
Relax tolerance for HCOMPRESS_1 tests with quantization
4e53373 
Fixes test failures with HCOMPRESS_1 compression using SUBTRACTIVE_DITHER_2
quantization method by using appropriate tolerances for lossy compression.

Fixes astropy#18468
@mohsinm-dev mohsinm-dev requested a review from saimn as a code owner July 30, 2025 19:19
@github-actions GitHub Actions
Copy link
Contributor

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

github-actions[bot]
github-actions bot reviewed Jul 30, 2025
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to Astropy 👋 and congratulations on your first pull request! 🎉

A project member will respond to you as soon as possible; in the meantime, please have a look over the Checklist for Contributed Code and make sure you've addressed as many of the questions there as possible.

If you feel that this pull request has not been responded to in a timely manner, please send a message directly to the development mailing list. If the issue is urgent or sensitive in nature (e.g., a security vulnerability) please send an e-mail directly to the private e-mail [email protected].

@pllim pllim added this to the v7.1.1 milestone Jul 31, 2025
@pllim pllim added the backport-v7.1.x on-merge: backport to v7.1.x label Jul 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@saimn saimn Awaiting requested review from saimn saimn is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
backport-v7.1.x on-merge: backport to v7.1.x Bug io.fits no-changelog-entry-needed testing
Projects
None yet
Milestone
v7.1.1
Development

Successfully merging this pull request may close these issues.

Tests failures with test_fitsio.py::test_compress (HCOMPRESS_1)
2 participants
@mohsinm-dev @pllim