Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: stats.ttest_ind: Welch degrees of freedom becomes 1 after rescaling despite finite nonzero variances #26169

Description

@tasuku-kobayashi

Describe your issue.

For scipy.stats.ttest_ind(..., equal_var=False) in SciPy 1.18.1, multiplying both samples by an extreme but exactly representable power of two changes the returned degrees of freedom from 4 to 1, even though the computed sample variances remain positive and finite.

The t-statistic remains accurate, but the returned p-value changes substantially. The shrinking case emits no warning under NumPy's default floating-point error settings.

Expected and observed behavior

The samples are:

x = [-1, 0, 1]
y = [1, 2, 3]

For these samples, and after multiplying both by the same positive constant, the mathematical results are:

  • t = -sqrt(6) ≈ -2.449489742783178
  • df = 4
  • Two-sided p = 1 - (6/5)*sqrt(3/5) ≈ 0.07048399691021995

The powers of two used below preserve the input information exactly.

Common scale Returned t Returned df Returned p
1 -2.449489742783178 4.0 0.07048399691021992
2**-300 -2.449489742783178 1.0 0.24675171442884988
2**300 -2.449489742783178 1.0 0.24675171442884988

The shrinking call emits no captured warning. The expanding call emits three RuntimeWarning: overflow encountered in scalar power messages.

Both probabilities remain above 0.05, so the 5% decision does not change in this particular example. These deliberately extreme synthetic scales demonstrate a numerical failure; no claim is made about its frequency in practical datasets.

Degrees-of-freedom calculation

The computed unbiased sample variances in both groups are:

  • Shrinking case: 2.409919865102884e-181
  • Expanding case: 4.149515568880993e+180

Both are positive and finite.

Writing a = variance_x / n_x and b = variance_y / n_y, the Welch–Satterthwaite expression is:

df = (a + b)**2 / (a**2 / (n_x - 1) + b**2 / (n_y - 1))

In the recorded diagnostics, the numerator and denominator both become zero in the shrinking case and infinity in the expanding case. Their ratio is NaN.

The installed _unequal_var_ttest_denom helper then applies:

df = xp.where(xp.isnan(df), 1., df)

A separate diagnostic call to this helper in the same environment returned df=1, matching the public API result. The adjacent source comment associates undefined degrees of freedom with zero variances, but the variances in this example are nonzero.

Inspected SciPy 1.18.1 source.

Reference checks

Exact rational moments computed from the actual binary64 inputs using Python's Fraction give df=4 and t²=6, with a negative t-statistic.

A separate mpmath 1.3.0 calculation at 100 decimal digits compares the elementary p-value formula above with the regularized incomplete beta expression. They agree to better than 1e-95. Neither reference calculation calls SciPy.

Related reports

This example concerns a different calculation stage: the sample variances remain finite, but squaring their contributions in the degrees-of-freedom calculation loses range. A limited tracker search did not identify an exact duplicate; it was not exhaustive.

Would improving the range robustness of this degrees-of-freedom calculation be in scope? This report does not propose changing the behavior for mathematically zero variance.

Reproduction materials and assistance

The executable diagnostics, exact hexadecimal inputs, complete recorded output, environment details, independent reference calculation and file hashes are publicly available:

https://www.licklider.ai/reports/welch-df-fallback/README.md

AI provided assistance with coding diagnostic scripts, inspecting source code, translating text, and checking for typos.

Reproducing Code Example

import warnings

import numpy as np
import scipy
from scipy import stats

print("SciPy:", scipy.__version__, "NumPy:", np.__version__)

for k in (0, -300, 300):
    x = np.ldexp(np.array([-1., 0., 1.]), k)
    y = np.ldexp(np.array([1., 2., 3.]), k)

    with warnings.catch_warnings(record=True) as caught:
        warnings.simplefilter("always")
        result = stats.ttest_ind(x, y, equal_var=False)

    print(
        k,
        float(result.statistic),
        float(result.df),
        float(result.pvalue),
    )
    print([f"{w.category.__name__}: {w.message}" for w in caught])

Error message

No exception is raised. Output from the reproducer:


SciPy: 1.18.1 NumPy: 2.3.5
0 -2.449489742783178 4.0 0.07048399691021992
[]
-300 -2.449489742783178 1.0 0.24675171442884988
[]
300 -2.449489742783178 1.0 0.24675171442884988
['RuntimeWarning: overflow encountered in scalar power', 'RuntimeWarning: overflow encountered in scalar power', 'RuntimeWarning: overflow encountered in scalar power']


NumPy's error settings were:


{'divide': 'warn', 'over': 'warn', 'under': 'ignore', 'invalid': 'warn'}


`warnings.simplefilter("always")` captures emitted warnings; it does not enable NumPy's normally ignored underflow warnings.

SciPy/NumPy/Python version and system information

Capture date: 2026-09-12 UTC
SciPy: 1.18.1
SciPy Git revision: e4e854eaa8f18d807cd3496028e257e36caa93cc
NumPy: 2.3.5
Python: 3.12.14 (main, Aug 25 2026, 14:00:49) [Clang 22.1.3 ]
Platform: Linux-6.18.35-x86_64-with-glibc2.39
Reference library: mpmath 1.3.0, 100 decimal digits

Installed scipy/stats/_stats_py.py SHA256:
bfd5ab202427ec19ad39a7d985b8fe3a5b283718ae1adec0bd8fc42f605e5988


The installed source hash matches the pinned upstream SciPy 1.18.1 source. The latest development branch has not been executed for this report.

The complete recorded `scipy.show_config()` output is included here:

https://www.licklider.ai/reports/welch-df-fallback/observed-output.txt

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    defectA clear bug or issue that prevents SciPy from being installed or used as expectedneeds-decisionItems that need further discussion before they are merged or closedscipy.stats

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions