[BUG] Fix infinite recursion in `_SuppressWarningPattern` and `LossySetitemError` in `SummaryTransformer` #7903

celestinoxp · 2025-02-26T19:12:30Z

Issue

The _custom_showwarning method in sktime/utils/warnings.py causes infinite recursion when handling certain warnings (e.g., pandas LossySetitemError), resulting in a RecursionError: maximum recursion depth exceeded. Additionally, in sktime/transformations/series/summarize.py, the line func_dict.loc[:, "window"] = func_dict["window"].astype("object") throws a LossySetitemError in pandas 2.x due to incompatible type coercion.

This issue was identified while using PyCaret with sktime, where tests failed with a crash in a joblib worker due to a PicklingError caused by excessive recursion.

Solution

Modified the _custom_showwarning method in the _SuppressWarningPattern class to add a guard against recursion using an _in_warning flag, while preserving delegation to original_showwarning.
Fixed SummaryTransformer to use a type-safe conversion with func_dict["window"] = func_dict["window"].astype("object", copy=False), avoiding the LossySetitemError.

Changes

sktime/utils/warnings.py: Added protection against infinite recursion in the _custom_showwarning method of _SuppressWarningPattern using an _in_warning flag.
sktime/transformations/series/summarize.py: Replaced the problematic assignment with an explicit conversion of the "window" column to object using copy=False for greater efficiency.
sktime/utils/tests/test_warnings.py: Added a new test file to verify that _custom_showwarning in _SuppressWarningPattern emits warnings without recursion.
sktime/transformations/series/tests/test_summarize.py: Added a new test test_summarize_no_lossy_setitem to confirm that SummaryTransformer.fit does not raise LossySetitemError.

Verification

Tested locally with PyCaret and sktime, resolving the crash in the test_blend_model_predict test. The change in summarize.py ensures compatibility with pandas 2.x while maintaining efficiency with copy=False.

…n SummaryTransformer

fkiraly · 2025-02-27T10:38:53Z

Thanks for the fix!

Failure occurs on main, so I would recommend a quick merge.

benHeid · 2025-02-27T21:28:33Z

I am not sure, if this is completely fixing the issue. I think it is related to some kind of race-condition see #7906 because of the threading backend used in some tests. If this is true, we need some kind of thread safe solution, i.e., guard this not only by a flag but by a lock.
But I might also be wrong.

fkiraly · 2025-02-28T17:32:13Z

Can we somehow test that this is a fix?

It does not seem too clear how we would establish the condition in a test.

benHeid · 2025-02-28T19:54:39Z

Can we somehow test that this is a fix?

It does not seem too clear how we would establish the condition in a test.

Not sure if this should be implemented in a test case. But you might check it using the following lines of code:

import time
import warnings

import joblib
from sktime.utils.warnings import _suppress_pd22_warning

def f(set_warning):
    if set_warning:
        with warnings.catch_warnings():
            with _suppress_pd22_warning:
                time.sleep(2)
                #warnings.warn("'HI' is deprecated and will be removed in a future version, please use 'HELLO' instead", category=FutureWarning)
    warnings.warn("'HI' is deprecated and will be removed in a future version, please use 'HELLO' instead" + str(set_warning), category=FutureWarning)

if __name__ == "__main__":
    joblib.Parallel(n_jobs=2, backend="threading")(joblib.delayed(f)(set_warning) for set_warning in [True, False])

This should print the warning two times (for both threads). But it is only printing the warning once. (Compare with multiprocessing, there the warning is printed two times).

I checked it on this branch and it is only printing the warning once.

EDIT
I changed the example since it does not work correctly before

fkiraly · 2025-02-28T20:36:55Z

@benHeid, but is the race condition addressed?

benHeid · 2025-02-28T20:45:20Z

No printing the warning once means the race condition is not adressed. Since the warning of thread 2 (with parameter false) is not printed, because thread 1 is inside of the with statement and replaced the show warning by entering the with statement.

benHeid · 2025-03-01T14:31:07Z

@fkiraly would it be possible to use the following statement everywhere were we use with _supress_pd22_warning

        with warnings.catch_warnings():
            warnings.filterwarnings("ignore", 
                                  category=FutureWarning,
                                  message=r"'[A-Z]+' is deprecated and will be removed in a future version, please use '[A-Z]+' instead")

This statement would not change the warning module globally and it seem to work. But would appreciate confirmation by someone else.

fkiraly · 2025-03-01T19:30:50Z

that would work.

Conjecture: using the same class causes the race condition.

Turning the static _suppress_pd22_warning into

def _suppress_pd22_warning():
    return _SuppressWarningPattern(
        FutureWarning,
        r"'[A-Z]+' is deprecated and will be removed in a future version, please use '[A-Z]+' instead",  # noqa
    )

and replacing with _suppress_pd22_warning with with _suppress_pd22_warning() would also work.

fkiraly · 2025-03-01T19:36:00Z

Proposed fix: #7910

fkiraly

I think the fix in #7910 is the solution - thanks for the fix, I think it put us on the right track.

What I suggest is adding the tests only; I also notice the fix gets the filter removed, I will restore that. I will take care of this and credit you for the fix.

Regarding the fix - should we also replace the internals with catch_warnings?

celestinoxp · 2025-03-01T20:45:59Z

Thanks @fkiraly and @benHeid for the feedback!

I'm glad my correction helped point you to a solution. I see that #7910 seems like a good way to handle the threaded warnings issue, and I agree that you could edit my PR to restore the old filter code in _custom_showwarning if that's needed. For the catch_warnings idea, I'm not sure what's best. You can adjust the PR however you see fit, and I trust you will find the best solution. Let me know if there is anything else I can do to help!

I'm looking forward to seeing how it turns out!

fkiraly

I changed the environment to dispatch to warnings.filterwarnings.

@celestinoxp, would you be able to check whether this still addresses your use case?

fkiraly · 2025-03-02T14:01:11Z

@benHeid, I checked your code - this PR seems to fix the infinite loop if the race condition occurs, but, unfortunately, not the race conidition itself. I would hence go and merge this, and we leave the issue with the race condition open.

…etitemError` in `SummaryTransformer` (sktime#7903) ### Issue The `_custom_showwarning` method in `sktime/utils/warnings.py` causes infinite recursion when handling certain warnings (e.g., pandas `LossySetitemError`), resulting in a `RecursionError: maximum recursion depth exceeded`. Additionally, in `sktime/transformations/series/summarize.py`, the line `func_dict.loc[:, "window"] = func_dict["window"].astype("object")` throws a `LossySetitemError` in pandas 2.x due to incompatible type coercion. This issue was identified while using PyCaret with sktime, where tests failed with a crash in a joblib worker due to a `PicklingError` caused by excessive recursion. ### Solution - Modified the `_custom_showwarning` method in the `_SuppressWarningPattern` class to add a guard against recursion using an `_in_warning` flag, while preserving delegation to `original_showwarning`. - Fixed `SummaryTransformer` to use a type-safe conversion with `func_dict["window"] = func_dict["window"].astype("object", copy=False)`, avoiding the `LossySetitemError`. ### Changes - `sktime/utils/warnings.py`: Added protection against infinite recursion in the `_custom_showwarning` method of `_SuppressWarningPattern` using an `_in_warning` flag. - `sktime/transformations/series/summarize.py`: Replaced the problematic assignment with an explicit conversion of the `"window"` column to `object` using `copy=False` for greater efficiency. - `sktime/utils/tests/test_warnings.py`: Added a new test file to verify that `_custom_showwarning` in `_SuppressWarningPattern` emits warnings without recursion. - `sktime/transformations/series/tests/test_summarize.py`: Added a new test `test_summarize_no_lossy_setitem` to confirm that `SummaryTransformer.fit` does not raise `LossySetitemError`. ### Related - Investigated in PyCaret PR [sktime#4150](pycaret/pycaret#4150) ### Verification Tested locally with PyCaret and sktime, resolving the crash in the `test_blend_model_predict` test. The change in `summarize.py` ensures compatibility with pandas 2.x while maintaining efficiency with `copy=False`.

celestinoxp force-pushed the fix-warning-recursion branch from ae76696 to a1a1417 Compare February 26, 2025 19:45

fkiraly added bugfix Fixes a known bug or removes unintended behavior module:base-framework BaseObject, registry, base framework module:transformations transformations module: time series transformation, feature extraction, pre-/post-processing labels Feb 26, 2025

fkiraly changed the title ~~Fix infinite recursion in _custom_showwarning and LossySetitemError in SummaryTransformer~~ [BUG] Fix infinite recursion in _custom_showwarning and LossySetitemError in SummaryTransformer Feb 26, 2025

celestinoxp force-pushed the fix-warning-recursion branch from a1a1417 to e5ae331 Compare February 26, 2025 21:44

Fix infinite recursion in _custom_showwarning and LossySetitemError i…

c7c2075

…n SummaryTransformer

celestinoxp force-pushed the fix-warning-recursion branch from e5ae331 to c7c2075 Compare February 26, 2025 22:07

celestinoxp marked this pull request as ready for review February 26, 2025 22:10

celestinoxp requested review from benHeid, felipeangelimvieira, fkiraly, fnhirwa, geetu040 and yarnabrina as code owners February 26, 2025 22:10

XinyuWuu mentioned this pull request Feb 27, 2025

[ENH] Extend forecasting benchmark to global setting #6774

Merged

fkiraly approved these changes Feb 27, 2025

View reviewed changes

fkiraly mentioned this pull request Feb 28, 2025

[BUG] Race Conditions in sktime.utils._SuppressWarningPattern with joblib backend threading #7906

Open

fkiraly requested changes Mar 1, 2025

View reviewed changes

use warnings filterwarnings

5c6b24e

fkiraly previously approved these changes Mar 2, 2025

View reviewed changes

fkiraly changed the title ~~[BUG] Fix infinite recursion in _custom_showwarning and LossySetitemError in SummaryTransformer~~ [BUG] Fix infinite recursion in _SuppressWarningPattern and LossySetitemError in SummaryTransformer Mar 2, 2025

linting

b2facef

fkiraly dismissed their stale review via b2facef March 2, 2025 12:59

Update test_warnings.py

f6888e0

fkiraly merged commit cc227fb into sktime:main Mar 2, 2025
65 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[BUG] Fix infinite recursion in `_SuppressWarningPattern` and `LossySetitemError` in `SummaryTransformer` #7903

[BUG] Fix infinite recursion in `_SuppressWarningPattern` and `LossySetitemError` in `SummaryTransformer` #7903

Uh oh!

celestinoxp commented Feb 26, 2025 •

edited by fkiraly

Loading

Uh oh!

fkiraly commented Feb 27, 2025

Uh oh!

benHeid commented Feb 27, 2025

Uh oh!

fkiraly commented Feb 28, 2025

Uh oh!

benHeid commented Feb 28, 2025 •

edited

Loading

Uh oh!

fkiraly commented Feb 28, 2025

Uh oh!

benHeid commented Feb 28, 2025

Uh oh!

benHeid commented Mar 1, 2025

Uh oh!

fkiraly commented Mar 1, 2025 •

edited

Loading

Uh oh!

fkiraly commented Mar 1, 2025

Uh oh!

fkiraly left a comment

Uh oh!

celestinoxp commented Mar 1, 2025

Uh oh!

fkiraly left a comment

Uh oh!

fkiraly commented Mar 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

[BUG] Fix infinite recursion in _SuppressWarningPattern and LossySetitemError in SummaryTransformer #7903

[BUG] Fix infinite recursion in _SuppressWarningPattern and LossySetitemError in SummaryTransformer #7903

Uh oh!

Conversation

celestinoxp commented Feb 26, 2025 • edited by fkiraly Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue

Solution

Changes

Related

Verification

Uh oh!

fkiraly commented Feb 27, 2025

Uh oh!

benHeid commented Feb 27, 2025

Uh oh!

fkiraly commented Feb 28, 2025

Uh oh!

benHeid commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fkiraly commented Feb 28, 2025

Uh oh!

benHeid commented Feb 28, 2025

Uh oh!

benHeid commented Mar 1, 2025

Uh oh!

fkiraly commented Mar 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fkiraly commented Mar 1, 2025

Uh oh!

fkiraly left a comment

Choose a reason for hiding this comment

Uh oh!

celestinoxp commented Mar 1, 2025

Uh oh!

fkiraly left a comment

Choose a reason for hiding this comment

Uh oh!

fkiraly commented Mar 2, 2025

Uh oh!

Uh oh!

Uh oh!

[BUG] Fix infinite recursion in `_SuppressWarningPattern` and `LossySetitemError` in `SummaryTransformer` #7903

[BUG] Fix infinite recursion in `_SuppressWarningPattern` and `LossySetitemError` in `SummaryTransformer` #7903

celestinoxp commented Feb 26, 2025 •

edited by fkiraly

Loading

benHeid commented Feb 28, 2025 •

edited

Loading

fkiraly commented Mar 1, 2025 •

edited

Loading