Thanks to visit codestin.com
Credit goes to github.com

Skip to content

🔒 🤖 CI Update lock files for free-threaded CI build(s) 🔒 🤖 - #34804

Merged
jeremiedbb merged 3 commits into
scikit-learn:mainfrom
scikit-learn-bot:auto-update-lock-files-free-threaded
Aug 25, 2026
Merged

🔒 🤖 CI Update lock files for free-threaded CI build(s) 🔒 🤖#34804
jeremiedbb merged 3 commits into
scikit-learn:mainfrom
scikit-learn-bot:auto-update-lock-files-free-threaded

Conversation

@scikit-learn-bot

Copy link
Copy Markdown
Contributor

Update lock files.

The update was generated by running:

python build_tools/update_environments_and_lock_files.py --select-tag free-threaded

Note

If the CI tasks fail, create a new branch based on this PR and add the required fixes to that branch.
Prefer re-running the same command above (with the matching --select-tag) rather than updating all lock files.
Remember to include [free-threaded] in your commit message so that the optional CI build runs.

@lesteve

lesteve commented Aug 24, 2026

Copy link
Copy Markdown
Member

Weird pickling error in callbacks, needs to be looked at in more details. I can reproduce locally with a free-threaded CPython with (need to install pytest-repeat for the --count flag):

pytest sklearn/callback/tests/test_callback_support.py::test_function_no_callback_support -k 'RecordingCall and processes-2' --count 10 -v

The tests fail 5-7 out of 10, not completely deterministic somehow.

The failing tests in the CI:

PARALLEL FAILED callback/tests/test_callback_support.py::test_function_no_callback_support[RecordingCallback-processes-2] - _pickle.PicklingError: Could not pickle the task to send it to the workers.
PARALLEL FAILED callback/tests/test_callback_support.py::test_function_no_callback_support[RecordingAutoPropagatedCallback-processes-2] - _pickle.PicklingError: Could not pickle the task to send it to the workers.
Details
_ ERROR at call of test_function_no_callback_support[RecordingCallback-processes-2] _
joblib.externals.loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/runner/miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/externals/loky/backend/queues.py", line 159, in _feed
    obj_ = dumps(obj, reducers=reducers)
  File "/home/runner/miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/externals/loky/backend/reduction.py", line 214, in dumps
    dump(obj, buf, reducers=reducers, protocol=protocol)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/externals/loky/backend/reduction.py", line 207, in dump
    _LokyPickler(file, reducers=reducers, protocol=protocol).dump(obj)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/home/runner/miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/externals/cloudpickle/cloudpickle.py", line 1313, in dump
    return super().dump(obj)
           ~~~~~~~~~~~~^^^^^
RuntimeError: list changed size during iteration
when serializing dict item 'record'
when serializing sklearn.callback.tests._common.callbacks.RecordingCallback state
when serializing sklearn.callback.tests._common.callbacks.RecordingCallback object
when serializing list item 0
when serializing dict item '_skl_callbacks'
when serializing sklearn.callback.tests._common.estimators.MaxIterEstimator state
when serializing sklearn.callback.tests._common.estimators.MaxIterEstimator object
when serializing tuple item 0
when serializing tuple item 1
when serializing list item 0
when serializing tuple item 0
when serializing joblib.parallel.BatchedCalls reconstructor arguments
when serializing joblib.parallel.BatchedCalls object
when serializing dict item 'fn'
when serializing joblib.externals.loky.process_executor._CallItem state
when serializing joblib.externals.loky.process_executor._CallItem object
"""

The above exception was the direct cause of the following exception:

n_jobs = 2, prefer = 'processes'
Callback = <class 'sklearn.callback.tests._common.callbacks.RecordingCallback'>

    @pytest.mark.parametrize("n_jobs", [1, 2])
    @pytest.mark.parametrize("prefer", ["threads", "processes"])
    @pytest.mark.parametrize(
        "Callback", [RecordingCallback, RecordingAutoPropagatedCallback]
    )
    def test_function_no_callback_support(n_jobs, prefer, Callback):
        """Check callbacks on estimators within function not supporting callbacks.
    
        Since the outer function does not support callbacks, there's no shared root context
        and the context trees of each sub-estimator are independent. As a result, the
        callback acts as a regular non-propagated callback: its on_fit_begin and on_fit_end
        are called once for each fit of the sub-estimator and the number of tasks is the sum
        of the number of tasks from all the sub-estimators.
        """
    
        def clone_and_fit(estimator):
            clone(estimator).fit()
    
        def func(estimator, n_fits, n_jobs, prefer):
            Parallel(n_jobs=n_jobs, prefer=prefer)(
                delayed(clone_and_fit)(estimator) for _ in range(n_fits)
            )
    
        n_fits, max_iter = 5, 7
        callback = Callback()
        estimator = MaxIterEstimator(max_iter=max_iter).set_callbacks(callback)
    
>       func(estimator, n_fits, n_jobs, prefer)

Callback   = <class 'sklearn.callback.tests._common.callbacks.RecordingCallback'>
callback   = <sklearn.callback.tests._common.callbacks.RecordingCallback object at 0x464b41d0310>
clone_and_fit = <function test_function_no_callback_support.<locals>.clone_and_fit at 0x464b41a0180>
estimator  = MaxIterEstimator(max_iter=7)
func       = <function test_function_no_callback_support.<locals>.func at 0x464b41a0c00>
max_iter   = 7
n_fits     = 5
n_jobs     = 2
prefer     = 'processes'

../sklearn/callback/tests/test_callback_support.py:150: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../sklearn/callback/tests/test_callback_support.py:142: in func
    Parallel(n_jobs=n_jobs, prefer=prefer)(
        clone_and_fit = <function test_function_no_callback_support.<locals>.clone_and_fit at 0x464b41a0180>
        estimator  = MaxIterEstimator(max_iter=7)
        n_fits     = 5
        n_jobs     = 2
        prefer     = 'processes'
../sklearn/utils/parallel.py:91: in __call__
    return super().__call__(iterable_with_config_and_warning_filters)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        __class__  = <class 'sklearn.utils.parallel.Parallel'>
        config     = {'assume_finite': False, 'working_memory': 1024, 'print_changed_only': True, 'display': 'diagram', ...}
        filters_func = <function _get_filters at 0x46340bd8700>
        iterable   = <generator object test_function_no_callback_support.<locals>.func.<locals>.<genexpr> at 0x464b4260d00>
        iterable_with_config_and_warning_filters = <generator object Parallel.__call__.<locals>.<genexpr> at 0x464b4330140>
        self       = Parallel(n_jobs=2)
        warning_filters = [('always', None, <class 'PendingDeprecationWarning'>, None, 0), ('always', None, <class 'DeprecationWarning'>, None, ...', re.IGNORECASE), <class 'Warning'>, None, 0), ('always', None, <class 'numpy.exceptions.RankWarning'>, None, 0), ...]
../../../../miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/parallel.py:2072: in __call__
    return output if self.return_generator else list(output)
                                                ^^^^^^^^^^^^
        _batched_calls_reducer_callback = <function Parallel.__call__.<locals>._batched_calls_reducer_callback at 0x464b41a0d80>
        backend_name = 'LokyBackend'
        iterable   = <generator object Parallel.__call__.<locals>.<genexpr> at 0x464b4330140>
        iterator   = <itertools.islice object at 0x464b41605a0>
        n_jobs     = 2
        output     = <generator object Parallel._get_outputs at 0x464b4330c80>
        pre_dispatch = 4
        self       = Parallel(n_jobs=2)
../../../../miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/parallel.py:1682: in _get_outputs
    yield from self._retrieve()
        _remaining_outputs = []
        detach_generator_exit = False
        dispatch_thread_id = 139674920146624
        iterator   = <itertools.islice object at 0x464b41605a0>
        pre_dispatch = 4
        self       = Parallel(n_jobs=2)
../../../../miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/parallel.py:1784: in _retrieve
    self._raise_error_fast()
        batched_results = [None]
        nb_jobs    = 3
        result     = None
        self       = Parallel(n_jobs=2)
        timeout_control_job = None
../../../../miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/parallel.py:1859: in _raise_error_fast
    error_job.get_result(self.timeout)
        error_job  = <joblib.parallel.BatchCompletionCallBack object at 0x463ba260790>
        self       = Parallel(n_jobs=2)
../../../../miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/parallel.py:758: in get_result
    return self._return_or_raise()
           ^^^^^^^^^^^^^^^^^^^^^^^
        backend    = <joblib._parallel_backends.LokyBackend object at 0x464b4210210>
        self       = <joblib.parallel.BatchCompletionCallBack object at 0x463ba260790>
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <joblib.parallel.BatchCompletionCallBack object at 0x463ba260790>

    def _return_or_raise(self):
        try:
            if self.status == TASK_ERROR:
>               raise self._result
E               _pickle.PicklingError: Could not pickle the task to send it to the workers.

self       = <joblib.parallel.BatchCompletionCallBack object at 0x463ba260790>

../../../../miniconda3/envs/testvenv/lib/python3.14t/site-packages/joblib/parallel.py:773: PicklingError

@jeremiedbb
jeremiedbb merged commit 65a85cf into scikit-learn:main Aug 25, 2026
37 checks passed
prady0t pushed a commit to prady0t/scikit-learn that referenced this pull request Sep 2, 2026
@jeremiedbb jeremiedbb mentioned this pull request Sep 8, 2026
14 tasks
jeremiedbb added a commit to jeremiedbb/scikit-learn that referenced this pull request Sep 8, 2026
jeremiedbb added a commit to jeremiedbb/scikit-learn that referenced this pull request Sep 8, 2026
jeremiedbb added a commit to jeremiedbb/scikit-learn that referenced this pull request Sep 9, 2026
jeremiedbb added a commit to jeremiedbb/scikit-learn that referenced this pull request Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants