Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BLD Pin cython<3.2.6 to avoid Cython __annotate__ pickling bug - #34533

Merged
betatim merged 5 commits into
scikit-learn:mainfrom
cakedev0:ci/fix/cython_326
Jul 21, 2026
Merged

BLD Pin cython<3.2.6 to avoid Cython __annotate__ pickling bug#34533
betatim merged 5 commits into
scikit-learn:mainfrom
cakedev0:ci/fix/cython_326

Conversation

@cakedev0

@cakedev0 cakedev0 commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Reference Issues/PRs

Fixes #34525

What does this implement/fix? Explain your changes.

Pins cython<3.2.6 in pyproject.toml, in both [build-system] requires and the build optional-dependencies extra. These two lists are kept in sync historically, and only [build-system] requires is actually read by pip's isolated build backend, so both need updating together for the fix to be complete and consistent.

(The build extra isn't invoked anywhere in our own docs/CI — development_setup.rst installs build deps by literal package name instead — but it's still valid PEP 508 metadata: anyone can pip install -e ".[build]" to pull in the pinned build deps before pip install --no-build-isolation --editable ., and it's also what tools that introspect pyproject.toml extras without doing a build, e.g. PyPI's own project page, will see.)

AI usage disclosure

  • research and understanding
  • writing this PR desc.

@cakedev0

cakedev0 commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

anyone can pip install -e ".[build]" to pull in the pinned build deps before pip install --no-build-isolation --editable .

Well actually, this doesn't build:

meson-python: error: Could not find ninja version 1.8.2 or newer.

@cakedev0

cakedev0 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

The failed job seems to be a flaky one, see #30810

@lesteve

lesteve commented Jul 21, 2026

Copy link
Copy Markdown
Member

anyone can pip install -e ".[build]" to pull in the pinned build deps before pip install --no-build-isolation --editable .

Well actually, this doesn't build:

meson-python: error: Could not find ninja version 1.8.2 or newer.

That doesn't surprise me too much, I guess this extra could still be used with pip install scikit-learn[build]. I don't think this is used to be honest. This was moved from the setuptools days and I don't think it was used at that time either ... I guess running a GitHub search may be useful to figure out whether pip install scikit-learn[something-goes-here] is used in the wild. But that would be for a separate issue.

@betatim

betatim commented Jul 21, 2026

Copy link
Copy Markdown
Member

I've been trying to follow the story/catch up with the discussion related to the cython __annotate__ problem. In #34344 (comment) Loic mentions that we use master for building the nightly wheel. Will this PR also limit that to <3.2.6? I think the answer is no. Which makes me wonder, why do we use the HEAD of cython for building the dev wheels? Should we also limit that to <3.2.5?

lesteve
lesteve reviewed Jul 21, 2026
View reviewed changes

@lesteve lesteve left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, only add TODO comments linking to the Cython issue cython/cython#7846 and saying that we can tweak the pin when Cython has a release with a fix.

Other than the flaky build, the other Python 3.14 wheels pass which is 🎉 see wheel build logs

@lesteve

lesteve commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Will this PR also limit that to <3.2.6?

My memory tells me no, but for the scipy-dev build we already have a temporary work-around in #34526 1

Why I think this PR is not enough to force the cython<3.2.6 installation (without #34526). IIRC we use --no-build-isolation in most places, which uses packages already present in the conda env rather than installing the build dependencies from the build section. IIRC there is some flag to check build dependencies but not many people use it and sometimes you want different build versions for the wheels than when installing for development (a bit hazy on the details).

Footnotes

  1. I mistakenly thought this would fix the wheels, this tells you that I am a bit CI-rusty 😜 ↩

@betatim

betatim commented Jul 21, 2026

Copy link
Copy Markdown
Member

My memory tells me no, but for the scipy-dev build we already have a temporary work-around in #34526 1

Aha! Sorry, I didn't pass by that PR and when I used the GitHub search to find the instal script I must have gotten a version from before that PR.

All good then for me

lesteve
lesteve approved these changes Jul 21, 2026
View reviewed changes

@lesteve lesteve left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Pushed a small tweak + [cd build] (mostly as another attempt to have a satisfying green wheel build)

@lesteve lesteve changed the title FIX: build - enforce cython <3.2.6 BLD Pin cython <3.2.6 to avoid Cython __annotate__ pickling bug Jul 21, 2026
@lesteve lesteve changed the title BLD Pin cython <3.2.6 to avoid Cython __annotate__ pickling bug BLD Pin cython<3.2.6 to avoid Cython __annotate__ pickling bug Jul 21, 2026
@lesteve lesteve changed the title BLD Pin cython<3.2.6 to avoid Cython __annotate__ pickling bug BLD Pin cython<3.2.6 to avoid Cython __annotate__ pickling bug Jul 21, 2026
betatim
betatim approved these changes Jul 21, 2026
View reviewed changes

@betatim betatim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I like using !=1.2.3 to exclude versions that are broken but leave the door open for newer versions. In particular for our CI env definitions where a bot comes to update them regularly that is a good pattern to use. In pyproject.toml I think it is fine if we implement the TODO comment when we know the fixed cython version

@betatim
betatim enabled auto-merge (squash) July 21, 2026 10:53
@betatim
betatim merged commit 28fca3f into scikit-learn:main Jul 21, 2026
68 checks passed
This was referenced Jul 21, 2026
Closed
Closed
@jeremiedbb jeremiedbb mentioned this pull request Jul 27, 2026
Merged
prady0t pushed a commit to prady0t/scikit-learn that referenced this pull request Sep 2, 2026
@cakedev0 @lesteve @prady0t
BLD Pin cython<3.2.6 to avoid Cython __annotate__ pickling bug (sciki…
1d76cca 
…t-learn#34533)

Co-authored-by: Loïc Estève <[email protected]>
@jeremiedbb jeremiedbb mentioned this pull request Sep 8, 2026
Merged
14 tasks
jeremiedbb pushed a commit to jeremiedbb/scikit-learn that referenced this pull request Sep 9, 2026
@cakedev0 @lesteve @jeremiedbb
BLD Pin cython<3.2.6 to avoid Cython __annotate__ pickling bug (sciki…
dd159a7 
…t-learn#34533)

Co-authored-by: Loïc Estève <[email protected]>
jeremiedbb pushed a commit to jeremiedbb/scikit-learn that referenced this pull request Sep 9, 2026
@cakedev0 @lesteve @jeremiedbb
BLD Pin cython<3.2.6 to avoid Cython __annotate__ pickling bug (sciki…
caa7fbb 
…t-learn#34533)

Co-authored-by: Loïc Estève <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@betatim betatim betatim approved these changes
@lesteve lesteve lesteve approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Cython >= 3.2.6 (including cython dev) issues related to KDTree picklability

3 participants

@cakedev0 @lesteve @betatim