What will be the recommended approach for packagers here? Is anything special needed to indicate that a package supports python ~gil? Should the packaging guide include documentation on this topic?

I assume we'd have things like:

    conflicts("^python ~gil")

but that means that all packages become automatically free-threading supported instead of that needing to be affirmatively stated by the maintainer.

but that means that all packages become automatically free-threading supported instead of that needing to be affirmatively stated by the maintainer.

Maybe this is just my lack of understanding what the "Free Threading" classifier is intended to document:
https://pypi.org/search/?c=Programming+Language+%3A%3A+Python+%3A%3A+Free+Threading

I think we'll have to leave it up to downstream packages to add this conflict. I suspect that most pure-Python packages will support GIL. Cython packages may need special changes. @rgommers would know more.

There are currently a lot of underspecified packages out there. We could definitely go further to scrape PyPI to add supported Python version ranges. The presence of both 3.13/3.14 and 3.13t/3.14t wheels would be strong evidence that GIL is supported, even if no tags are added.

I think it's fine to not worry about conflicts unless the build itself is actually broken (and fixing that is usually easy). Free-threading is opt-in and known to be under development, and the users who want this now do know that. It's actually quite useful to be able to build packages for a free-threading interpreter even if the package itself hasn't been made thread-safe yet or re-enables the GIL on import. Running code with PYTHON_GIL=0 python to experiment tends to work for many packages.

I'm inclined to recommend that it should not be supported for 3.13t in Spack at all. It's so much more buggy than 3.14, and lots of key libraries (cryptography, cffi, pyyaml`, etc.) will never support 3.13t. So there isn't much point in spending any effort on 3.13t anymore.

Cython packages may need special changes

Yes, Cython/C/C++/Rust-using libraries all need changes (as described at https://py-free-threading.github.io/porting/), but a decent amount of that for the most popular packages is done now (see https://hugovk.github.io/free-threaded-wheels/).

It'd be good to call the option freethreading rather than gil, as per the Python Steering Council's guidance: python/steering-council#221 (comment)

I think we can let people play around with 3.13t if they want, but I agree that we won't be trying to fix packages to make 3.13t work.

I'm fine with renaming the variant "freethreading", it's more clear to me as well.

@spackbot fix style

Let me see if I can fix that for you!

I was able to run spack style --fix for you!

packages/pcl/package.py
repos/spack_repo/builtin/packages/armadillo/package.py
repos/spack_repo/builtin/packages/esmf/package.py
repos/spack_repo/builtin/packages/fcgi/package.py
repos/spack_repo/builtin/packages/flecsi/package.py
repos/spack_repo/builtin/packages/pcl/package.py
repos/spack_repo/builtin/packages/petsc/package.py
repos/spack_repo/builtin/packages/precice/package.py
repos/spack_repo/builtin/packages/py_metatensor_learn/package.py
repos/spack_repo/builtin/packages/py_metatensor_operations/package.py
repos/spack_repo/builtin/packages/py_metatensor_torch/package.py
repos/spack_repo/builtin/packages/py_metatomic_torch/package.py
repos/spack_repo/builtin/packages/py_metatrain/package.py
repos/spack_repo/builtin/packages/py_petsc4py/package.py
repos/spack_repo/builtin/packages/py_pydantic/package.py
repos/spack_repo/builtin/packages/py_pydantic_core/package.py
repos/spack_repo/builtin/packages/py_shroud/package.py
repos/spack_repo/builtin/packages/py_vesin/package.py
repos/spack_repo/builtin/packages/python/package.py
repos/spack_repo/builtin/packages/r/package.py
repos/spack_repo/builtin/packages/root/package.py
repos/spack_repo/builtin/packages/superlu/package.py
repos/spack_repo/builtin/packages/superlu_dist/package.py
repos/spack_repo/builtin/packages/velvet/package.py
repos/spack_repo/builtin/packages/xrootd/package.py
�[1;34m==> �[0mrunning flake8
repos/spack_repo/builtin/packages/python/package.py:176: [E501] line too long (109 > 99 characters)
�[1;34m==> �[0mrunning isort
�[1;34m==> �[0mrunning black

I've updated the branch with style fixes.

Should we add external variant detection? Does the executable name change?

Should we add external variant detection? Does the executable name change?

Not sure about the first question, but for the second: the executable name does change, it's python3.14t rather than python3.14. However, depending on the install method the plain python or python3 names may be aliased to a free-threaded interpreter. If you're only looking at the 3.xx-named ones, it should be robust.

For the first question: the context here is that spack external find ... can locate non-Spack installed packages from the system and register them in Spack's database. We can detect whether or not Python was built with +freethreading using https://docs.python.org/3/howto/free-threading-python.html#identifying-free-threaded-python. This would go in the determine_variants function.

For the second question, we currently check for pythonX.Y, pythonX, and python in that order. I would propose also adding pythonX.Yt to that list, possibly in the front.

Thanks for the context - sounds like a good plan to add that then!

@spackbot run pipeline

I've started that pipeline for you!

I'm testing this now and it's mostly working, although there are a ton of packages that will need updating. Many packages use the limited CPython API, which is not compatible with free-threading. This has mostly been fixed in newer versions, but not always.

I'm testing this now and it's mostly working, although there are a ton of packages that will need updating. Many packages use the limited CPython API, which is not compatible with free-threading. This has mostly been fixed in newer versions, but not always.

It's usually not hard to fix upstream. If there are packages that are important/blocking, I'm happy to help with those.

I gave this a spin, all seems to be in order. Not all packages support free-threading or 3.14 yet, but a lot do - this built fine:

spack:
  specs:
  - [email protected]+freethreading
  - py-numpy
  - py-scipy
  - py-scikit-learn
  - [email protected]
  - py-pytest
  - py-hypothesis
  - py-cython
  - py-cffi
  - meson
  - ninja
  - py-aiohttp
  - py-pandas
  - py-pywavelets
  - py-sentencepiece
  - py-tornado
  view: true
  concretizer:
    unify: true

The main issue I ran into is too old versions being pulled in (e.g., matplotlib, swig) but that's easy to resolve when you see it.

I just submitted a bunch of PRs to fix 3.14 support. After those are merged, I'll try to get the TorchGeo deps building with 3.14t.