The library will import other librairies most of the time (most likely torch) which do not provide support just yet.

Before doing free threaded GIL, supporting correctly zero-copy memoryview is probably going to be easier, even though it's highly unsafe (given that Python can modify everything while it's being processed).

For torch, there are free-threaded PyTorch 2.6 wheels for Linux, so at least on Linux that’s unblocked.

pytorch/pytorch#130249

Not sure about other dependencies, I haven’t started looking closely at this repo yet.

Well that's not really enough to claim support overall in release binaries is it ? (The actual pypi releases doesn't contain the freethreaded support)

Is there any place in pyo3 docs or elsewhere to know what kind of behavior we should be defending against ? In safetensors we keep a lot of references to python objects, I'm wondering how to make sure the behavior is correct.

In total honesty, threading/parallelism is rather useless in this lib, so requiring the GIL doesn't seem that crazy to me. (Any parallelism is always handled at a multiprocessing level, because there's a mutex deep in CUDA which somehow forbids multithreading, by making it abysmally sequential even in non Python).

Well that's not really enough to claim support overall in release binaries is it ?

No definitely not, I’m more talking about experimenting with the free-threaded build being unblocked.

Is there any place in pyo3 docs or elsewhere to know what kind of behavior we should be defending against ?

You can read more here:

https://py-free-threading.github.io/
https://pyo3.rs/v0.24.0/free-threading.html

I’m one of the authors of both of those links - please feel free to open issues if you have questions that aren’t answered, we want to make these docs really good.

In total honesty, threading/parallelism is rather useless in this lib, so requiring the GIL doesn't seem that crazy to me.

One approach you can take is to make it a hard runtime error to use a tensor simultaneously from multiple threads. Depending on what you’re doing you can depend in pyo3’s runtime borrow checking of data stored in pyobjects or implement it yourself using an atomic integer flag.

Keep in mind that the GIL itself is something of a house of cards and relying on it for thread safety can lead to issues. See e.g. this cryptography issue I ran into recently, where we fixed a thread safety issue due to implicitly relying on the GIL in pure python code leading to the possibility of a race to append to a bytestring if there is an unlucky thread switch at the right moment.

Of course implementing things in Rust helps a lot :)

Thanks for the first link, it contained what I was looking for which is more strategy into testing this stuff.

I'm not sure it's totally complete, though, because here we're highly dependant of moving objects across threads (like tensors which are not owned by this lib). In general the rust side code is always almost trivial, while understanding what's valid to do in python and the boundaries is quite hard.

For instance here something like

f = safe_open(...)
t = Threading(whatever_fn, args=(f,))
t.start()

Or anything like that were users are agressively moving things in random order.

I'm having this issue where I need to write on disk the content of memoryview, however I would like to make sure the content is not modified (readonly is not True because tensors) and also afraid of other thread/processes modifying the data before it hits the disk.

Any ideas on how to do this without copying (which is what Iḿ currently doing) ?

I think the issue you’re describing is more or less what Alex Gaynor is talking about here?

https://alexgaynor.net/2022/oct/23/buffers-on-the-edge/

I’m not sure there is a safe way to expose objects implementing the python buffer protocol to rust and expect any safety guarantees. IMO the PyBuffer API in PyO3 should be unsafe and long term we need to think about alternatives to the buffer protocol that are compatible with borrow checking.

Also all of this is equally true with the GIL or without, the free-threaded build just makes these issues easier to trigger.

I’m on vacation this week and probably won’t be able to participate more until next week. Also if you want to set up a call to talk about this stuff I’m happy to do that. My email is on my github profile.

In essence yes, and yes the issue is unrelated to free threading (it's just that it's more likely to enable more real world bugs).

But the issue ultimately always ends up with &[ReadOnlyCell<u8>] to &[u8] safely (so we can write the bytes to disk.)