BLD: fix build failure with PYTHONSAFEPATH=1 #30731
Conversation
by using relative import and `python -m`. Also fix dependency handling for code generators.
| sys.path.insert(0, os.path.dirname(__file__)) | ||
| import tempita | ||
|
|
||
| sys.path.pop(0) |
There was a problem hiding this comment.
If it works for the other ones, why doesn't from . import tempita work here?
There was a problem hiding this comment.
It's because _build_utils isn't a package, and I don't think it should be a package.
There was a problem hiding this comment.
I'd be pretty surprised if there isn't a way to do what you're trying to do without manipulating sys.path and I'm not a fan of including sys.path hacks in NumPy's build system.
There was a problem hiding this comment.
But looking at the context in the linked issue I guess there isn't a way to avoid this if we want to support the environment variable.
If we do want to support this, then I think we need to test it in our CI. Can you convert one of our CI jobs to set this environment variable in its numpy build?
I'd also appreciate some explanatory comments that point readers to the github issue about this.
There was a problem hiding this comment.
I'd be pretty surprised if there isn't a way to do what you're trying to do without manipulating sys.path and I'm not a fan of including sys.path hacks in NumPy's build system.
It's possible with importlib.util.spec_from_file_location, but I found that the sys.path.insert pattern it more readable. And an rg 'sys\.path' on the main branch gives me multiple results.
If we do want to support this, then I think we need to test it in our CI. Can you convert one of our CI jobs to set this environment variable in its numpy build?
100% agree. Done in b523bd5.
I'd also appreciate some explanatory comments that point readers to the github issue about this.
Done.
There was a problem hiding this comment.
Yeah it's this or spec_from_file_location. In other files in this directory we do use spec_from_file_location, which is more self-contained. I wouldn't really mind it here though if it was only this change. However, it seems like you need to touch many more files to make this work, as well as manipulate PYTHONPATH in _core/meson.build. That is way too much - a fix with a local use of spec_from_file_location will be cleaner, it shouldn't have to touch any other file than this one. So I suggest changing to that.
rgommers
left a comment
There was a problem hiding this comment.
Thanks @GalaxySnail! Overall this looks good and we should be able to get this in soon - a few requests for changes related to the PYTHONSAFEPATH issue.
The changes for depend_files: codegen_files looks useful and correct to me.
| - name: Build wheels | ||
| uses: pypa/cibuildwheel@298ed2fb2c105540f5ed055e8a6ad78d82dd3a7e # v3.3.1 | ||
| env: | ||
| PYTHONSAFEPATH: 1 # make sure the build doesn't depend on implicit unsafe paths in sys.path |
There was a problem hiding this comment.
It's fine to add this env var to a CI job, however this isn't a great place to add it - we need the wheel builds to be as clean as possible. Please add it instead to linux.yml right next to PYTHONOPTIMIZE: 2
Also, can you replace the comment with # regression check for gh-24907?
| sys.path.insert(0, os.path.dirname(__file__)) | ||
| import tempita | ||
|
|
||
| sys.path.pop(0) |
There was a problem hiding this comment.
Yeah it's this or spec_from_file_location. In other files in this directory we do use spec_from_file_location, which is more self-contained. I wouldn't really mind it here though if it was only this change. However, it seems like you need to touch many more files to make this work, as well as manipulate PYTHONPATH in _core/meson.build. That is way too much - a fix with a local use of spec_from_file_location will be cleaner, it shouldn't have to touch any other file than this one. So I suggest changing to that.
by using relative import and
python -m.Also fix dependency handling for code generators.
Fixes gh-24907.