You may want to use the build tag for new wheels build on an already-released version. CFFI recently discovered this feature and is using it on PyPI to update wheels without changing the version due to the recent problems with patchelf and arm64. You can rename the wheel file after creating it and before pushing to PyPI, with no internal changes needed. This would clearly indicate that the Python3.9 wheels were built from a different process than the other ones.

This was just to check for the presence of Python 3.9, not for releasing wheels.

Any progress? FYI, cibuildwheel has supported Python 3.9 on all CI services including Azure since the betas. :)

We produce nightly wheels for manylinux2010, this PR was just to track when cached versions became available. The current schedule looks like Mac tomorrow and Windows next week.

multibuild supports Python 3.9

The manylinux builds all work except for the 32 bit manylinux1 builds that have an unrelated problem on master. Mac and Windows are what we are waiting on. We could do it now with github/actions, or we can wait a bit until it shows up in the usual places. We should be ready to go sometime next week.We are building on Azure which has changed the workflow somewhat.

That might help for posix, let's see if an update helps. I don't think it will make a difference on windows.

Are the Mac builds not working on multibuild?

Apparently not, at least the way we are running the builds on azure.

appveyor

We don't use appveyor. We could use github actions with azure, but things will settle on that front soon enough. I was curious how things would go with this transition and it looks like two weeks. I expect things will go faster when Python 3.10 is released and we could try github actions if there is more delay.

Looks like 3.9 has arrived on azure ubuntu. We didn't need it for the manylinux builds, but it went in on schedule.

Still no 3.9 on windows :(

This is waaaay too long to be without Numpy wheels. Would it be very hard at this stage to resurrect an Appveyor config to build them?

This is waaaay too long to be without Numpy wheels.

@matthew-brett Python 3.9 should be available on Windows sometime tomorrow, if all goes well I will do a release Wednesday. There is also a potential fix for the Windows 2004 problem with the new OpenBLAS that was merged to master today.

Maybe try restarting the CI?

The rollout progress is nearing completion:

• https://github.com/actions/virtual-environments#available-environments

And I had a Windows/3.9 build now pass on another repo:

• https://dev.azure.com/hugovk/hugovk/_build/results?buildId=1981&view=logs&j=3c0a9d00-a287-5cb5-e420-b7a0de833172

The Windows build succeeds and tests run, but the test_cython tests fail. Hmm...

The error is

 C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe
/c /nologo /Ox /W3 /GL /DNDEBUG /MD -DNPY_NO_DEPRECATED_API=0 
-ID:\a\1\s\test_venv\lib\site-packages\numpy\core\include 
-ID:\a\1\s\test_venv\include -IC:\hostedtoolcache\windows\Python\3.9.0\x64\include 
-IC:\hostedtoolcache\window\Python\3.9.0\x64\include 
-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\ATLMFC\include 
-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\include 
-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt 
/TcC:\Users\VssAdministrator\AppData\Local\Temp\pytest-of-VssAdministrator\pytest-0\test_get_timedelta64_value0\cytest\checks.c 
/Fobuild\temp.win-amd64-3.9\Release\Users\VssAdministrator\AppData\Local\Temp\pytest-of-VssAdministrator\pytest-0\test_get_timedelta64_value0\cytest\checks.obj

checks.c

C:\Users\VssAdministrator\AppData\Local\Temp\pytest-of-VssAdministrator\pytest-0\test_get_timedelta64_value0\cytest\checks.c
: fatal error C1083: Cannot open compiler generated file: '': Invalid argument

Stack overflow suggests this may be due to the %TMP% directory name being too long, but maybe the whole command is too long and is getting truncated by subprocess.Popen ?

maybe the whole command is too long and is getting truncated by subprocess.Popen

It seems to have worked for other Python versions in the most recent nightly.

Looking at one of the successful runs, a compilation command line is

C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe 
/c /nologo /Ox /W3 /GL /DNDEBUG /MD 
-Inumpy\core\src\common 
-Inumpy\core\src 
-Inumpy\core 
-Inumpy\core\src\npymath 
-Inumpy\core\src\multiarray 
-Inumpy\core\src\umath 
-Inumpy\core\src\npysort 
-IC:\hostedtoolcache\windows\Python\3.8.6\x64\include 
-IC:\hostedtoolcache\windows\Python\3.8.6\x64\include 
-Ibuild\src.win-amd64-3.8\numpy\core\src\common 
-Ibuild\src.win-amd64-3.8\numpy\core\src\npymath 
-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\ATLMFC\include 
-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\include 
-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt 
-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE 
-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt 
-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um 
-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt 
/TcD:\a\1\s\numpy\numpy\distutils\checks\extra_avx512f_reduce.c 
/FoC:\Users\VSSADM~1\AppData\Local\Temp\tmpk4hn3n74\a\1\s\numpy\numpy\distutils\checks\extra_avx512f_reduce.obj 
/arch:AVX512 /WX

The output file name starts with C:\Users\VSSADM~1\AppData\Local not build\temp.win-amd64-3.9\Release\Users\VssAdministrator\AppData\Local. The latter looks wrong.

Let's give it another shot and see if it remains a problem. I don't know if this is a Python 3.9 problem or something added on by Windows.

Hmm, the failed test uses tmp_path from pytest. Note that the update was not just for Python 3.9, I wonder if the azure environment has changed? I'll do a push to master to check that.

There seems to be a problem with where distutils creates its temp files when building. CPython 3.8 puts the output here

"C:\Users\matti\AppData\Local\Temp\pytest-of-matti\pytest-35\test_is_timedelta64_object0\cytest\build\temp.win-amd64-3.8\Release\checks.obj"

but 3.9 is putting it here, note that the path is duplicated. So it is close to the MAX_PATH character limit

"C:\Users\matti\AppData\Local\Temp\pytest-of-matti\pytest-34\test_is_timedelta64_object0\cytest\build\temp.win-amd64-3.9\Release\Users\matti\AppData\Local\Temp\pytest-of-matti\pytest-34\test_is_timedelta64_object0\cytest\checks.obj"

Apparently you can change the limit with Set-ItemProperty 'HKLM:\System\CurrentControlSet\Control\FileSystem' -Name 'LongPathsEnabled' -value 1 in powershell.

That didn't work. I've cleaned up and rebased on master to continue experiments.

For the 1.19.3 release, I'd be happy to skip those failing tests on Python 3.9.

probably wise to skip them and get this over with

probably wise to skip them and get this over with

I've done that for the 1.19.x branch. I wonder if it is a pytest bug that might be fixed on later releases. We are pinned to an older version because of the hypothesis problem.

Looks to be fixed in master, so closing.

Not sure if relevant, but the mention of pytest brings to mind this bug with long parametrized IDs:

pytest-dev/pytest#6881