Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Py.exe not working after 3.11rc1 install #96559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
barry-scott opened this issue Sep 4, 2022 · 17 comments
Closed

Py.exe not working after 3.11rc1 install #96559

barry-scott opened this issue Sep 4, 2022 · 17 comments
Assignees
Labels
3.11 only security fixes OS-windows release-blocker type-bug An unexpected behavior, bug, or error

Comments

@barry-scott
Copy link
Contributor

Bug report

I have a Windows 10 VM that I use to build extensions for python.

Previously I had 3.11a3 installed.

After installing both python 3.11rc1 32 and 64 bit versions the py command is not working. Here is what I see in cmd:

: 10:12:15.43 C:\Users\barry
: \\VM-WIN10\barry> py
No suitable Python runtime found
Pass --list (-0) to see all detected environments on your machine

: 10:12:18.43 C:\Users\barry
: \\VM-WIN10\barry> py -0
 -V:3.11          Python 3.11 (64-bit)
 -V:3.11-32       Python 3.11 (32-bit)
 -V:3.10          Python 3.10 (64-bit)
 -V:3.10-32       Python 3.10 (32-bit)
 -V:3.9           Python 3.9 (64-bit)
 -V:3.9-32        Python 3.9 (32-bit)
 -V:3.8           Python 3.8 (64-bit)
 -V:3.8-32        Python 3.8 (32-bit)
 -V:3.7           Python 3.7 (64-bit)
 -V:3.7-32        Python 3.7 (32-bit)
 -V:3.6           Python 3.6 (64-bit)
 -V:3.6-32        Python 3.6 (32-bit)
 -V:3.5
 -V:3.5-32
 -V:3.4
 -V:2.7
: 10:13:49.37 C:\Users\barry
: \\VM-WIN10\barry> type C:\Users\barry\AppData\Local\py.ini
[defaults]
python=3.10-64
python3=3.10-64

Your environment

  • CPython versions tested on: python 3.11.0-rc1
  • Operating system and architecture: windows 10
@barry-scott barry-scott added the type-bug An unexpected behavior, bug, or error label Sep 4, 2022
@AlexWaygood AlexWaygood added the 3.11 only security fixes label Sep 4, 2022
@pfmoore
Copy link
Member

pfmoore commented Sep 4, 2022

This seems to be a behaviour change - the default of "3.10-64" is no longer recognised. It's documented as valid here:

Furthermore it is possible to specify if a 32 or 64 bit implementation shall be requested by adding “-32” or “-64”.

@pablogsal should this be marked as a release blocker?

@mwichmann
Copy link

I had a dig through the current "whatsnew" page and there's no mention of this change, fwiw. I had also run into this issue - the launcher seems to have dropped the -64 suffix, so if you have a version pinned in your py.ini file that uses the -64 syntax, you get into this state where there seem to be no versions. From memory, this changed at the rc1 release.

@barry-scott
Copy link
Contributor Author

By all means drop the need for the -64, but I think it must be supported for existing installations.
The error messages gives the user no clue what is wrong either.

@pablogsal
Copy link
Member

@pablogsal should this be marked as a release blocker?

Hummmm, is not a bad crash but I think it may be important enough that should block the release. I will re-evaluate on release day if is not fixed, but for now it should be a release blocker.

Can someone bisect the problem to point to a specific commit?

@pfmoore
Copy link
Member

pfmoore commented Sep 4, 2022

Can someone bisect the problem to point to a specific commit?

Unlikely, as the py.exe in 3.11 is a complete rewrite, so it's more probable that this behaviour simply wasn't included in the new version. @zooba any thoughts?

@pfmoore
Copy link
Member

pfmoore commented Sep 4, 2022

Note, this seems to only apply to configured defaults in py.ini. Using py -3.10-64 at the command line works fine. This would suggest to me that while this is a regression, its impact is manageable.

@eryksun
Copy link
Contributor

eryksun commented Sep 4, 2022

Currently the "-32" and "-64" suffixes are only implemented for the command line. They're not implemented for shebangs and not for the defaults in the "PY_*" environment variables or "py.ini" files. The "-32" suffix should work for newer 32-bit releases, which use the "-32" suffix in the registry key name. In this case, the launcher won't actually validate that the architecture is 32-bit, but that shouldn't be a problem.

In parseCommandLine() , suffixed version tags are handled by setting search->exclude32Bit ("-64") or search->only32Bit ("-32") and then ignoring the suffix by subtracting 3 from the tag length.

cpython/PC/launcher2.c

Lines 646 to 662 in 9e55685

if (STARTSWITH(L"2") || STARTSWITH(L"3")) {
// All arguments starting with 2 or 3 are assumed to be version tags
search->tag = arg;
search->tagLength = argLen;
search->oldStyleTag = true;
search->restOfCmdLine = tail;
// If the tag ends with -64, we want to exclude 32-bit runtimes
// (If the tag ends with -32, it will be filtered later)
if (argLen > 3) {
if (0 == _compareArgument(&arg[argLen - 3], 3, L"-64", 3)) {
search->tagLength -= 3;
search->exclude32Bit = true;
} else if (0 == _compareArgument(&arg[argLen - 3], 3, L"-32", 3)) {
search->tagLength -= 3;
search->only32Bit = true;
}
}

Handling the suffix could be factored out into a common function. Alternatively, I think it could be moved into performSearch() as a post-processing step for old-style tags, after parseCommandLine(), checkShebang(), and checkDefaults() have been called.

@zooba
Copy link
Member

zooba commented Sep 5, 2022

Defaults picked up from the config files should probably be treated as old-style arguments if they don't contain a slash.

The -64 was dropped because it's too confusing in the presence of ARM64. I opposed adding -64 as a suffix because it seemed likely to become an issue, and was never part of PEP 514 tags anyway (unless someone chose it explicitly).

In this case, it's just the oldStyleTag field that needs to be set. But people should be able to set their defaults to any runtime - not just PythonCore - so we can't just set it all the time.

We should fix this one up. It's a pretty significant takeback. I'll take a look tonight

zooba added a commit to zooba/cpython that referenced this issue Sep 5, 2022
zooba added a commit that referenced this issue Sep 5, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Sep 5, 2022
…-style tags, and adds What's New section (pythonGH-96595)

(cherry picked from commit 80a9bd2)

Co-authored-by: Steve Dower <[email protected]>
@gvanrossum
Copy link
Member

Meta issue — are there any core devs besides Steve working on the codebase for Py? I wouldn’t even know how to find the source code. :-/

@pfmoore
Copy link
Member

pfmoore commented Sep 5, 2022

I’m keeping an eye on things as they come up but I’ve not really looked at the new code in depth.

@pfmoore
Copy link
Member

pfmoore commented Sep 5, 2022

The source is launcher2.c in the PC directory

@eryksun
Copy link
Contributor

eryksun commented Sep 5, 2022

@zooba, why doesn't the new launcher implement the old launcher's check for 32-bit and 64-bit binaries via GetBinaryTypeW(), where possible at least? "SysArchitecture" is only defined for 3.6+. The fallback architecture may be wrong (e.g. KEY_WOW64_64KEY and KEY_WOW64_32KEY are ignored in 32-bit systems), and a fallback isn't possible for per-user installations in HKCU. Due to the latter, 32-bit releases prior to 3.6 can't be selected with the "-32" suffix if they're installed for the current user. Even py -3.5-32 fails in this case (e.g. debug: "Excluding PythonCore/3.5-32 because it doesn't look 32bit"). Why not at least try calling GetBinaryTypeW() on the executable before using the fallback, if any? Is this a conscious decision or an oversight?

miss-islington added a commit that referenced this issue Sep 5, 2022
… tags, and adds What's New section (GH-96595)

(cherry picked from commit 80a9bd2)

Co-authored-by: Steve Dower <[email protected]>
@zooba
Copy link
Member

zooba commented Sep 6, 2022

Conscious decision. There are very few cases that will no longer be handled, and all of them in end-of-life interpreters released by us, as well as many more cases that will simply fail GetBinaryTypeW. The cost of the code isn't worth the few edge cases - I'd much rather hard-code the versions for the specific check (i.e. 2.x-32 for 0<=x<=7 and 3.x-32 for 0<=x<=5) than keep the additional complexity out of the code. (I also want to minimize file system access and rely on registration information.)

But there are vanishingly few 32-bit installs out there anymore, potentially more 32-bit 2.7 installs than any other version. I've never even seen the -32 options discussed outside of our own docs, other than the discussion that asked for -64 instead. I don't have real data on usage, but I find it hard to believe this is a widely used option.

@zooba zooba closed this as completed Sep 6, 2022
Repository owner moved this from Todo to Done in Release and Deferred blockers 🚫 Sep 6, 2022
@mwichmann
Copy link

I use a -32, but it's specifically for testing that something still works with a 32-bit Windows Python, not because I actually have a use for it myself :)

@zooba
Copy link
Member

zooba commented Sep 6, 2022

I use a -32, but it's specifically for testing that something still works with a 32-bit Windows Python

This is a valid use, though, so thanks for mentioning it. But are you testing that something still works on 3.5 or earlier 32-bit?

@mwichmann
Copy link

In our specific case we dropped 3.5 support a number of months ago, so no.

@eryksun
Copy link
Contributor

eryksun commented Sep 6, 2022

Conscious decision

Okay. I haven't used a 32-bit build of Windows in longer than I can remember, so the fallback works fine for me for all-users installations, based on whether the "HKLM\Software" hive is accessed with KEY_WOW64_64KEY or KEY_WOW64_32KEY.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes OS-windows release-blocker type-bug An unexpected behavior, bug, or error
Projects
Development

No branches or pull requests

8 participants