Thanks to visit codestin.com
Credit goes to github.com

Skip to content

utf8 mode seems not working while loading site module with Python 3.11/3.12 in Windows #117793

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wkwkw opened this issue Apr 12, 2024 · 1 comment
Closed

utf8 mode seems not working while loading site module with Python 3.11/3.12 in Windows #117793

wkwkw opened this issue Apr 12, 2024 · 1 comment
Assignees
@methane
Labels
3.11 only security fixes 3.12 only security fixes stdlib Python modules in the Lib dir topic-unicode type-bug An unexpected behavior, bug, or error

Comments

@wkwkw
Copy link

wkwkw commented Apr 12, 2024 •
edited by github-actions bot
Loading

Bug report

Bug description:

I installed python 3.12.2 with rye 0.32.0, and it created a pth file that contains non-ascii file path.

$ cat .venv/Lib/site-packages/_tesuto.pth
C:\Users\XXXXXXXXX\Documents\tmp\テスト\src

Processing this file causes a UnicodeDecodeError and python prompt fails to open. (#87380)
I tried running python in utf-8 mode via $ python -X utf8 and PYTHONUTF8=1 environment variable, but not worked.
(not also with 3.11.8)

Error message is like following:

$ python -X utf8
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1176, in exec_module
  File "<frozen site>", line 624, in <module>
  File "<frozen site>", line 607, in main
  File "<frozen site>", line 539, in venv
  File "<frozen site>", line 394, in addsitepackages
  File "<frozen site>", line 236, in addsitedir
  File "<frozen site>", line 188, in addpackage
UnicodeDecodeError: 'cp932' codec can't decode byte 0x86 in position 35: illegal multibyte sequence

related part in messages from -v option:

Processing global site-packages
Adding directory: 'C:\\Users\\XXXXXXXX\\Documents\\tmp\\テスト\\.venv'
Adding directory: 'C:\\Users\\XXXXXXXX\\Documents\\tmp\\テスト\\.venv\\Lib\\site-packages'
Processing .pth file: 'C:\\Users\\XXXXXXXX\\Documents\\tmp\\テスト\\.venv\\Lib\\site-packages\\_tesuto.pth'
# C:\Users\XXXXXXXX\.rye\py\[email protected]\Lib\encodings\__pycache__\cp932.cpython-312.pyc matches C:\Users\XXXXXXXX\.rye\py\[email protected]\Lib\encodings\cp932.py

tried but not worked

  • installing python via pyenv

worked

  • using python 3.10 or earlier
  • changing the encoding of the pth file to cp932

CPython versions tested on:

3.12

Operating systems tested on:

Windows
@wkwkw wkwkw added the type-bug An unexpected behavior, bug, or error label Apr 12, 2024
@arhadthedev arhadthedev added 3.11 only security fixes 3.12 only security fixes stdlib Python modules in the Lib dir topic-unicode labels Apr 12, 2024
@methane methane self-assigned this Apr 12, 2024
@methane
Copy link
Member

methane commented Apr 12, 2024

It seems duplicate of #77102

@methane methane closed this as completed Apr 16, 2024
@sk-ys sk-ys mentioned this issue Jul 6, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@methane methane

Labels
3.11 only security fixes 3.12 only security fixes stdlib Python modules in the Lib dir topic-unicode type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@methane @arhadthedev @wkwkw