gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore #110108

zooba · 2023-09-29T14:00:15Z

Issue: test_tools: test_freeze_simple_script() fails on s390x SLES 3.x: [Errno 2] No such file or directory: (...) #109615

…src_ignore

Lib/test/test_support.py

vstinner · 2023-09-29T14:12:12Z

I didn't expect that finding the location of the Python source code and checking if it's actually available or not would be so complicated!

At least, it's good, we are finding and fixing issues :-)

For now, I'm ok to keep the code in test_support.py, but later we might share more code in support\__init__.py.

test_freeze and test_venv copy the stdlib. test_venv only cares about Python code, whereas test_freeze wants the C code.

vstinner · 2023-09-29T14:12:59Z

The last change makes test_copy_python_src_ignore fail rather than skip on Windows installs,

Oh. How did you notice that? Is Azure Pipelines running tests which such use cases, or did you discover the issue manually?

vstinner

Your PR now includes unrelated change. Is it on purpose?

zooba · 2023-09-29T14:16:11Z

I didn't expect that finding the location of the Python source code and checking if it's actually available or not would be so complicated!

Yeah, there's a reason I listed this on my wishlist for a new build system. Would be great if we just always built an install-like layout, rather than trying to intermingle it with the source tree.

The pybuilddir.txt file should always be there in a build, even on Windows, adjacent to the main executable (approximately, os.path.realpath(sys.orig_argv[0])). The official check is in getpath.py, and it sets the _is_python_build config value. I forget where that ends up precisely.

zooba · 2023-09-29T14:17:37Z

Your PR now includes unrelated change.

Nope. That's what I get for trying to fix OpenSSL stuff 😆

zooba · 2023-09-29T14:21:00Z

Oh. How did you notice that? Is Azure Pipelines running tests which such use cases, or did you discover the issue manually?

It should, but they seem to have been passing for some reason. I haven't looked closely, but possibly they got disabled recently because people want to move to GHA.

I also run my own daily tests at work, and it was found in the "layout test", which generates an install layout (python PC/layout --preset-default --copy <DIR>), copies it to a brand new machine, and then runs the test suite. That was where I noticed it first.

vstinner

LGTM

miss-islington · 2023-09-29T15:24:42Z

Thanks @zooba for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

miss-islington · 2023-09-29T15:24:44Z

Sorry, @zooba, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 20bc5f7c28a6f8a2e156c4a748ffabb5efc7c761 3.12

bedevere-app · 2023-09-29T15:24:58Z

GH-110110 is a backport of this pull request to the 3.11 branch.

…src_ignore (pythonGH-110108) (cherry picked from commit 20bc5f7) Co-authored-by: Steve Dower <[email protected]>

zooba · 2023-09-29T15:25:48Z

I'm guessing 3.12 has conflicts because the earlier PRs haven't been merged yet? I'll wait until that's done and then trigger it again

vstinner · 2023-09-29T15:25:56Z

Sorry, @zooba, I could not cleanly backport this to 3.12 due to a conflict.

Last days, I gave up on backports since 3.12 is blocked. I plan to try to backport my Tests and Docs changes once 3.12 will be reopened again next week. Until that, it's a backport nightmare :-(

…nore (GH-110108) (cherry picked from commit 20bc5f7) Co-authored-by: Steve Dower <[email protected]>

…src_ignore (pythonGH-110108) (cherry picked from commit 20bc5f7)

…#110340) * gh-109615: Fix support test_copy_python_src_ignore() (#109958) Fix the test when run on an installed Python: use "abs_srcdir" of sysconfig, and skip the test if the Python source code cannot be found. * Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and Lib/test/libregrtest/utils.py now first try to get "abs_srcdir" from sysconfig, before getting "srcdir" from sysconfig. * test.pythoninfo logs sysconfig "abs_srcdir". (cherry picked from commit b89ed9d) * gh-109615: Fix support test_copy_python_src_ignore() on WASM (#109970) Not only check if src_dir exists, but look also for Lib/os.py landmark. (cherry picked from commit cc54bcf) * gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore (GH-110108) (cherry picked from commit 20bc5f7) * gh-109748: Fix again venv test_zippath_from_non_installed_posix() (#110149) Call also copy_python_src_ignore() on listdir() names. shutil.copytree(): replace set() with an empty tuple. An empty tuple becomes a constant in the compiler and checking if an item is in an empty tuple is cheap. (cherry picked from commit 0def8c7) --------- Co-authored-by: Steve Dower <[email protected]>

miss-islington-app · 2023-10-17T06:41:56Z

Thanks @zooba for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington-app · 2023-10-17T06:41:59Z

Sorry, @zooba, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 20bc5f7c28a6f8a2e156c4a748ffabb5efc7c761 3.12

vstinner · 2023-10-17T06:43:48Z

Ah, this fix was backported to 3.12 as part of larger backport: commit 80f9585.

…src_ignore (pythonGH-110108)

pythongh-109615: Look for 'Modules' as landmark for test_copy_python_…

ee811e5

…src_ignore

zooba added tests Tests in the Lib/test dir skip news labels Sep 29, 2023

bedevere-app bot added the awaiting review label Sep 29, 2023

bedevere-app bot mentioned this pull request Sep 29, 2023

test_tools: test_freeze_simple_script() fails on s390x SLES 3.x: [Errno 2] No such file or directory: (...) #109615

Closed

zooba added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Sep 29, 2023

vstinner reviewed Sep 29, 2023

View reviewed changes

Lib/test/test_support.py Show resolved Hide resolved

Add comment

3bf0d92

zooba requested a review from a team as a code owner September 29, 2023 14:13

vstinner reviewed Sep 29, 2023

View reviewed changes

Revert unrelated changes

8deec70

vstinner approved these changes Sep 29, 2023

View reviewed changes

bedevere-app bot added awaiting core review and removed awaiting review labels Sep 29, 2023

zooba merged commit 20bc5f7 into python:main Sep 29, 2023

bedevere-app bot removed the awaiting core review label Sep 29, 2023

zooba deleted the gh-109615 branch September 29, 2023 15:24

miss-islington assigned zooba Sep 29, 2023

bedevere-app bot removed the needs backport to 3.11 only security fixes label Sep 29, 2023

zooba added a commit that referenced this pull request Sep 29, 2023

gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ig…

a95e8cf

…nore (GH-110108) (cherry picked from commit 20bc5f7) Co-authored-by: Steve Dower <[email protected]>

vstinner pushed a commit to vstinner/cpython that referenced this pull request Oct 4, 2023

pythongh-109615: Look for 'Modules' as landmark for test_copy_python_…

4f4c799

…src_ignore (pythonGH-110108) (cherry picked from commit 20bc5f7)

vstinner added needs backport to 3.12 only security fixes and removed needs backport to 3.12 only security fixes labels Oct 17, 2023

vstinner removed the needs backport to 3.12 only security fixes label Oct 18, 2023

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-109615: Look for 'Modules' as landmark for test_copy_python_…

4e5e6f3

…src_ignore (pythonGH-110108)

Uh oh!

gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore #110108

gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore #110108

Uh oh!

Conversation

zooba commented Sep 29, 2023 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

vstinner commented Sep 29, 2023

Uh oh!

vstinner commented Sep 29, 2023

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

zooba commented Sep 29, 2023

Uh oh!

zooba commented Sep 29, 2023

Uh oh!

zooba commented Sep 29, 2023

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

miss-islington commented Sep 29, 2023

Uh oh!

miss-islington commented Sep 29, 2023

Uh oh!

bedevere-app bot commented Sep 29, 2023

Uh oh!

zooba commented Sep 29, 2023

Uh oh!

vstinner commented Sep 29, 2023

Uh oh!

miss-islington-app bot commented Oct 17, 2023

Uh oh!

miss-islington-app bot commented Oct 17, 2023

Uh oh!

vstinner commented Oct 17, 2023

Uh oh!

Uh oh!

zooba commented Sep 29, 2023 •

edited by bedevere-app bot

Loading