Thanks to visit codestin.com
Credit goes to github.com

Skip to content

test_peg_generator fails on a PGO+LTO build with clang #117889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
vstinner opened this issue Apr 15, 2024 · 6 comments
Closed

test_peg_generator fails on a PGO+LTO build with clang #117889

vstinner opened this issue Apr 15, 2024 · 6 comments
Labels
tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@vstinner
Copy link
Member

vstinner commented Apr 15, 2024 •
edited by bedevere-app bot
Loading

Bug report

Example on aarch64 Debian Clang LTO + PGO 3.x: https://buildbot.python.org/all/#/builders/1084/builds/4019

0:02:04 load avg: 5.35 [290/473/1] test_peg_generator failed (22 errors) -- running (2): test_io (32.6 sec), test_socket (32.8 sec)
(...)
Successfully installed setuptools-67.6.1 wheel-0.43.0
(...)
test_ternary_operator (test.test_peg_generator.test_c_parser.TestCParser.test_ternary_operator) ... warning: profile data may be out of date: of 9 functions, 1 has mismatched data that will be ignored [-Wprofile-instr-out-of-date]
1 warning generated.
./parse.o: file not recognized: file format not recognized
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ERROR

======================================================================
ERROR: test_ternary_operator (test.test_peg_generator.test_c_parser.TestCParser.test_ternary_operator)
----------------------------------------------------------------------
(...)
distutils.errors.LinkError: command '/usr/local/bin/clang' failed with exit code 1

Linked PRs
@vstinner vstinner added type-bug An unexpected behavior, bug, or error tests Tests in the Lib/test dir labels Apr 15, 2024
@vstinner
Copy link
Member Author

I can reproduce the issue on Fedora 39 with:

./configure --prefix $PWD/target CC=clang LD=clang --with-lto --enable-optimizations
time make
./python -m test test_peg_generator -v -m test_ternary_operator -u all

@vstinner
Copy link
Member Author

It seems like it's a regression introduced by the change: commit 1ddbeae.

commit 1ddbeae040beb35614f3933f340e0dedcc1e98e4
Author: Erlend E. Aasland <[email protected]>
Date:   Fri Apr 12 11:36:40 2024 +0200

    gh-117752: Autoconf: fix PGO builds for 'make -C build' incantations (#117803)

 configure    |  8 ++++----
 configure.ac | 16 ++++++++++++----
 2 files changed, 16 insertions(+), 8 deletions(-)

cc @erlend-aasland

@erlend-aasland
Copy link
Contributor

Yes, I suspected this. Perhaps we should revert that change and just don't support the make -C build scenario.

@erlend-aasland
Copy link
Contributor

I still think we should keep 396b831, though.

@vstinner
Copy link
Member Author

For git bisect, I used:

git clean -fdx
./configure --prefix $PWD/target CC=clang LD=clang --enable-optimizations PROFILE_TASK="-c pass" CFLAGS="-O0" --with-lto
time make
./python -m test test_peg_generator -v -m test_ternary_operator -u all

For old commits, it seems like test_ternary_operator() didn't exist, so instead I ran:

./python -m test test_peg_generator -u all -v

vstinner added a commit to vstinner/cpython that referenced this issue Apr 15, 2024
@vstinner
pythongh-117889: Fix PGO test in test_peg_generator
03007ba 
Reuse support.check_cflags_pgo() in test_peg_generator to check if
PGO build is used or not.
@bedevere-app bedevere-app bot mentioned this issue Apr 15, 2024
Merged
@vstinner
Copy link
Member Author

The test_peg_generator check for PGO build no longer works since the variable contains a shell command:

$ ./python
Python 3.13.0a6+ (heads/main:a9107fe5c0, Apr 15 2024, 15:17:09) [Clang 17.0.6 (Fedora 17.0.6-2.fc39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('PGO_PROF_USE_FLAG')
'-fprofile-instr-use="$(shell pwd)/code.profclangd"'

I wrote PR gh-117893 to fix the test.

vstinner added a commit to vstinner/cpython that referenced this issue Apr 15, 2024
@vstinner
pythongh-117889: Fix PGO test in test_peg_generator
b0afdac 
Reuse support.check_cflags_pgo() in test_peg_generator to check if
PGO build is used or not.
vstinner added a commit that referenced this issue Apr 15, 2024
@vstinner
gh-117889: Fix PGO test in test_peg_generator (#117893)
64cd6fc 
Reuse support.check_cflags_pgo() in test_peg_generator to check for
PGO build.

Log PGO_PROF_USE_FLAG in test.pythoninfo.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Apr 15, 2024
@vstinner @miss-islington
pythongh-117889: Fix PGO test in test_peg_generator (pythonGH-117893)
a6a1573 
Reuse support.check_cflags_pgo() in test_peg_generator to check for
PGO build.

Log PGO_PROF_USE_FLAG in test.pythoninfo.
(cherry picked from commit 64cd6fc)

Co-authored-by: Victor Stinner <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Apr 15, 2024
Merged
erlend-aasland pushed a commit that referenced this issue Apr 15, 2024
@miss-islington @vstinner
[3.12] gh-117889: Fix PGO test in test_peg_generator (GH-117893) (#11…
6a4c06b 
…7895)

Reuse support.check_cflags_pgo() in test_peg_generator to check for
PGO build.

Log PGO_PROF_USE_FLAG in test.pythoninfo.
(cherry picked from commit 64cd6fc)

Co-authored-by: Victor Stinner <[email protected]>
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
@vstinner @diegorusso
pythongh-117889: Fix PGO test in test_peg_generator (python#117893)
e015fc7 
Reuse support.check_cflags_pgo() in test_peg_generator to check for
PGO build.

Log PGO_PROF_USE_FLAG in test.pythoninfo.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vstinner @erlend-aasland