Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Tests: Don't measure CI performance in tests, don't test maximum elapsed time #110666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
vstinner opened this issue Oct 11, 2023 · 1 comment
Closed

Tests: Don't measure CI performance in tests, don't test maximum elapsed time #110666

vstinner opened this issue Oct 11, 2023 · 1 comment

Comments

@vstinner
Copy link
Member

vstinner commented Oct 11, 2023 •
edited by bedevere-app bot
Loading

A test should not measure a CI performance: don't test maximum elapsed time.

Linked PRs
@vstinner
Copy link
Member Author

Example: fail on GHA Windows x86: https://github.com/python/cpython/actions/runs/6476324922/job/17584898557?pr=110663

FAIL: test_terminate (test.test_multiprocessing_spawn.test_threads.WithThreadsTestPool.test_terminate)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\cpython\cpython\Lib\test\_test_multiprocessing.py", line 2688, in test_terminate
    self.assertLess(join.elapsed, 2.0)
AssertionError: 27.2650000000001 not less than 2.0

On the same PR, the same test failed on GHA macOS and GHA Ubuntu jobs.

vstinner added a commit to vstinner/cpython that referenced this issue Oct 11, 2023
@vstinner
pythongh-110666: Fix multiprocessing test_terminate() elapsed
a95feb0 
multiprocessing test_terminate() and test_wait_socket_slow() no
longer test the CI performance: no longer check maximum elapsed time.

Add CLOCK_RES constant: tolerate a difference of 100 ms.
vstinner added a commit to vstinner/cpython that referenced this issue Oct 11, 2023
@vstinner
pythongh-110666: Fix multiprocessing test_terminate() elapsed
8a7e4ca 
multiprocessing test_terminate() and test_wait_socket_slow() no
longer test the CI performance: no longer check maximum elapsed time.

Add CLOCK_RES constant: tolerate a difference of 100 ms.
@bedevere-app bedevere-app bot mentioned this issue Oct 11, 2023
Merged
@vstinner vstinner mentioned this issue Oct 11, 2023
Merged
vstinner added a commit that referenced this issue Oct 11, 2023
@vstinner
gh-110666: Fix multiprocessing test_terminate() elapsed (#110667)
1556f42 
multiprocessing test_terminate() and test_wait_socket_slow() no
longer test the CI performance: no longer check maximum elapsed time.

Add CLOCK_RES constant: tolerate a difference of 100 ms.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 11, 2023
@vstinner @miss-islington
pythongh-110666: Fix multiprocessing test_terminate() elapsed (python…
52b8c19 
…GH-110667)

multiprocessing test_terminate() and test_wait_socket_slow() no
longer test the CI performance: no longer check maximum elapsed time.

Add CLOCK_RES constant: tolerate a difference of 100 ms.
(cherry picked from commit 1556f42)

Co-authored-by: Victor Stinner <[email protected]>
This was referenced Oct 11, 2023
Merged
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 11, 2023
@vstinner @miss-islington
pythongh-110666: Fix multiprocessing test_terminate() elapsed (python…
45f6797 
…GH-110667)

multiprocessing test_terminate() and test_wait_socket_slow() no
longer test the CI performance: no longer check maximum elapsed time.

Add CLOCK_RES constant: tolerate a difference of 100 ms.
(cherry picked from commit 1556f42)

Co-authored-by: Victor Stinner <[email protected]>
vstinner added a commit that referenced this issue Oct 11, 2023
@miss-islington @vstinner
[3.11] gh-110666: Fix multiprocessing test_terminate() elapsed (GH-11…
8ca7a23 
…0667) (#110669)

gh-110666: Fix multiprocessing test_terminate() elapsed (GH-110667)

multiprocessing test_terminate() and test_wait_socket_slow() no
longer test the CI performance: no longer check maximum elapsed time.

Add CLOCK_RES constant: tolerate a difference of 100 ms.
(cherry picked from commit 1556f42)

Co-authored-by: Victor Stinner <[email protected]>
vstinner added a commit that referenced this issue Oct 11, 2023
@miss-islington @vstinner
[3.12] gh-110666: Fix multiprocessing test_terminate() elapsed (GH-11…
18458a5 
…0667) (#110668)

gh-110666: Fix multiprocessing test_terminate() elapsed (GH-110667)

multiprocessing test_terminate() and test_wait_socket_slow() no
longer test the CI performance: no longer check maximum elapsed time.

Add CLOCK_RES constant: tolerate a difference of 100 ms.
(cherry picked from commit 1556f42)

Co-authored-by: Victor Stinner <[email protected]>
Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
@vstinner @Glyphack
pythongh-110666: Fix multiprocessing test_terminate() elapsed (python…
da0dc8a 
…#110667)

multiprocessing test_terminate() and test_wait_socket_slow() no
longer test the CI performance: no longer check maximum elapsed time.

Add CLOCK_RES constant: tolerate a difference of 100 ms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vstinner