Thanks to visit codestin.com
Credit goes to github.com

Skip to content

test_flock/test_lockf are racy #107851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
andreas-schwab opened this issue Aug 10, 2023 · 3 comments
Closed

test_flock/test_lockf are racy #107851

andreas-schwab opened this issue Aug 10, 2023 · 3 comments
Labels
tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@andreas-schwab
Copy link
Contributor

andreas-schwab commented Aug 10, 2023 •
edited by bedevere-app bot
Loading

The tests test_flock and test_lockf are racy and fail spuriously. The parent process trying to synchronize the subprocess may sleep up to max_delay=1.0 seconds between tries, while the subprocess locks the file at most sleep_time=0.2 seconds until it exits (and unlocks the file). This may happen completely while the parent sleeps between two tries.

Linked PRs
@andreas-schwab andreas-schwab added the type-bug An unexpected behavior, bug, or error label Aug 10, 2023
@Eclips4
Copy link
Member

Eclips4 commented Aug 11, 2023

Hello!
Can you give an example of the failure of these tests?
Did you see failure at our buildbot fleet?

@AlexWaygood AlexWaygood added the tests Tests in the Lib/test dir label Aug 11, 2023
@andreas-schwab
Copy link
Contributor Author

You get a timeout while waiting for the synchronisation.

@bedevere-bot bedevere-bot mentioned this issue Aug 14, 2023
Closed
@colesbury
Copy link
Contributor

I've been seeing this fairly often in the nogil refleaks buildbot (even before we disabled the GIL in the CI).

For example, https://buildbot.python.org/all/#/builders/1287/builds/218/steps/5/logs/stdio

======================================================================
ERROR: test_flock (__main__.FNTLEINTRTest.test_flock)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/buildarea/pull_request.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 535, in test_flock
    self._lock(fcntl.flock, "flock")
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/buildarea/pull_request.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 517, in _lock
    raise Exception("failed to sync child in %.1f sec" % dt)
Exception: failed to sync child in 300.4 sec
======================================================================
ERROR: test_lockf (__main__.FNTLEINTRTest.test_lockf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/buildarea/pull_request.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 532, in test_lockf
    self._lock(fcntl.lockf, "lockf")
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/buildarea/pull_request.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 517, in _lock
    raise Exception("failed to sync child in %.1f sec" % dt)
Exception: failed to sync child in 300.6 sec

colesbury added a commit to colesbury/cpython that referenced this issue Jul 9, 2024
@colesbury
pythongh-107851: Fix spurious failures in fcntl eintr tests
90ba394 
On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.

This leads to errors like:

  Exception: failed to sync child in 300.3 sec

Use pipes instead to mutually synchronize between parent and child.
@bedevere-app bedevere-app bot mentioned this issue Jul 9, 2024
Merged
colesbury added a commit that referenced this issue Jul 10, 2024
@colesbury
gh-107851: Fix spurious failures in fcntl eintr tests (#121556)
af9f6de 
On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.

This leads to errors like:

  Exception: failed to sync child in 300.3 sec

Use pipes instead to mutually synchronize between parent and child.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 10, 2024
@colesbury @miss-islington
pythongh-107851: Fix spurious failures in fcntl eintr tests (pythonGH…
c29642d 
…-121556)

On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.

This leads to errors like:

  Exception: failed to sync child in 300.3 sec

Use pipes instead to mutually synchronize between parent and child.
(cherry picked from commit af9f6de)

Co-authored-by: Sam Gross <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 10, 2024
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 10, 2024
@colesbury @miss-islington
pythongh-107851: Fix spurious failures in fcntl eintr tests (pythonGH…
2e03f02 
…-121556)

On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.

This leads to errors like:

  Exception: failed to sync child in 300.3 sec

Use pipes instead to mutually synchronize between parent and child.
(cherry picked from commit af9f6de)

Co-authored-by: Sam Gross <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 10, 2024
Merged
colesbury added a commit that referenced this issue Jul 10, 2024
@miss-islington @colesbury
[3.12] gh-107851: Fix spurious failures in fcntl eintr tests (GH-121556…
847c803 
…) (#121586)

On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.

This leads to errors like:

  Exception: failed to sync child in 300.3 sec

Use pipes instead to mutually synchronize between parent and child.
(cherry picked from commit af9f6de)

Co-authored-by: Sam Gross <[email protected]>
colesbury added a commit that referenced this issue Jul 10, 2024
@miss-islington @colesbury
[3.13] gh-107851: Fix spurious failures in fcntl eintr tests (GH-121556…
0113c56 
…) (#121585)

On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.

This leads to errors like:

  Exception: failed to sync child in 300.3 sec

Use pipes instead to mutually synchronize between parent and child.
(cherry picked from commit af9f6de)

Co-authored-by: Sam Gross <[email protected]>
noahbkim pushed a commit to hudson-trading/cpython that referenced this issue Jul 11, 2024
@colesbury @noahbkim
pythongh-107851: Fix spurious failures in fcntl eintr tests (python#1…
9479c30 
…21556)

On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.

This leads to errors like:

  Exception: failed to sync child in 300.3 sec

Use pipes instead to mutually synchronize between parent and child.
estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024
@colesbury @estyxx
pythongh-107851: Fix spurious failures in fcntl eintr tests (python#1…
d5d0072 
…21556)

On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.

This leads to errors like:

  Exception: failed to sync child in 300.3 sec

Use pipes instead to mutually synchronize between parent and child.
@vstinner vstinner closed this as completed Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@vstinner @colesbury @andreas-schwab @AlexWaygood @Eclips4