-
Notifications
You must be signed in to change notification settings - Fork 1.1k
[release-1.22] utils/RunUnderSystemdScope: fix wrt channel deadlock #6061
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[release-1.22] utils/RunUnderSystemdScope: fix wrt channel deadlock #6061
Conversation
As seen in [1], sometimes coreos/go-systemd/dbus package deadlocks: the jobCompete is stuck trying to send job result string to the channel while holding the jobListener lock, while startJob (called by StartTransientUnit) waits for the same lock. Alas, it is not clear why the channel is not being read, nor was I able to reproduce it locally. Make the job result channel buffered, so jobJistener won't block on channel send and thus StartTransientUnit won't be stuck either. While at it, - move the error wrapping out of mgr.RetryOnDisconnect function, and use fmt.Errorf with %w instead of obsoleted errors.Wrap; - improve error messages, printing the systemd unit name (so we can check it in systemd log); - do check the job result string -- in case it is not "done", return an error back to the caller, which should help avoid other issues down the line. [1] https://bugzilla.redhat.com/show_bug.cgi?id=2082344 Signed-off-by: Kir Kolyshkin <[email protected]>
|
@openshift-cherrypick-robot: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
/lgtm |
|
@haircommander: once the present PR merges, I will cherry-pick it on top of release-1.21 in a new PR and assign it to you. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: haircommander, openshift-cherrypick-robot The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
8 similar comments
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
@haircommander: #6061 failed to apply on top of branch "release-1.21": DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
This is an automated cherry-pick of #5961
/assign haircommander