[1.19] Introduce *all of* internal wipe #5032
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6fac0f7 to
efb6616
Compare
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: haircommander, saschagrunert The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
so all tests can properly access the runtime root without hardcoding Signed-off-by: Peter Hunt <[email protected]>
as well as fix a bug where the resource store would cleanup the resource out of order Signed-off-by: Peter Hunt <[email protected]>
or else the cleanup will always fail on cleanup from context deadline Signed-off-by: Peter Hunt <[email protected]>
We may encounter situations where sandboxes gets killed. In this case, we now try to cleanup the network on removal of the sandbox to ensure that no resources (like networks) left stale on the machine. Signed-off-by: Sascha Grunert <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
as it was duplicated pretty much with DeleteContainer Signed-off-by: Peter Hunt <[email protected]>
which allows us to not have to query for the image or pod each time we want to stop or remove it Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
and update tests to use containers with an image Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Sascha Grunert <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
this logically groups two related functions: stopping network and cleaning up namespaces We do this to allow for network stop failures to be retried, as we'll fail to cleanup the network if the network namespace was removed first Signed-off-by: Peter Hunt <[email protected]>
WaitContainerStopped will always fail if the ctx has expired, which it always could be when running the cleanupFuncs on deadline exceeded Signed-off-by: Peter Hunt <[email protected]>
if we are cleaning up a sandbox in parallel, we may run into situations where we cleanup the infra directory before we stop the sandbox's network. That should not be fatal Signed-off-by: Peter Hunt <[email protected]>
This commit refactors a couple of things: - rename ContainerStop to StopContainer (more consistent) - reuses StopContainer more - make StopContainer not query for the container again (every place it's called already has it) - drops the test for StopContainer (it only checks that we query a container, so it's no longer testing anything relevant) in doing so, we also fix a bug where stopping the sandbox would fail because "container is already stopped" Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Before, we would not actually call CNI DEL on sandboxes that failed to restore. This was because we iterated through a map of *successfully* restored sandboxed, and checked if they failed. Instead, we need to rework LoadSandbox a bit to return a sandbox that was loaded. Even if the restore fails, we can pass the shell of a sandbox to the CNI plugin to best-effort cleanup Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
as it should not be a fatal error, as it indicates we've already cleaned up Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
Before we were sequentially calling CNI del before seeing that they failed and calling them as a cleanup func. The problem is this significantly slows down server startup. A process that used to take effectively no times takes a couple of minutes. Instead, we immediately register the cleanup funcs and call Cleanup() in a separate goroutine, and proceeed to allow the server to start Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
as we don't know if the server previously started with namespaces managed, and then switched to have them not be managed Signed-off-by: Peter Hunt <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
efb6616 to
0007aff
Compare
Contributor
|
@haircommander: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
Contributor
|
@haircommander: PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
A friendly reminder that this PR had no activity for 30 days. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind bug
What this PR does / why we need it:
this PR picks commits from ahem #4722 #4796 #4767 #4900 #4929 #4966 #4100 #5006 #5137
It introduces the InternalWipe feature, and which fixes a bug where CNI del (nor any container cleanup function) is retried if it fails (specifically on server startup
Which issue(s) this PR fixes:
Special notes for your reviewer:
Does this PR introduce a user-facing change?