Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[1.25] cgmgr: create cgroups for systemd cgroup driver for dropped infra pods #6930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
openshift-merge-robot merged 2 commits into from
May 22, 2023
Merged

[1.25] cgmgr: create cgroups for systemd cgroup driver for dropped infra pods #6930

openshift-merge-robot merged 2 commits into from
May 22, 2023

Conversation

@haircommander
Copy link
Member

@haircommander haircommander commented May 17, 2023 

 The history here is a bit convoluted. Originally, runc created the cgroup for the infra container.
cAdvisor was built to assume the cgroup for the infra container would be created, and it uses
this to find the network metrics for the pod. When we dropped the infra container, cri-o needed to make this
cgroup so cAdvisor could still find the network metrics.

However, systemd didn't like the way we did it, and would remove the cgroup mid pod creation, which was fixed in
https://github.com/cri-o/cri-o/pull/6196. This actually caused the cgroup to not be created at all, which then caused
the networking metrics to not be gathered at all.

Thus, we do need to create a cgroup underneath the systemd cgroup. Attempt to use a slice for this, as systemd won't require a
process be underneath it.

What type of PR is this?

/kind bug

What this PR does / why we need it:

reverts #6196
fixes #6657

Which issue(s) this PR fixes:

Special notes for your reviewer:

partial cherry-pick of #6856, ignoring the last commit because it's not relevant for 1.25

Does this PR introduce a user-facing change?

Fix a bug where network metrics collection is broken with systemd cgroup driver and dropped infra containers.

@haircommander
cgmgr: create cgroups for systemd cgroup driver for dropped infra pods
7523ed4 
The history here is a bit convoluted. Originally, runc created the cgroup for the infra container.
cAdvisor was built to assume the cgroup for the infra container would be created, and it uses
this to find the network metrics for the pod. When we dropped the infra container, cri-o needed to make this
cgroup so cAdvisor could still find the network metrics.

However, systemd didn't like the way we did it, and would remove the cgroup mid pod creation, which was fixed in
cri-o#6196. This actually caused the cgroup to not be created at all, which then caused
the networking metrics to not be gathered at all.

Thus, we do need to create a cgroup underneath the systemd cgroup. Attempt to use a slice for this, as systemd won't require a
process be underneath it.

Signed-off-by: Peter Hunt <[email protected]>
@haircommander
test/pod.bats: update to current setup
dadb141 
- skip test for cgroupfs
- remove skip for runc 1.0.0-rc11 (very old now)
- drop removal of cgroup parent (not required)

Signed-off-by: Peter Hunt <[email protected]>
@openshift-ci openshift-ci bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. kind/bug Categorizes issue or PR as related to a bug. labels May 17, 2023
@openshift-ci openshift-ci bot requested review from klihub and wgahnagl May 17, 2023 15:00
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 17, 2023
@codecov
Copy link

codecov bot commented May 17, 2023

Codecov Report

Merging #6930 (dadb141) into release-1.25 (7022a8c) will decrease coverage by 0.11%.
The diff coverage is 6.52%.

Additional details and impacted files 
@@               Coverage Diff                @@
##           release-1.25    #6930      +/-   ##
================================================
- Coverage         43.70%   43.59%   -0.11%     
================================================
  Files               122      122              
  Lines             13675    13712      +37     
================================================
+ Hits               5976     5978       +2     
- Misses             7052     7087      +35     
  Partials            647      647

@sohankunkerkar
Copy link
Member

sohankunkerkar commented May 18, 2023

/retest

1 similar comment
@haircommander
Copy link
Member Author

haircommander commented May 18, 2023

/retest

@haircommander
Copy link
Member Author

haircommander commented May 18, 2023

fixes #6805

@umohnani8
Copy link
Member

umohnani8 commented May 18, 2023

LGTM

@haircommander
Copy link
Member Author

haircommander commented May 19, 2023

/retest
@cri-o/cri-o-maintainers this is ready for merge 😃

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 22, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented May 22, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: haircommander, saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [haircommander,saschagrunert]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 76ceef4 into cri-o:release-1.25 May 22, 2023
@haircommander
Copy link
Member Author

haircommander commented May 22, 2023

/cherry-pick release-1.24

@openshift-cherrypick-robot
Copy link

openshift-cherrypick-robot commented May 22, 2023

@haircommander: #6930 failed to apply on top of branch "release-1.24":

Applying: cgmgr: create cgroups for systemd cgroup driver for dropped infra pods
Using index info to reconstruct a base tree...
M	internal/config/cgmgr/cgmgr.go
M	internal/config/cgmgr/cgroupfs.go
M	internal/config/cgmgr/systemd.go
M	server/sandbox_remove.go
M	test/cgroups.bats
Falling back to patching base and 3-way merge...
Auto-merging test/cgroups.bats
Auto-merging server/sandbox_remove.go
Auto-merging internal/config/cgmgr/systemd.go
Auto-merging internal/config/cgmgr/cgroupfs.go
CONFLICT (content): Merge conflict in internal/config/cgmgr/cgroupfs.go
Auto-merging internal/config/cgmgr/cgmgr.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 cgmgr: create cgroups for systemd cgroup driver for dropped infra pods
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
Details

In response to this:

/cherry-pick release-1.24

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@RCarretta RCarretta mentioned this pull request Jun 2, 2023
Closed
@sohankunkerkar sohankunkerkar mentioned this pull request Jul 28, 2023
Closed
@openshift-ci openshift-ci bot mentioned this pull request Apr 29, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@saschagrunert saschagrunert saschagrunert approved these changes

@mrunalp mrunalp Awaiting requested review from mrunalp mrunalp is a code owner

@runcom runcom Awaiting requested review from runcom runcom is a code owner

@klihub klihub Awaiting requested review from klihub

@wgahnagl wgahnagl Awaiting requested review from wgahnagl

Assignees

@saschagrunert saschagrunert

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@haircommander @sohankunkerkar @umohnani8 @openshift-cherrypick-robot @saschagrunert @openshift-merge-robot