workloads: set workload for conmon as well #4907

haircommander · 2021-05-17T20:20:23Z

What type of PR is this?

/kind bug

What this PR does / why we need it:

before, we were only setting workload settings for the containers in a pod. This was a gap, as a user may want to have the whole pod-i.e conmon-be confined as well.

Add functionality to put conmon in the workload, configurable by the "POD" label

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Allow users to customize conmon's resources if a pod is in a workload.

haircommander · 2021-05-17T20:27:50Z

still not super happy with it, needs some tweaking of the APIs but I wante to throw it at ci

haircommander · 2021-05-19T16:26:33Z

/retest

haircommander · 2021-05-20T19:56:03Z

I've been puzzling over whether I can come up with a cleaner API between internal/oci/oci_linux and internal/config/cgmgr and I haven't. LinuxResources can be scraped from a spec (so we can reuse the pkg/config/workloads.go function MutateCgroupGivenAnnotation), and holds all the information we may want (configuration of cgroups). if someone can think of something better I'm here to listen, but I think this is ready

haircommander · 2021-05-20T20:14:14Z

/retest

haircommander · 2021-05-21T12:57:06Z

/retest

kolyshkin · 2021-05-21T17:19:17Z

internal/config/cgmgr/systemd.go

+
 	logrus.Debugf("Running conmon under slice %s and unitName %s", cgroupParent, conmonUnitName)
-	if err := utils.RunUnderSystemdScope(pid, cgroupParent, conmonUnitName, killSignalProp, systemdDbus.PropAfter("crio.service")); err != nil {
+	if err := utils.RunUnderSystemdScope(pid, cgroupParent, conmonUnitName, props...); err != nil {


Wonder if it's possible to reuse runc systemd manager's Apply()/Set() here. It does a job similar to utils.RunUnderSystemdScope, can be used to set arbitrary properties (via cgroups.SystemdProps), and in general does a better job.

unfortunatley Apply/Set aren't yet a very portable API (from what I was able to put together). Here's what I had to do to use them for this case: haircommander@82b0997 . I would believe I haven't used the API correctly, but I think I'd like to port this to using runc at a later time (if ever)

haircommander · 2021-05-25T15:39:46Z

/retest

mrunalp

Left some comments.

mrunalp · 2021-06-04T16:52:42Z

server/cri/types/types.go

 	PodSandboxStateSandboxReady    PodSandboxState = 0
 	PodSandboxStateSandboxNotReady PodSandboxState = 1
+
+	InfraName = "POD"


Can we name this InfraContainerName ?

mrunalp · 2021-06-04T16:57:09Z

test/workloads.bats


 	if [ -z "$cpushares" ]; then
-		[[ $(jq -r .linux.resources.cpu.shares < "$config") == 0 ]]
+		[[ $(jq .linux.resources.cpu.shares < "$config") == *"$cpushares"* ]]


Why is the code in both the branches the same?

no idea how that got there haha

mrunalp · 2021-06-04T16:59:23Z

test/workloads.bats

+		else
+			[[ "$cpuset" == *"$found_cpuset"* ]]
+		fi
+		# cgroup_v2 doesn't have an analogue for cpushares


cgroups v2 equivalent is cpu.weight.
See the conversion table here - https://github.com/containers/crun/blob/master/crun.1.md

ah! I attempted to update

haircommander · 2021-06-04T17:45:19Z

/retest

Signed-off-by: Peter Hunt <[email protected]>

there's a line in the kubelet leaky file that says it should be deleted we need an importable InfraName anyway, so move all references from leaky.PodContainerName or infraName to server/cri/types.InfraName Signed-off-by: Peter Hunt <[email protected]>

Signed-off-by: Peter Hunt <[email protected]>

currently, workloads are only configured for the containers in the pod. This is insufficient for clients that want to keep a whole pod confined in a workload. Add functionality where conmon will also be put into the workload Signed-off-by: Peter Hunt <[email protected]>

Signed-off-by: Peter Hunt <[email protected]>

…CPUs Signed-off-by: Peter Hunt <[email protected]>

mrunalp

/lgtm

openshift-ci · 2021-06-07T18:50:31Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: haircommander, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [haircommander,mrunalp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2021-06-07T19:45:35Z

/retest

Please review the full test history for this PR and help us cut down flakes.

haircommander · 2021-06-07T20:40:15Z

/retest

openshift-bot · 2021-06-07T21:11:40Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-06-07T22:59:42Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-06-07T23:12:23Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci · 2021-06-08T00:16:04Z

@haircommander: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/openshift-jenkins/e2e_crun_cgroupv2	`4860dd2`	link	`/test e2e_cgroupv2`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-bot · 2021-06-08T01:23:40Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-06-08T01:47:39Z

/retest

Please review the full test history for this PR and help us cut down flakes.

haircommander · 2021-06-08T13:20:44Z

/cherry-pick release-1.21

openshift-cherrypick-robot · 2021-06-08T13:21:26Z

@haircommander: #4907 failed to apply on top of branch "release-1.21":

Applying: storage: succeed in DeleteContainer if container is unknown
Applying: server: export InfraName and drop references to leaky
Using index info to reconstruct a base tree...
M	internal/oci/oci_linux.go
M	server/sandbox_run_linux.go
M	server/server.go
M	vendor/modules.txt
Falling back to patching base and 3-way merge...
Auto-merging vendor/modules.txt
Removing vendor/k8s.io/kubernetes/pkg/kubelet/leaky/leaky.go
Auto-merging server/server.go
Auto-merging server/sandbox_run_linux.go
Auto-merging internal/oci/oci_linux.go
Applying: Bump runc to get public RangeToBits function
Using index info to reconstruct a base tree...
M	go.mod
M	go.sum
M	vendor/modules.txt
Falling back to patching base and 3-way merge...
Auto-merging vendor/modules.txt
CONFLICT (content): Merge conflict in vendor/modules.txt
Auto-merging go.sum
CONFLICT (content): Merge conflict in go.sum
Auto-merging go.mod
CONFLICT (content): Merge conflict in go.mod
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0003 Bump runc to get public RangeToBits function
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

Details

In response to this:

/cherry-pick release-1.21

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

haircommander requested review from mrunalp and runcom as code owners May 17, 2021 20:20

openshift-ci bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels May 17, 2021

openshift-ci bot requested a review from rhatdan May 17, 2021 20:20

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 17, 2021

haircommander changed the title ~~workloads: set workload for conmon as well~~ wip: workloads: set workload for conmon as well May 17, 2021

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 17, 2021

haircommander force-pushed the workload-settings-7 branch 2 times, most recently from 42f1833 to c589989 Compare May 18, 2021 18:15

haircommander force-pushed the workload-settings-7 branch 2 times, most recently from 8692892 to 67bd771 Compare May 20, 2021 19:43

haircommander changed the title ~~wip: workloads: set workload for conmon as well~~ workloads: set workload for conmon as well May 20, 2021

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 20, 2021

haircommander force-pushed the workload-settings-7 branch from 184350f to 00b5c19 Compare May 20, 2021 20:01

haircommander force-pushed the workload-settings-7 branch from 00b5c19 to 569eeeb Compare May 21, 2021 14:12

kolyshkin reviewed May 21, 2021

View reviewed changes

haircommander force-pushed the workload-settings-7 branch from 569eeeb to 6213b85 Compare May 21, 2021 20:14

openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 23, 2021

haircommander force-pushed the workload-settings-7 branch from 6213b85 to 78939f2 Compare May 24, 2021 16:31

openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 24, 2021

haircommander force-pushed the workload-settings-7 branch 2 times, most recently from ad21927 to 66cedc5 Compare May 27, 2021 18:00

mrunalp reviewed Jun 4, 2021

View reviewed changes

haircommander force-pushed the workload-settings-7 branch 3 times, most recently from a120519 to 7a4bc63 Compare June 4, 2021 17:31

haircommander force-pushed the workload-settings-7 branch from 7a4bc63 to 6035b9a Compare June 7, 2021 13:39

haircommander added 6 commits June 7, 2021 14:12

storage: succeed in DeleteContainer if container is unknown

6a3e173

Signed-off-by: Peter Hunt <[email protected]>

Bump runc to get public RangeToBits function

18a3d64

Signed-off-by: Peter Hunt <[email protected]>

test: add test for conmon in workloads

349ee7c

Signed-off-by: Peter Hunt <[email protected]>

workloads: don't set conmon cpuset if systemd doesn't support Allowed…

4860dd2

…CPUs Signed-off-by: Peter Hunt <[email protected]>

haircommander force-pushed the workload-settings-7 branch from 6035b9a to 4860dd2 Compare June 7, 2021 18:12

mrunalp approved these changes Jun 7, 2021

View reviewed changes

openshift-ci bot assigned mrunalp Jun 7, 2021

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 7, 2021

openshift-merge-robot merged commit eedb15e into cri-o:master Jun 8, 2021

This was referenced Jun 8, 2021

[1.21] workloads: set workload for conmon as well #4979

Closed

[1.21] workloads: set workload for conmon as well #4980

Merged

workloads: set workload for conmon as well #4907

workloads: set workload for conmon as well #4907

Uh oh!

Conversation

haircommander commented May 17, 2021

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Uh oh!

haircommander commented May 17, 2021

Uh oh!

haircommander commented May 19, 2021

Uh oh!

haircommander commented May 20, 2021

Uh oh!

haircommander commented May 20, 2021

Uh oh!

haircommander commented May 21, 2021

Uh oh!

kolyshkin May 21, 2021

Choose a reason for hiding this comment

Uh oh!

haircommander May 21, 2021

Choose a reason for hiding this comment

Uh oh!

haircommander commented May 25, 2021

Uh oh!

mrunalp left a comment

Choose a reason for hiding this comment

Uh oh!

mrunalp Jun 4, 2021

Choose a reason for hiding this comment

Uh oh!

mrunalp Jun 4, 2021

Choose a reason for hiding this comment

Uh oh!

haircommander Jun 4, 2021

Choose a reason for hiding this comment

Uh oh!

mrunalp Jun 4, 2021

Choose a reason for hiding this comment

Uh oh!

haircommander Jun 4, 2021

Choose a reason for hiding this comment

Uh oh!

haircommander commented Jun 4, 2021

Uh oh!

mrunalp left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Jun 7, 2021

Uh oh!

openshift-bot commented Jun 7, 2021

Uh oh!

haircommander commented Jun 7, 2021

Uh oh!

openshift-bot commented Jun 7, 2021

Uh oh!

openshift-bot commented Jun 7, 2021

Uh oh!

openshift-bot commented Jun 7, 2021

Uh oh!

openshift-ci bot commented Jun 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-bot commented Jun 8, 2021

Uh oh!

openshift-bot commented Jun 8, 2021

Uh oh!

haircommander commented Jun 8, 2021

Uh oh!

openshift-cherrypick-robot commented Jun 8, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

openshift-ci bot commented Jun 8, 2021 •

edited

Loading