WIP Sandbox: drop infra and use pinns to configure sysctls #3474

haircommander · 2020-03-24T20:29:30Z

What type of PR is this?

/kind api-change

What this PR does / why we need it:

a new version of #3076 that adapts the sysctl code from crun and copies it into pinns, as it's the only part of libcrun that is needed. This, along with fixing a remaining port-forward bug, should be the last needed to pass tests while dropping infra container (still lots of more refactoring needed)

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

If manage_ns_lifecycle is true, and pod isn't sharing PID namespace, pods will be deployed without an infra container

openshift-ci-robot · 2020-03-24T20:29:33Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: haircommander
To complete the pull request process, please assign giuseppe
You can assign the PR to them by writing /assign @giuseppe in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2020-03-24T20:29:33Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: haircommander
To complete the pull request process, please assign giuseppe
You can assign the PR to them by writing /assign @giuseppe in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

haircommander · 2020-03-24T20:33:21Z

test/sysctl.bats

@@ -0,0 +1,54 @@
+#!/usr/bin/env bats


todo: remove this test, it was just for not needing to run the entire ctr.bats

codecov · 2020-03-24T20:35:07Z

Codecov Report

Merging #3474 into master will increase coverage by 0.08%.
The diff coverage is 40.54%.

@@            Coverage Diff             @@
##           master    #3474      +/-   ##
==========================================
+ Coverage   43.95%   44.03%   +0.08%     
==========================================
  Files         101      101              
  Lines        7770     7807      +37     
==========================================
+ Hits         3415     3438      +23     
- Misses       4056     4072      +16     
+ Partials      299      297       -2

haircommander · 2020-03-24T20:37:20Z

pinns/src/sysctl.c

+  return 0;
+}
+
+static int write_sysctl_to_file (char * sysctl_key, char* sysctl_value)


todo: give @giuseppe credit for this function

could we use directly libcrun or is it overkill? It doesn't need to be linked and it could just be a git submodule

It is kind of overkill. All the autoconf/automake configuration for a 15 line function ultimately doesn't seem worth it.

The interaction with libcrun was also awkward, because we need to call pinns before we write the config.json to disk. To get the sysctls from it, we needed to write a temporary one, then use it to configure the namespaces, then use those namespaces to configure the real one.

Ultimately, I like this flow better.

I thought we could use directly the function without going through the config.json part.

But I agree autoconf/automake for a function is overkill :-)

haircommander · 2020-03-24T21:06:51Z

/test e2e

haircommander · 2020-03-25T17:52:36Z

/test integration_rhel

some changes this includes: allow for the infra container to not be created by the runtime, if sb.NeedsInfra() is false add concept of a 'spoofed' container. This allows us to make calls to ID(), Name(), and Description() without having the runtime create the container using spoof, change checks of podInfraContainer == nil to Spoofed(), as well as re-add Description() calls reorder some infra container creation code add unit tests for NeedsInfra and Spoofed add integration test for drop infra Signed-off-by: Peter Hunt <[email protected]>

Signed-off-by: Peter Hunt <[email protected]>

sysctls are configured in the runtime after the process unshares in the shared namespaces. since we aren't using a runtime, we need to repeat this step. After unsharing, but before binding, configure the sysctls for the namespace. Signed-off-by: Peter Hunt <[email protected]>

Signed-off-by: Peter Hunt <[email protected]>

haircommander · 2020-03-25T20:17:44Z

/retest

openshift-ci-robot · 2020-03-26T21:39:32Z

@haircommander: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-03-26T21:39:47Z

@haircommander: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tedyu · 2020-05-08T18:38:42Z

internal/lib/container_server.go

-	scontainer.SetSpec(&m)
-	scontainer.SetMountPoint(m.Annotations[annotations.MountPoint])
+	var scontainer *oci.Container
+	if sb.NeedsInfra(c.config.ManageNSLifecycle) {


When NeedsInfra() returns true, infra container would still be instantiated.
From the subject of the PR, I thought infra container would be gone.

yeah it's still being used to hold pod level information (and is likely needed to store it for a restore). The first step is to drop the container at the runtime level

tedyu · 2020-05-08T19:35:21Z

server/sandbox_remove.go

-		if err := s.CtrIDIndex().Delete(podInfraContainer.ID()); err != nil {
-			return nil, fmt.Errorf("failed to delete infra container %s in pod sandbox %s from index: %v", podInfraContainer.ID(), sb.ID(), err)
-		}
+	s.ReleaseContainerName(podInfraContainer.Name())


We don't need the 'if podInfraContainer != nil ' check because there would be spoofed container if infra container is not needed.

tedyu · 2020-05-09T03:41:42Z

server/sandbox_run_linux.go


-	if err := s.ContainerStateToDisk(container); err != nil {
-		log.Warnf(ctx, "unable to write containers %s state to disk: %v", container.ID(), err)
+		if err := s.MonitorConmon(container); err != nil {


I noticed the MonitorConmon() call originally on line 638 has been removed.
Do we still want the call here ?

no we've dropped conmonmon

tedyu · 2020-05-09T03:54:23Z

internal/lib/sandbox/namespaces.go

+	namespaces, err := pinFunc(managedNamespaces, cfg, sysctls)
 	if err != nil {
-		return typesAndPaths, nil
+		return typesAndPaths, err


Since err is returned, how useful is typesAndPaths ?

it's not it could be nil

tedyu · 2020-05-09T04:11:28Z

pinns/src/pinns.c

-
-  return 0;
-}
+static int bind_ns(const char *pin_path, const char *ns_name);


Current declaration is:
static int bind_ns(const char *pin_path, const char *filename, const char *ns_name);

filename is no longer needed ?

the way the function works has been changed

openshift-ci-robot · 2020-06-22T19:12:46Z

@haircommander: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/openshift-jenkins/critest_rhel	`7531de8`	link	`/test critest_rhel`
ci/openshift-jenkins/integration_crun_cgroupv2	`7531de8`	link	`/test integration_cgroupv2`
ci/openshift-jenkins/integration_rhel	`7531de8`	link	`/test integration_rhel`
ci/openshift-jenkins/integration_fedora	`7531de8`	link	`/test integration_fedora`
ci/openshift-jenkins/integration_crun	`7531de8`	link	`/test integration_crun`
ci/openshift-jenkins/e2e_features_rhel	`7531de8`	link	`/test e2e_features_rhel`
ci/kata-jenkins	`7531de8`	link	`/test kata-containers`
ci/openshift-jenkins/e2e_crun_cgroupv2	`7531de8`	link	`/test e2e_cgroupv2`
ci/openshift-jenkins/e2e_rhel	`7531de8`	link	`/test e2e_rhel`
ci/prow/images	`7531de8`	link	`/test images`
ci/prow/launch-aws	`7531de8`	link	`/test launch-aws`
ci/prow/e2e-aws	`7531de8`	link	`/test e2e-aws`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

haircommander · 2020-07-21T20:46:16Z

replacing with #3997

haircommander requested review from mrunalp and runcom as code owners March 24, 2020 20:29

openshift-ci-robot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API labels Mar 24, 2020

openshift-ci-robot requested review from rhatdan, saschagrunert, sboeuf and umohnani8 March 24, 2020 20:29

haircommander changed the title ~~Sandbox: drop infra and use pinns to configure sysctls~~ WIP Sandbox: drop infra and use pinns to configure sysctls Mar 24, 2020

openshift-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Mar 24, 2020

haircommander mentioned this pull request Mar 24, 2020

Sandbox: Don't use an infra container in some cases #3076

Closed

haircommander commented Mar 24, 2020

View reviewed changes

haircommander force-pushed the manage-ns-no-infra-sysctl-pinns branch 2 times, most recently from b3197e3 to 1345fb2 Compare March 24, 2020 20:51

haircommander force-pushed the manage-ns-no-infra-sysctl-pinns branch 3 times, most recently from c7da1d4 to 8ea9545 Compare March 25, 2020 17:31

haircommander force-pushed the manage-ns-no-infra-sysctl-pinns branch 4 times, most recently from 55dfe5a to 60e9c99 Compare March 25, 2020 19:41

haircommander added 4 commits March 25, 2020 15:51

spoof container state

12d4a7d

Signed-off-by: Peter Hunt <[email protected]>

port forward: use file instead of pid

546e981

Signed-off-by: Peter Hunt <[email protected]>

error on failed pinns call

e041b3a

Signed-off-by: Peter Hunt <[email protected]>

haircommander force-pushed the manage-ns-no-infra-sysctl-pinns branch from 60e9c99 to b1961cd Compare March 25, 2020 19:54

WIP: set manage ns lifecycle to true by default

7531de8

Signed-off-by: Peter Hunt <[email protected]>

haircommander force-pushed the manage-ns-no-infra-sysctl-pinns branch from b1961cd to 7531de8 Compare March 25, 2020 20:10

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 26, 2020

tedyu reviewed May 8, 2020

View reviewed changes

tedyu reviewed May 9, 2020

View reviewed changes

haircommander mentioned this pull request Jul 21, 2020

WIP: drop the infra container (again!) #3997

Closed

haircommander closed this Jul 21, 2020

WIP Sandbox: drop infra and use pinns to configure sysctls #3474

WIP Sandbox: drop infra and use pinns to configure sysctls #3474

Uh oh!

Conversation

haircommander commented Mar 24, 2020

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Uh oh!

openshift-ci-robot commented Mar 24, 2020

Uh oh!

openshift-ci-robot commented Mar 24, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Mar 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

haircommander commented Mar 24, 2020

Uh oh!

haircommander commented Mar 25, 2020

Uh oh!

haircommander commented Mar 25, 2020

Uh oh!

openshift-ci-robot commented Mar 26, 2020

Uh oh!

openshift-ci-robot commented Mar 26, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tedyu May 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Jun 22, 2020

Uh oh!

haircommander commented Jul 21, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Mar 24, 2020 •

edited

Loading

tedyu May 8, 2020 •

edited

Loading