Make capabilities an option in crio.conf #1546

umohnani8 · 2018-05-02T19:12:56Z

The list of capabilities to be added can be defined in
/etc/crio/crio.conf.

Fixes #1536

Signed-off-by: umohnani8 [email protected]

umohnani8 · 2018-05-02T19:13:09Z

@rhatdan @mrunalp PTAL

mrunalp · 2018-05-02T19:39:03Z

Let us add an integration test that uses this configuration.

rhatdan · 2018-05-02T20:24:59Z

lib/config.go

+	"NET_BIND_SERVICE",
+	"SYS_CHROOT",
+	"KILL",
+	"AUDIT_WRITE",


I propose we remove AUDIT_WRITE from defaults.
/* Allow writing the audit log via unicast netlink socket */

This allows an application to send a message to the audit log system, I think this should be blocked by default.

rhatdan · 2018-05-02T20:25:59Z

lib/config.go

+	"NET_RAW",
+	"SETGID",
+	"SETUID",
+	"SETFCAP",


I propose we remove SETFCAP from defaults, this is only used for installation software, like rpm. The ability to create a setuid or file cap file should be blocked by default.

rhatdan · 2018-05-02T20:27:22Z

lib/config.go

+	"DAC_OVERRIDE",
+	"FSETID",
+	"FOWNER",
+	"MKNOD",


I propose we remove MKNOD from default. This will block the ability for apps to create device nodes. I don't believe that containers should have this by default. This is needed for installation not for running containers in production. The one potential problem with this would be applications creating something like a loopback device.

rhatdan · 2018-05-02T20:32:46Z

cmd/crio/config.go

 cgroup_manager = "{{ .CgroupManager }}"

+# capabilities is the list of capabilities to add and can be modified here.
+# If capabilities below is commented out, the default list of capabilites defined in the


Spelling capabilites

rhatdan · 2018-05-02T20:36:12Z

I think there should be a --default-capabilities flag for CRI-O so distros can specify the list of capabilities to add by default

mrunalp · 2018-05-07T18:53:57Z

test/ctr.bats

+}
+
+@test "ctr with list of capabilities given by user in crio.conf" {
+	start_crio "" "" "" "--default-capabilities CHOWN --default-capabilities KILL" ""


I think we should do --default-capabilities=CHOWN,KILL instead of one argument per capability.

mrunalp · 2018-05-07T18:54:42Z

test/helpers.bash

 	fi
+
+	if [[ -n "$4" ]]; then
+		capabilities="$4"


I think this might just gobble the first in the list as is. Try with set -x.

mrunalp · 2018-05-09T19:56:14Z

We need to set the default capabilities for the sandbox infra container as well.

umohnani8 · 2018-05-14T15:51:19Z

/test all

umohnani8 · 2018-05-14T16:51:44Z

@rhatdan @mrunalp PTAL. The unit tests pass. I don't understand why integration and e2e are failing - looks like something to do with go environment.

rhatdan · 2018-05-14T17:09:18Z

@mrunalp
The error is:

Detected go version: go version go1.10.1 linux/amd64.
Kubernetes requires go1.10.2 or greater.
Please install go1.10.2 or later.

Does this mean we need to update the AMI?

umohnani8 · 2018-05-15T14:33:35Z

/test all

umohnani8 · 2018-05-15T16:43:01Z

/test critest_fedora
/test critest_rhel

umohnani8 · 2018-05-15T19:22:58Z

/test integration_fedora
/test critest_fedora

umohnani8 · 2018-05-15T19:45:29Z

/test critest_rhel

umohnani8 · 2018-05-15T19:50:30Z

/test integration_rhel

umohnani8 · 2018-05-15T20:13:42Z

/test integration_fedora

umohnani8 · 2018-05-16T16:22:39Z

/test integration_fedora

umohnani8 · 2018-05-17T14:19:23Z

/test all

umohnani8 · 2018-05-17T15:42:41Z

/test kata-containers

umohnani8 · 2018-05-17T18:00:38Z

Tests pass!
@rhatdan @mrunalp PTAL

rhatdan · 2018-05-17T18:09:37Z

LGTM

rhatdan · 2018-05-18T11:20:28Z

@runcom @wking @mrunalp PTAL

wking · 2018-05-18T13:42:00Z

test/helpers.bash

 	"$COPYIMG_BINARY" --root "$TESTDIR/crio" $STORAGE_OPTIONS --runroot "$TESTDIR/crio-run" --image-name=quay.io/crio/busybox:latest --import-from=dir:"$ARTIFACTS_PATH"/busybox-image --signature-policy="$INTEGRATION_ROOT"/policy.json
 	"$COPYIMG_BINARY" --root "$TESTDIR/crio" $STORAGE_OPTIONS --runroot "$TESTDIR/crio-run" --image-name=quay.io/crio/stderr-test:latest --import-from=dir:"$ARTIFACTS_PATH"/stderr-test --signature-policy="$INTEGRATION_ROOT"/policy.json
-	"$CRIO_BINARY" ${DEFAULT_MOUNTS_OPTS} ${HOOKS_OPTS} --conmon "$CONMON_BINARY" --listen "$CRIO_SOCKET" --cgroup-manager "$CGROUP_MANAGER" --default-mounts-file "$TESTDIR/containers/mounts.conf" --registry "quay.io" --runtime "$RUNTIME_BINARY" --root "$TESTDIR/crio" --runroot "$TESTDIR/crio-run" $STORAGE_OPTIONS --seccomp-profile "$seccomp" --apparmor-profile "$apparmor" --cni-config-dir "$CRIO_CNI_CONFIG" --cni-plugin-dir "$CRIO_CNI_PLUGIN" --signature-policy "$INTEGRATION_ROOT"/policy.json --image-volumes "$IMAGE_VOLUMES" --pids-limit "$PIDS_LIMIT" --log-size-max "$LOG_SIZE_MAX_LIMIT" --config /dev/null config >$CRIO_CONFIG
+	"$CRIO_BINARY" ${DEFAULT_MOUNTS_OPTS} ${HOOKS_OPTS} --default-capabilities "$capabilities" --conmon "$CONMON_BINARY" --listen "$CRIO_SOCKET" --cgroup-manager "$CGROUP_MANAGER" --default-mounts-file "$TESTDIR/containers/mounts.conf" --registry "quay.io" --runtime "$RUNTIME_BINARY" --root "$TESTDIR/crio" --runroot "$TESTDIR/crio-run" $STORAGE_OPTIONS --seccomp-profile "$seccomp" --apparmor-profile "$apparmor" --cni-config-dir "$CRIO_CNI_CONFIG" --cni-plugin-dir "$CRIO_CNI_PLUGIN" --signature-policy "$INTEGRATION_ROOT"/policy.json --image-volumes "$IMAGE_VOLUMES" --pids-limit "$PIDS_LIMIT" --log-size-max "$LOG_SIZE_MAX_LIMIT" --config /dev/null config >$CRIO_CONFIG


I think the easiest approach to "for a handful of tests I want to set this option" is to use an ADDITIONAL_CRIO_OPTIONS environment variable. For an example, see here and here in the otherwise unrelated #1280. Passing these in as positional arguments seems more brittle than:

ADDITIONAL_CRIO_OPTIONS='--default-capabilities "CHOWN,DAC_OVERRIDE,FSETID,FOWNER,NET_RAW,SETGID,SETUID"' start_crio

The default capabilities needs to be set for all the tests as we are getting rid of some of the capabilities given by the default spec. There is just one test where I change the capabilities to test out that when the user updates the list of capabilities, cri-o uses the new list.

rhatdan · 2018-05-19T10:07:21Z

@umohnani8 Sadly this needs a rebase, also could you check to make sure the seccomp handling is done correctly. IE Seccomp rules are supposed to reflect the current settings in the capabilities list.

Change the configs under testdata to default to not share the pid namespace unless required to. Signed-off-by: umohnani8 <[email protected]>

umohnani8 · 2018-05-21T14:18:12Z

rebased.

umohnani8 · 2018-05-21T14:53:36Z

/test all

umohnani8 · 2018-05-21T19:52:46Z

/test e2e_rhel

The list of capabilities to be added can be defined in /etc/crio/crio.conf. Signed-off-by: umohnani8 <[email protected]>

umohnani8 · 2018-05-22T17:00:44Z

/test all

rhatdan · 2018-05-22T20:57:41Z

LGTM

mrunalp

LGTM

umohnani8 requested review from mrunalp and runcom as code owners May 2, 2018 19:12

k8s-ci-robot added cncf-cla: yes size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 2, 2018

umohnani8 force-pushed the cap branch from e0a89d2 to ba3ee54 Compare May 2, 2018 19:14

rhatdan reviewed May 2, 2018

View reviewed changes

umohnani8 force-pushed the cap branch 4 times, most recently from 0e7672d to 1d7dfd9 Compare May 4, 2018 20:42

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 4, 2018

umohnani8 force-pushed the cap branch 2 times, most recently from 412c344 to 7f2e159 Compare May 7, 2018 15:17

mrunalp reviewed May 7, 2018

View reviewed changes

umohnani8 force-pushed the cap branch 2 times, most recently from 02fd63c to 5b70294 Compare May 7, 2018 20:17

umohnani8 force-pushed the cap branch 4 times, most recently from f696cd8 to 4441f22 Compare May 14, 2018 15:16

umohnani8 force-pushed the cap branch from f25a510 to 8bc05eb Compare May 17, 2018 14:18

wking reviewed May 18, 2018

View reviewed changes

Default to not share the pid namespace

2de918d

Change the configs under testdata to default to not share the pid namespace unless required to. Signed-off-by: umohnani8 <[email protected]>

umohnani8 force-pushed the cap branch from 8bc05eb to 5e69ef4 Compare May 21, 2018 14:17

umohnani8 force-pushed the cap branch from 5e69ef4 to d92284d Compare May 22, 2018 16:12

Make capabilities an option in crio.conf

bb66787

The list of capabilities to be added can be defined in /etc/crio/crio.conf. Signed-off-by: umohnani8 <[email protected]>

umohnani8 force-pushed the cap branch from d92284d to bb66787 Compare May 22, 2018 16:12

mrunalp approved these changes May 23, 2018

View reviewed changes

mrunalp merged commit 7b3dab8 into cri-o:master May 23, 2018

umohnani8 mentioned this pull request May 23, 2018

Make capabilities a crio.conf option. #1536

Closed

umohnani8 deleted the cap branch January 31, 2019 19:29

umohnani8 mentioned this pull request Mar 1, 2019

REQUEST: New membership for umohnani8 kubernetes/org#550

Closed

6 tasks

Make capabilities an option in crio.conf #1546

Make capabilities an option in crio.conf #1546

Uh oh!

Conversation

umohnani8 commented May 2, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

umohnani8 commented May 2, 2018

Uh oh!

mrunalp commented May 2, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rhatdan commented May 2, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrunalp commented May 9, 2018

Uh oh!

umohnani8 commented May 14, 2018

Uh oh!

umohnani8 commented May 14, 2018

Uh oh!

rhatdan commented May 14, 2018

Uh oh!

umohnani8 commented May 15, 2018

Uh oh!

umohnani8 commented May 15, 2018

Uh oh!

umohnani8 commented May 15, 2018

Uh oh!

umohnani8 commented May 15, 2018

Uh oh!

umohnani8 commented May 15, 2018

Uh oh!

umohnani8 commented May 15, 2018

Uh oh!

umohnani8 commented May 16, 2018

Uh oh!

umohnani8 commented May 17, 2018

Uh oh!

umohnani8 commented May 17, 2018

Uh oh!

umohnani8 commented May 17, 2018

Uh oh!

rhatdan commented May 17, 2018

Uh oh!

rhatdan commented May 18, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rhatdan commented May 19, 2018

Uh oh!

umohnani8 commented May 21, 2018

Uh oh!

umohnani8 commented May 21, 2018

Uh oh!

umohnani8 commented May 21, 2018

Uh oh!

umohnani8 commented May 22, 2018

Uh oh!

rhatdan commented May 22, 2018

Uh oh!

mrunalp left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

umohnani8 commented May 2, 2018 •

edited

Loading