rkt · steveej · Dec 18, 2015 · Dec 3, 2015 · Dec 3, 2015 · Dec 3, 2015
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,6 +1,6 @@
 ## vUNRELEASED
 
-rkt vUNRELEASED is an important release with new features like resource isolators in the kvm stage1, bug fixes and improved documentation.
+rkt vUNRELEASED is an important release with new features like resource isolators in the kvm stage1, a new stage1 flavor called *fly*, bug fixes and improved documentation.
 The appc spec version has been updated to v0.7.4
 
 #### New features and UX changes
@@ -9,6 +9,7 @@ The appc spec version has been updated to v0.7.4
 - CPU and memory resource isolators can be specified on the command line to override the limits specified in the image manifest ([#1851](https://github.com/coreos/rkt/pull/1851), [#1874](https://github.com/coreos/rkt/pull/1874)). See rkt's [overriding isolators](https://github.com/coreos/rkt/blob/master/Documentation/subcommands/run.md#overriding-isolators) documentation.
 - CPU and memory resource isolators can now be used within the kvm stage1 ([#1404](https://github.com/coreos/rkt/pull/1404))
 - The `rkt image list` command can now display the image size ([#1865](https://github.com/coreos/rkt/pull/1865)).
+- A new stage1 flavor has been added: fly; and it represents the first experimental implementation of the upcoming rkt fly feature. ([#1833](https://github.com/coreos/rkt/pull/1833))
 
 #### Build improvements
 

diff --git a/Documentation/build-configure.md b/Documentation/build-configure.md
@@ -10,14 +10,15 @@ For a quick help of available parameters, run `./configure --help`.
 #### `--with-stage1-flavors`
 
 This parameter takes a comma-separated list of all the flavors that the build system should assemble.
-Depending on a default stage1 image setup, this list is by default either empty or set to `coreos,kvm` for, respectively, detailed setup and flavor setup.
+Depending on a default stage1 image setup, this list is by default either empty or set to `coreos,kvm,fly` for, respectively, detailed setup and flavor setup.
 Note that specifying this parameter does not necessarily mean that rkt will use them in the end.
 Available flavors are:
 
 - `coreos` - it takes systemd and bash from a CoreOS PXE image; uses systemd-nspawn
 - `kvm` - it takes systemd, bash and other binaries from a CoreOS PXE image; uses lkvm
 - `src` - it builds systemd, takes bash from the host at build time; uses built systemd-nspawn
 - `host` - it takes systemd and bash from host at runtime; uses systemd-nspawn from the host
+- `fly` - chroot-only approach for single-application minimal isolation containers; native Go implementation
 
 The `host` flavor is probably the best suited flavor for distributions that have strict rules about software sources.
 

diff --git a/Documentation/running-fly-stage1.md b/Documentation/running-fly-stage1.md
@@ -0,0 +1,94 @@
+# Running rkt with the *fly* stage1
+
+The *fly* stage1 is an alternative stage1 that runs a single-application ACI with only `chroot`-isolation.
+
+
+## Motivation
+
+The motivation of the fly feature is to add the ability to run applications with full privileges on the host but still benefit from the image management and discovery from rkt.
+The Kubernetes kubelet is one candidate for rkt fly.
+
+
+## How does it work?
+
+In comparison to the default stage1, there is no process manager involved in the stage1.
+
+The rkt application sets up bind mounts for `/dev`, `/proc`, `/sys`, and the user-provided volumes.
+In addition to the bind mounts, An additional *tmpfs* mount is done at `/tmp`.
+After the mounts are set up, rkt `chroot`s to the application's RootFS and finally executes the application.
+
+Here's a comparison of the default and fly stage1:
+
+stage1-coreos.aci:
+
+```
+host OS
+  └─ rkt
+    └─ systemd-nspawn
+      └─ systemd
+        └─ chroot
+          └─ user-app1
+```
+
+
+stage1-fly.aci:
+
+```
+host OS
+  └─ rkt
+    └─ chroot
+      └─ user-app1
+```
+
+### Mount propagation modes
+The *fly* stage1 makes use of Linux' [mount propagation modes](https://www.kernel.org/doc/Documentation/filesystems/sharedsubtree.txt).
+If a volume source path is a mountpoint on the host, this mountpoint is made recursively shared before the host path is mounted on the target path in the container.
+Hence, changes to the mounts on the target mount path inside the container will be propagated back to the host.
+
+The bind mounts for `/dev`, `/proc`, and `/sys` are done automatically and are recursive, because their hierarchy contains mounts which also need to be available for the container to function properly.
+User provided volumes are not mounted recursively.
+This is a safety measure to prevent system crashes when multiple containers are started that mount `/` into the container. 
+
+
+## Getting started
+
+You can either use `stage1-fly.aci` from the official release, or build rkt yourself with the right options:
+
+```
+$ ./autogen.sh && ./configure --with-stage1-flavors=fly && make
+```
+
+For more details about configure parameters, see [configure script parameters documentation](build-configure.md).
+This will build the rkt binary and the stage1-fly.aci in `build-rkt-0.13.0+git/bin/`.
+
+### Selecting stage1 at runtime
+
+Curious readers can read a whole document on how to [choose which stage1.aci to use at runtime](https://github.com/coreos/rkt/blob/master/Documentation/commands.md#use-a-custom-stage-1).
+
+Here is a quick example of how to use a container stage1 named `stage1-fly.aci` in `/usr/local/rkt/`:
+```
+# rkt run --stage1-image=/usr/local/rkt/stage1-fly.aci coreos.com/etcd:v2.0.9
+```
+
+
+## WARNING: missing isolation and security features
+
+The *fly* stage1 does **NOT** support the isolators and security features as the default stage1 does.
+
+Here's an incomplete list of features that are missing:
+- network namespace isolation
+- CPU isolators
+- Memory isolators
+- CAPABILITY bounding
+- SELinux
+
+### Winning missing features back with systemd
+
+If you run systemd on your host, you can [wrap rkt with a systemd unit file](using-rkt-with-systemd.md#advanced-unit-file).
+For more information please consult the systemd manual. 
+
+The following should get you started:
+
+* [systemd.resource-control](http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html) 
+* [systemd.directives](http://www.freedesktop.org/software/systemd/man/systemd.directives.html)
+
diff --git a/Documentation/subcommands/fly.md b/Documentation/subcommands/fly.md
@@ -0,0 +1,6 @@
+# rkt fly
+
+***This subcommand does not exist yet.***
+It will be a synonym for `rkt run --stage1=/path/to/stage1-fly.aci`.
+
+For more information please read about [running the fly stage1](../running-fly-stage1.md).
diff --git a/Makefile b/Makefile
@@ -13,7 +13,7 @@ TOPLEVEL_STAMPS :=
 TOPLEVEL_CHECK_STAMPS :=
 TOPLEVEL_UNIT_CHECK_STAMPS :=
 TOPLEVEL_FUNCTIONAL_CHECK_STAMPS :=
-TOPLEVEL_SUBDIRS := rkt tests stage1
+TOPLEVEL_SUBDIRS := rkt tests stage1 stage1_fly
 
 $(call inc-one,tools/tools.mk)
 $(call inc-many,$(foreach sd,$(TOPLEVEL_SUBDIRS),$(sd)/$(sd).mk))

diff --git a/configure.ac b/configure.ac
@@ -59,7 +59,7 @@ AC_ARG_WITH([stage1-default-flavor],
 
 AC_ARG_WITH([stage1-flavors],
             [AS_HELP_STRING([--with-stage1-flavors],
-                            [comma-separated list of stage1 flavors; choose from 'src', 'coreos', 'host', 'kvm'; default: 'coreos,kvm'])],
+                            [comma-separated list of stage1 flavors; choose from 'src', 'coreos', 'host', 'kvm', 'fly'; default: 'coreos,kvm,fly'])],
             [RKT_STAGE1_FLAVORS="${withval}"],
             [RKT_STAGE1_FLAVORS=auto])
 
@@ -153,7 +153,7 @@ AS_CASE([${RKT_STAGE1_SETUP_KIND}],
                   dnl and set the first flavor in the list as a
                   dnl default unless specified.
                   [AS_VAR_IF([RKT_STAGE1_FLAVORS],[auto],
-                             [RKT_STAGE1_FLAVORS='coreos,kvm'])
+                             [RKT_STAGE1_FLAVORS='coreos,kvm,fly'])
                    AS_VAR_IF([RKT_STAGE1_DEFAULT_FLAVOR],[auto],
                              [RKT_STAGE1_DEFAULT_FLAVOR=`AS_ECHO([${RKT_STAGE1_FLAVORS}]) | cut -d, -f1`])
                    RKT_STAGE1_DEFAULT_NAME=''
@@ -172,7 +172,7 @@ AS_VAR_IF([RKT_STAGE1_FLAVORS_VERSION_OVERRIDE], [auto],
 ## Built stage1 flavors verification
 
 dnl a list of all flavors
-RKT_STAGE1_ALL_FLAVORS=coreos,kvm,host,src
+RKT_STAGE1_ALL_FLAVORS=coreos,kvm,host,src,fly
 
 dnl RKT_ITERATE_FLAVORS iterates all comma-separated flavors stored in
 dnl $1 using an iterator variable $2 and executes body $3.
@@ -188,7 +188,7 @@ dnl additional string to an error message.
 AC_DEFUN([RKT_IS_VALID_FLAVOR],
          [AS_CASE([$1],
                   dnl Correct flavor, nothing to do.
-                  [coreos|kvm|host|src],
+                  [coreos|kvm|host|src|fly],
                           [],
                   dnl Bogus flavor, bail out.
                   [AC_MSG_ERROR([*** unknown stage1 flavor "$1" $2])])])
@@ -231,6 +231,8 @@ RKT_ITERATE_FLAVORS([${RKT_STAGE1_FLAVORS}],[flavor],
                                       RKT_REQ_PROG([BC],[bc],[bc])],
                              [host],
                                      [],
+                             [fly],
+                                     [AC_MSG_WARN([* fly is an experimental stage1 implementation with almost no isolation and less features])],
                              [AC_MSG_ERROR([*** Unhandled flavor "${flavor}", should not happen])])])
 
 dnl Validate passed default flavor, it should be one of the built

diff --git a/stage1/common/run.go b/stage1/common/run.go
@@ -0,0 +1,50 @@
+// Copyright 2015 The rkt Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package stage1_common
+
+import (
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+
+	"github.com/coreos/rkt/pkg/sys"
+)
+
+func WithClearedCloExec(lfd int, f func() error) error {
+	err := sys.CloseOnExec(lfd, false)
+	if err != nil {
+		return err
+	}
+	defer sys.CloseOnExec(lfd, true)
+
+	return f()
+}
+
+func WritePpid(pid int) error {
+	// write ppid file as specified in
+	// Documentation/devel/stage1-implementors-guide.md
+	out, err := os.Getwd()
+	if err != nil {
+		return fmt.Errorf("cannot get current working directory: %v\n", err)
+	}
+	// we are the parent of the process that is PID 1 in the container so we write our PID to "ppid"
+	err = ioutil.WriteFile(filepath.Join(out, "ppid"),
+		[]byte(fmt.Sprintf("%d\n", pid)), 0644)
+	if err != nil {
+		return fmt.Errorf("cannot write ppid file: %v\n", err)
+	}
+	return nil
+}
diff --git a/stage1/common/types/pod.go b/stage1/common/types/pod.go
@@ -0,0 +1,80 @@
+// Copyright 2015 The rkt Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package types
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+
+	"github.com/coreos/rkt/common"
+
+	"github.com/coreos/rkt/Godeps/_workspace/src/github.com/appc/spec/schema"
+	"github.com/coreos/rkt/Godeps/_workspace/src/github.com/appc/spec/schema/types"
+)
+
+// Pod encapsulates a PodManifest and ImageManifests
+type Pod struct {
+	Root               string // root directory where the pod will be located
+	UUID               types.UUID
+	Manifest           *schema.PodManifest
+	Images             map[string]*schema.ImageManifest
+	MetadataServiceURL string
+	Networks           []string
+}
+
+// LoadPod loads a Pod Manifest (as prepared by stage0) and
+// its associated Application Manifests, under $root/stage1/opt/stage1/$apphash
+func LoadPod(root string, uuid *types.UUID) (*Pod, error) {
+	p := &Pod{
+		Root:   root,
+		UUID:   *uuid,
+		Images: make(map[string]*schema.ImageManifest),
+	}
+
+	buf, err := ioutil.ReadFile(common.PodManifestPath(p.Root))
+	if err != nil {
+		return nil, fmt.Errorf("failed reading pod manifest: %v", err)
+	}
+
+	pm := &schema.PodManifest{}
+	if err := json.Unmarshal(buf, pm); err != nil {
+		return nil, fmt.Errorf("failed unmarshalling pod manifest: %v", err)
+	}
+	p.Manifest = pm
+
+	for i, app := range p.Manifest.Apps {
+		ampath := common.ImageManifestPath(p.Root, app.Name)
+		buf, err := ioutil.ReadFile(ampath)
+		if err != nil {
+			return nil, fmt.Errorf("failed reading app manifest %q: %v", ampath, err)
+		}
+
+		am := &schema.ImageManifest{}
+		if err = json.Unmarshal(buf, am); err != nil {
+			return nil, fmt.Errorf("failed unmarshalling app manifest %q: %v", ampath, err)
+		}
+
+		if _, ok := p.Images[app.Name.String()]; ok {
+			return nil, fmt.Errorf("got multiple definitions for app: %v", app.Name)
+		}
+		if app.App == nil {
+			p.Manifest.Apps[i].App = am.App
+		}
+		p.Images[app.Name.String()] = am
+	}
+
+	return p, nil
+}
diff --git a/stage1/init/kvm/mount.go → stage1/init/common/kvm_mount.go b/stage1/init/kvm/mount.go → stage1/init/common/kvm_mount.go
@@ -33,7 +33,7 @@
 // - bind mounting is realized by appToSystemdMountUnits (for each app),
 //   which creates mount.units (bind) required and ordered before particular application
 // note: systemd mount units require /usr/bin/mount
-package kvm
+package common
 
 import (
 	"fmt"
@@ -47,7 +47,6 @@ import (
 	"github.com/coreos/rkt/Godeps/_workspace/src/github.com/appc/spec/schema/types"
 	"github.com/coreos/rkt/Godeps/_workspace/src/github.com/coreos/go-systemd/unit"
 	"github.com/coreos/rkt/common"
-	initcommon "github.com/coreos/rkt/stage1/init/common"
 )
 
 const (
@@ -162,7 +161,7 @@ func AppToSystemdMountUnits(root string, appName types.ACName, volumes []types.V
 		vols[v.Name] = v
 	}
 
-	mounts := initcommon.GenerateMounts(ra, vols)
+	mounts := GenerateMounts(ra, vols)
 	for _, m := range mounts {
 		vol := vols[m.Volume]
 
@@ -212,7 +211,7 @@ func AppToSystemdMountUnits(root string, appName types.ACName, volumes []types.V
 		// TODO(iaguis) when we update util-linux to 2.27, this code can go
 		// away and we can bind-mount RO with one unit file.
 		// http://ftp.kernel.org/pub/linux/utils/util-linux/v2.27/v2.27-ReleaseNotes
-		if initcommon.IsMountReadOnly(vol, app.MountPoints) {
+		if IsMountReadOnly(vol, app.MountPoints) {
 			opts := []*unit.UnitOption{
 				unit.NewUnitOption("Unit", "Description", fmt.Sprintf("Remount read-only unit for %s", wherePath)),
 				unit.NewUnitOption("Unit", "DefaultDependencies", "false"),

diff --git a/stage1/init/kvm/mount_test.go → stage1/init/common/kvm_mount_test.go b/stage1/init/kvm/mount_test.go → stage1/init/common/kvm_mount_test.go
@@ -12,7 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-package kvm
+package common
 
 import (
 	"testing"