clearlinux/rkt
2
0
Fork 0
mirror of https://github.com/clearlinux/rkt.git synced 2026-08-26 18:35:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

*: update to 0.5.0 spec

Browse Source
This commit is contained in:
Jonathan Boulle
2015-03-24 18:49:50 -07:00
parent 6ef34caf20
commit 56530ef74e
10 changed files with 77 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Documentation/architecture.md
+3 -3
View File
@@ -14,8 +14,8 @@ Execution with Rocket is divided into several distinct stages.
The first stage is the actual `rkt` binary itself. When running a container, this binary is responsible for performing a number of initial preparatory tasks:
- Fetching the specified ACIs, including the stage 1 ACI of --stage1-image if specified.
- Generating a Container UUID
- Generating a Container Runtime Manifest
- Generating a Pod UUID
- Generating a Pod Manifest
- Creating a filesystem for the container
- Setting up stage 1 and stage 2 directories in the filesystem
- Unpacking the stage 1 ACI into the container filesystem
@@ -55,7 +55,7 @@ At this point the stage0 execs `/stage1/rootfs/init` with the current working di
The next stage is a binary that the user trusts to set up cgroups, execute processes, and perform other operations as root on the host. This stage has the responsibility of taking the execution group filesystem that was created by stage 0 and creating the necessary cgroups, namespaces and mounts to launch the execution group:
- Generate systemd unit files from the Image and Container Runtime Manifests. The Image Manifest defines the default `exec` specifications of each application; the Container Runtime Manifest defines the ordering of the units, as well as any `exec` overrides.
- Generate systemd unit files from the Image and Pod Manifests. The Image Manifest defines the default `exec` specifications of each application; the Pod Manifest defines the ordering of the units, as well as any `exec` overrides.
- (containing, respectively, the exec specifications of each container and the ordering given by the user)
- Set up any external volumes (undefined at this point)
- nspawn attaching to the bridge and launch the execution group systemd
Documentation/stage1-implementors-guide.md
+15 -15
View File
@@ -12,11 +12,11 @@ Rocket's execution of containers is divided roughly into three separate stages:
This separation of concerns is reflected in the file-system and layout of the composite image prepared by stage 0:
0. Stage 0: `rkt` executable, and the Container Runtime Manifest created at "/var/lib/rkt/containers/$uuid/container".
1. Stage 1: "stage1.aci", made available at "/var/lib/rkt/containers/$uuid/stage1" by `rkt run`.
2. Stage 2: "$app.aci", made available at "/var/lib/rkt/containers/$uuid/stage1/rootfs/opt/stage2/$imageid" by `rkt run`.
0. Stage 0: `rkt` executable, and the Pod Manifest created at "/var/lib/rkt/pods/$uuid/container".
1. Stage 1: "stage1.aci", made available at "/var/lib/rkt/pods/$uuid/stage1" by `rkt run`.
2. Stage 2: "$app.aci", made available at "/var/lib/rkt/pods/$uuid/stage1/rootfs/opt/stage2/$imageid" by `rkt run`.
The stage 1 implementation is what creates the execution environment for the contained applications. This occurs via entrypoints from stage 0 on behalf of `rkt run` and `rkt enter`. These entrypoints are nothing more than executable programs located via Annotations from within the stage 1 ACI manifest, and executed from within the stage 1 of a given container at "/var/lib/rkt/containers/$uuid/stage1/rootfs".
The stage 1 implementation is what creates the execution environment for the contained applications. This occurs via entrypoints from stage 0 on behalf of `rkt run` and `rkt enter`. These entrypoints are nothing more than executable programs located via Annotations from within the stage 1 ACI manifest, and executed from within the stage 1 of a given container at "/var/lib/rkt/pods/$uuid/stage1/rootfs".
Stage 2 is the destination application images and stage 1 is the vehicle for getting us there from stage 0. For any given container instance, the stage 1 may be completely different, allowing for flexibility in containment strategies employed within the same host while utilizing reusable application ACIs.
@@ -25,18 +25,18 @@ Entrypoints
### `rkt run` => "coreos.com/rocket/stage1/run"
0. rkt prepares the container's stage 1 and stage 2 images and Container Runtime Manifest under "/var/lib/rkt/containers/$uuid", acquiring an exclusive advisory lock on the directory.
1. chdirs to "/var/lib/rkt/containers/$uuid"
2. resolves the "coreos.com/rocket/stage1/run" entrypoint via Annotations found within "/var/lib/rkt/containers/$uuid/stage1/manifest"
3. executes the resolved entrypoint relative to "/var/lib/rkt/containers/$uuid/stage1/rootfs"
0. rkt prepares the container's stage 1 and stage 2 images and Pod Manifest under "/var/lib/rkt/pods/$uuid", acquiring an exclusive advisory lock on the directory.
1. chdirs to "/var/lib/rkt/pods/$uuid"
2. resolves the "coreos.com/rocket/stage1/run" entrypoint via Annotations found within "/var/lib/rkt/pods/$uuid/stage1/manifest"
3. executes the resolved entrypoint relative to "/var/lib/rkt/pods/$uuid/stage1/rootfs"
It is the responsibility of this entrypoint to consume the Container Runtime Manifest and execute the constituent apps in the appropriate environments as specified by the Container Runtime Manifest.
It is the responsibility of this entrypoint to consume the Pod Manifest and execute the constituent apps in the appropriate environments as specified by the Pod Manifest.
The environment variable "RKT_LOCK_FD" contains the file descriptor number of the open directory handle for "/var/lib/rkt/containers/$uuid". It is necessary that stage 1 leave this file descriptor open and in its locked state for the duration of the `rkt run`.
The environment variable "RKT_LOCK_FD" contains the file descriptor number of the open directory handle for "/var/lib/rkt/pods/$uuid". It is necessary that stage 1 leave this file descriptor open and in its locked state for the duration of the `rkt run`.
In the bundled rocket stage 1 which includes systemd-nspawn and systemd, the entrypoint is a static Go program found at "/init" within the stage 1 ACI rootfs. The majority of its execution entails generating a systemd-nspawn argument list and writing systemd unit files for the constituent apps before executing systemd-nspawn. Systemd-nspawn then boots the stage 1 systemd with the just-written unit files for launching the contained apps. The "/init" program is essentially a Container Runtime Manifest to systemd-nspawn + systemd.service translator.
In the bundled rocket stage 1 which includes systemd-nspawn and systemd, the entrypoint is a static Go program found at "/init" within the stage 1 ACI rootfs. The majority of its execution entails generating a systemd-nspawn argument list and writing systemd unit files for the constituent apps before executing systemd-nspawn. Systemd-nspawn then boots the stage 1 systemd with the just-written unit files for launching the contained apps. The "/init" program is essentially a Pod Manifest to systemd-nspawn + systemd.service translator.
An alternative stage 1 could forego systemd-nspawn and systemd altogether, or retain these and introduce something like novm or qemu-kvm for greater isolation by first starting a VM. All that is required is an executable at the place indicated by the "coreos.com/rocket/stage1/run" entrypoint which knows how to apply the Container Runtime Manifest and prepared ACI file-systems to good effect.
An alternative stage 1 could forego systemd-nspawn and systemd altogether, or retain these and introduce something like novm or qemu-kvm for greater isolation by first starting a VM. All that is required is an executable at the place indicated by the "coreos.com/rocket/stage1/run" entrypoint which knows how to apply the Pod Manifest and prepared ACI file-systems to good effect.
#### Arguments
@@ -47,9 +47,9 @@ An alternative stage 1 could forego systemd-nspawn and systemd altogether, or re
### `rkt enter` => "coreos.com/rocket/stage1/enter"
0. rkt verifies the container and image to enter are valid and running
1. chdirs to "/var/lib/rkt/containers/$uuid"
2. resolves the "coreos.com/rocket/stage1/enter" entrypoint via Annotations found within "/var/lib/rkt/containers/$uuid/stage1/manifest"
3. executes the resolved entrypoint relative to "/var/lib/rkt/containers/$uuid/stage1/rootfs"
1. chdirs to "/var/lib/rkt/pods/$uuid"
2. resolves the "coreos.com/rocket/stage1/enter" entrypoint via Annotations found within "/var/lib/rkt/pods/$uuid/stage1/manifest"
3. executes the resolved entrypoint relative to "/var/lib/rkt/pods/$uuid/stage1/rootfs"
In the bundled rocket stage 1 the entrypoint is a statically-linked C program found at "/enter" within the stage 1 ACI rootfs. This program enters the namespaces of the systemd-nspawn container's PID 1 before executing the "/diagexec" program which then chroots into the specific application's rootfs loading the application's environment variables as well.
common/common.go
+2 -2
View File
@@ -54,8 +54,8 @@ func Stage1ManifestPath(root string) string {
return filepath.Join(Stage1ImagePath(root), aci.ManifestFile)
}
// ContainerManifestPath returns the path in root to the Container Runtime Manifest
func ContainerManifestPath(root string) string {
// PodManifestPath returns the path in root to the Pod Manifest
func PodManifestPath(root string) string {
return filepath.Join(root, "container")
}
rkt/containers.go
+15 -15
View File
@@ -43,14 +43,14 @@ type container struct {
nets []netinfo.NetInfo // list of networks (name, IP, iface) this container is using
isEmbryo bool // directory starts as embryo before entering preparing state, serves as stage for acquiring lock before rename to prepare/.
isPreparing bool // when locked at containers/prepare/$uuid the container is actively being prepared
isAbortedPrepare bool // when unlocked at containers/prepare/$uuid the container never finished preparing
isPrepared bool // when at containers/prepared/$uuid the container is prepared, serves as stage for acquiring lock before rename to run/.
isExited bool // when locked at containers/run/$uuid the container is running, when unlocked it's exited.
isExitedGarbage bool // when unlocked at containers/exited-garbage/$uuid the container is exited and is garbage
isExitedDeleting bool // when locked at containers/exited-garbage/$uuid the container is exited, garbage, and is being actively deleted
isGarbage bool // when unlocked at containers/garbage/$uuid the container is garbage that never ran
isDeleting bool // when locked at containers/garbage/$uuid the container is garbage that never ran, and is being actively deleted
isPreparing bool // when locked at pods/prepare/$uuid the container is actively being prepared
isAbortedPrepare bool // when unlocked at pods/prepare/$uuid the container never finished preparing
isPrepared bool // when at pods/prepared/$uuid the container is prepared, serves as stage for acquiring lock before rename to run/.
isExited bool // when locked at pods/run/$uuid the container is running, when unlocked it's exited.
isExitedGarbage bool // when unlocked at pods/exited-garbage/$uuid the container is exited and is garbage
isExitedDeleting bool // when locked at pods/exited-garbage/$uuid the container is exited, garbage, and is being actively deleted
isGarbage bool // when unlocked at pods/garbage/$uuid the container is garbage that never ran
isDeleting bool // when locked at pods/garbage/$uuid the container is garbage that never ran, and is being actively deleted
isGone bool // when a container no longer can be located at its uuid anywhere XXX: only set by refreshState()
}
@@ -162,7 +162,7 @@ func newContainer() (*container, error) {
}
// At this point we we have:
// /var/lib/rkt/containers/prepare/$uuid << exclusively locked to indicate "preparing"
// /var/lib/rkt/pods/prepare/$uuid << exclusively locked to indicate "preparing"
return c, nil
}
@@ -695,17 +695,17 @@ func (c *container) getStage1Hash() (*types.Hash, error) {
// getAppsHashes returns a list of the app hashes in the container
func (c *container) getAppsHashes() ([]types.Hash, error) {
crmb, err := c.readFile("container")
pmb, err := c.readFile("pod")
if err != nil {
return nil, err
}
var crm *schema.ContainerRuntimeManifest
if err = json.Unmarshal(crmb, &crm); err != nil {
var pm *schema.PodManifest
if err = json.Unmarshal(pmb, &pm); err != nil {
return nil, err
}
var imgs []types.Hash
for _, app := range crm.Apps {
for _, app := range pm.Apps {
imgs = append(imgs, app.Image.ID)
}
@@ -734,12 +734,12 @@ func (c *container) getAppCount() (int, error) {
return -1, fmt.Errorf("error: only prepared containers can get their app count")
}
b, err := ioutil.ReadFile(common.ContainerManifestPath(c.path()))
b, err := ioutil.ReadFile(common.PodManifestPath(c.path()))
if err != nil {
return -1, fmt.Errorf("error reading container manifest: %v", err)
}
m := schema.ContainerRuntimeManifest{}
m := schema.PodManifest{}
if err = m.UnmarshalJSON(b); err != nil {
return -1, fmt.Errorf("unable to load manifest: %v", err)
}
rkt/enter.go
+4 -4
View File
@@ -111,20 +111,20 @@ func runEnter(args []string) (exit int) {
// getAppImageID returns the image id to enter
// If one was supplied in the flags then it's simply returned
// If the CRM contains a single image, that image's id is returned
// If the CRM has multiple images, the ids and names are printed and an error is returned
// If the PM contains a single image, that image's id is returned
// If the PM has multiple images, the ids and names are printed and an error is returned
func getAppImageID(c *container) (*types.Hash, error) {
if !flagAppImageID.Empty() {
return &flagAppImageID, nil
}
// figure out the image id, or show a list if multiple are present
b, err := ioutil.ReadFile(common.ContainerManifestPath(c.path()))
b, err := ioutil.ReadFile(common.PodManifestPath(c.path()))
if err != nil {
return nil, fmt.Errorf("error reading container manifest: %v", err)
}
m := schema.ContainerRuntimeManifest{}
m := schema.PodManifest{}
if err = m.UnmarshalJSON(b); err != nil {
return nil, fmt.Errorf("unable to load manifest: %v", err)
}
rkt/list.go
+2 -2
View File
@@ -48,12 +48,12 @@ func runList(args []string) (exit int) {
}
if err := walkContainers(includeMostDirs, func(c *container) {
m := schema.ContainerRuntimeManifest{}
m := schema.PodManifest{}
app_zero := ""
if !c.isPreparing && !c.isAbortedPrepare && !c.isExitedDeleting {
// TODO(vc): we should really hold a shared lock here to prevent gc of the container
manifFile, err := c.readFile(common.ContainerManifestPath(""))
manifFile, err := c.readFile(common.PodManifestPath(""))
if err != nil {
stderr("Unable to read manifest: %v", err)
return
rkt/metadata_service.go
+7 -7
View File
@@ -46,7 +46,7 @@ var (
)
type mdsContainer struct {
manifest schema.ContainerRuntimeManifest
manifest schema.PodManifest
apps map[string]*schema.ImageManifest
ip string
}
@@ -253,7 +253,7 @@ func handleContainerAnnotation(w http.ResponseWriter, r *http.Request, c *mdsCon
w.Write([]byte(v))
}
func handleContainerManifest(w http.ResponseWriter, r *http.Request, c *mdsContainer) {
func handlePodManifest(w http.ResponseWriter, r *http.Request, c *mdsContainer) {
defer r.Body.Close()
w.Header().Add("Content-Type", "application/json")
@@ -274,7 +274,7 @@ func handleContainerUID(w http.ResponseWriter, r *http.Request, c *mdsContainer)
w.Write([]byte(uid))
}
func mergeAppAnnotations(im *schema.ImageManifest, cm *schema.ContainerRuntimeManifest) types.Annotations {
func mergeAppAnnotations(im *schema.ImageManifest, cm *schema.PodManifest) types.Annotations {
merged := types.Annotations{}
for _, annot := range im.Annotations {
@@ -449,9 +449,9 @@ func logReq(h func(w http.ResponseWriter, r *http.Request)) http.HandlerFunc {
func makeHandlers() http.Handler {
r := mux.NewRouter()
r.HandleFunc("/containers/", logReq(handleRegisterContainer)).Methods("POST")
r.HandleFunc("/containers/{uid}", logReq(handleUnregisterContainer)).Methods("DELETE")
r.HandleFunc("/containers/{uid}/{app:.*}", logReq(handleRegisterApp)).Methods("PUT")
r.HandleFunc("/pods/", logReq(handleRegisterContainer)).Methods("POST")
r.HandleFunc("/pods/{uid}", logReq(handleUnregisterContainer)).Methods("DELETE")
r.HandleFunc("/pods/{uid}/{app:.*}", logReq(handleRegisterApp)).Methods("PUT")
acRtr := r.Headers("Metadata-Flavor", "AppContainer").
PathPrefix("/acMetadata/v1").Subrouter()
@@ -460,7 +460,7 @@ func makeHandlers() http.Handler {
mr.HandleFunc("/container/annotations/", logReq(containerGet(handleContainerAnnotations)))
mr.HandleFunc("/container/annotations/{name}", logReq(containerGet(handleContainerAnnotation)))
mr.HandleFunc("/container/manifest", logReq(containerGet(handleContainerManifest)))
mr.HandleFunc("/container/manifest", logReq(containerGet(handlePodManifest)))
mr.HandleFunc("/container/uid", logReq(containerGet(handleContainerUID)))
mr.HandleFunc("/apps/{app:.*}/annotations/", logReq(appGet(handleAppAnnotations)))
stage0/run.go
+20 -21
View File
@@ -64,19 +64,19 @@ type PrepareConfig struct {
// configuration parameters needed by Run
type RunConfig struct {
CommonConfig
PrivateNet bool // container should have its own network stack
PrivateNet bool // pod should have its own network stack
SpawnMetadataService bool // launch metadata service
LockFd int // lock file descriptor
Interactive bool // whether the container is interactive or not
Interactive bool // whether the pod is interactive or not
}
// configuration shared by both Run and Prepare
type CommonConfig struct {
Store *cas.Store // store containing all of the configured application images
Stage1Image types.Hash // stage1 image containing usable /init and /enter entrypoints
Images []types.Hash // application images
ContainersDir string // root directory for rocket containers
Debug bool
Store *cas.Store // store containing all of the configured application images
Stage1Image types.Hash // stage1 image containing usable /init and /enter entrypoints
Images []types.Hash // application images
PodsDir string // root directory for rocket pods
Debug bool
}
func init() {
@@ -106,7 +106,7 @@ func MergeEnvs(appEnv *types.Environment, inheritEnv bool, setEnv []string) {
}
}
// Prepare sets up a container based on the given config.
// Prepare sets up a pod based on the given config.
func Prepare(cfg PrepareConfig, dir string, uuid *types.UUID) error {
if cfg.Debug {
log.SetOutput(os.Stderr)
@@ -119,9 +119,8 @@ func Prepare(cfg PrepareConfig, dir string, uuid *types.UUID) error {
return fmt.Errorf("error preparing stage1: %v", err)
}
cm := schema.ContainerRuntimeManifest{
ACKind: "ContainerRuntimeManifest",
UUID: *uuid, // TODO(vc): later appc spec omits uuid from the crm, this is a temp hack.
cm := schema.PodManifest{
ACKind: "PodManifest",
Apps: make(schema.AppList, 0),
}
@@ -146,7 +145,7 @@ func Prepare(cfg PrepareConfig, dir string, uuid *types.UUID) error {
// TODO(vc): leverage RuntimeApp.Name for disambiguating the apps
Name: am.Name,
Image: schema.RuntimeImage{
Name: am.Name,
Name: &am.Name,
ID: img,
},
Annotations: am.Annotations,
@@ -172,13 +171,13 @@ func Prepare(cfg PrepareConfig, dir string, uuid *types.UUID) error {
cdoc, err := json.Marshal(cm)
if err != nil {
return fmt.Errorf("error marshalling container manifest: %v", err)
return fmt.Errorf("error marshalling pod manifest: %v", err)
}
log.Printf("Writing container manifest")
fn := common.ContainerManifestPath(dir)
log.Printf("Writing pod manifest")
fn := common.PodManifestPath(dir)
if err := ioutil.WriteFile(fn, cdoc, 0700); err != nil {
return fmt.Errorf("error writing container manifest: %v", err)
return fmt.Errorf("error writing pod manifest: %v", err)
}
fn = path.Join(dir, common.Stage1IDFilename)
@@ -187,7 +186,7 @@ func Prepare(cfg PrepareConfig, dir string, uuid *types.UUID) error {
}
if useOverlay {
// mark the container as prepared with overlay
// mark the pod as prepared with overlay
f, err := os.Create(filepath.Join(dir, overlayFilename))
if err != nil {
return fmt.Errorf("error writing overlay marker file: %v", err)
@@ -227,14 +226,14 @@ func preparedWithOverlay(dir string) (bool, error) {
}
if !supportsOverlay() {
return false, fmt.Errorf("the container was prepared with overlay but overlay is not supported")
return false, fmt.Errorf("the pod was prepared with overlay but overlay is not supported")
}
return true, nil
}
// Run mounts the right overlay filesystems and actually runs the prepared
// container by exec()ing the stage1 init inside the container filesystem.
// pod by exec()ing the stage1 init inside the pod filesystem.
func Run(cfg RunConfig, dir string) {
useOverlay, err := preparedWithOverlay(dir)
if err != nil {
@@ -243,7 +242,7 @@ func Run(cfg RunConfig, dir string) {
if useOverlay {
// create a separate mount namespace so the overlay mounts are
// unmounted when exiting the container
// unmounted when exiting the pod
if err := syscall.Unshare(syscall.CLONE_NEWNS); err != nil {
log.Fatalf("error unsharing: %v", err)
}
@@ -434,7 +433,7 @@ func overlayRender(cfg RunConfig, img types.Hash, cdir string, dest string) erro
log.Printf("Writing image manifest")
if err := ioutil.WriteFile(filepath.Join(dest, "manifest"), mb, 0700); err != nil {
return fmt.Errorf("error writing container manifest: %v", err)
return fmt.Errorf("error writing pod manifest: %v", err)
}
destRootfs := path.Join(dest, "rootfs")
stage1/init/container.go
+7 -7
View File
@@ -33,16 +33,16 @@ import (
"github.com/coreos/rocket/common"
)
// Container encapsulates a ContainerRuntimeManifest and ImageManifests
// Container encapsulates a PodManifest and ImageManifests
type Container struct {
Root string // root directory where the container will be located
Manifest *schema.ContainerRuntimeManifest
Manifest *schema.PodManifest
Apps map[string]*schema.ImageManifest
MetadataServiceURL string
Networks []string
}
// LoadContainer loads a Container Runtime Manifest (as prepared by stage0) and
// LoadContainer loads a Pod Manifest (as prepared by stage0) and
// its associated Application Manifests, under $root/stage1/opt/stage1/$apphash
func LoadContainer(root string) (*Container, error) {
c := &Container{
@@ -50,14 +50,14 @@ func LoadContainer(root string) (*Container, error) {
Apps: make(map[string]*schema.ImageManifest),
}
buf, err := ioutil.ReadFile(common.ContainerManifestPath(c.Root))
buf, err := ioutil.ReadFile(common.PodManifestPath(c.Root))
if err != nil {
return nil, fmt.Errorf("failed reading container runtime manifest: %v", err)
return nil, fmt.Errorf("failed reading pod manifest: %v", err)
}
cm := &schema.ContainerRuntimeManifest{}
cm := &schema.PodManifest{}
if err := json.Unmarshal(buf, cm); err != nil {
return nil, fmt.Errorf("failed unmarshalling container runtime manifest: %v", err)
return nil, fmt.Errorf("failed unmarshalling pod manifest: %v", err)
}
c.Manifest = cm
stage1/init/registration.go
+2 -2
View File
@@ -26,13 +26,13 @@ import (
)
func registerContainer(c *Container, ip net.IP) error {
cmf, err := os.Open(common.ContainerManifestPath(c.Root))
cmf, err := os.Open(common.PodManifestPath(c.Root))
if err != nil {
return fmt.Errorf("failed opening runtime manifest: %v", err)
}
defer cmf.Close()
pth := fmt.Sprintf("/containers/?ip=%v", ip.To4().String())
pth := fmt.Sprintf("/pods/?ip=%v", ip.To4().String())
if err := httpRequest("POST", pth, cmf); err != nil {
return fmt.Errorf("failed to register container with metadata svc: %v", err)
}