Clean out remaining cruft from 'tianon-separate-dockerinit'

Conflicts:
	runtime_test.go

.gitignore

@@ -1,6 +1,3 @@
-# Docker project generated files to ignore
-#  if you want to ignore files created by your editor/tools,
-#  please consider a global .gitignore https://help.github.com/articles/ignoring-files
 .vagrant*
 bin
 docker/docker
@@ -20,5 +17,3 @@ docs/_templates
 bundles/
 .hg/
 .git/
-vendor/pkg/
-pyenv

AUTHORS

@@ -44,7 +44,6 @@ Daniel Nordberg <dnordberg@gmail.com>
 Daniel Robinson <gottagetmac@gmail.com>
 Daniel Von Fange <daniel@leancoder.com>
 Daniel YC Lin <dlin.tw@gmail.com>
-Darren Coxall <darren@darrencoxall.com>
 David Calavera <david.calavera@gmail.com>
 David Sissitka <me@dsissitka.com>
 Deni Bertovic <deni@kset.org>
@@ -70,14 +69,12 @@ Gabriel Monroy <gabriel@opdemand.com>
 Gareth Rushgrove <gareth@morethanseven.net>
 Greg Thornton <xdissent@me.com>
 Guillaume J. Charmes <guillaume.charmes@dotcloud.com>
-Gurjeet Singh <gurjeet@singh.im>
 Guruprasad <lgp171188@gmail.com>
 Harley Laue <losinggeneration@gmail.com>
 Hector Castro <hectcastro@gmail.com>
 Hunter Blanks <hunter@twilio.com>
 Isao Jonas <isao.jonas@gmail.com>
 James Carr <james.r.carr@gmail.com>
-James Turnbull <james@lovedthanlost.net>
 Jason McVetta <jason.mcvetta@gmail.com>
 Jean-Baptiste Barth <jeanbaptiste.barth@gmail.com>
 Jeff Lindsay <progrium@gmail.com>
@@ -95,7 +92,6 @@ Jonathan Rudenberg <jonathan@titanous.com>
 Joost Cassee <joost@cassee.net>
 Jordan Arentsen <blissdev@gmail.com>
 Joseph Anthony Pasquale Holsten <joseph@josephholsten.com>
-Josh Poimboeuf <jpoimboe@redhat.com>
 Julien Barbier <write0@gmail.com>
 Jérôme Petazzoni <jerome.petazzoni@dotcloud.com>
 Karan Lyons <karan@karanlyons.com>
@@ -121,7 +117,6 @@ Marko Mikulicic <mmikulicic@gmail.com>
 Markus Fix <lispmeister@gmail.com>
 Martin Redmond <martin@tinychat.com>
 Matt Apperson <me@mattapperson.com>
-Mathieu Le Marec - Pasquet <kiorky@cryptelium.net>
 Matt Bachmann <bachmann.matt@gmail.com>
 Matthew Mueller <mattmuelle@gmail.com>
 Maxim Treskin <zerthurd@gmail.com>
@@ -145,7 +140,6 @@ odk- <github@odkurzacz.org>
 Pascal Borreli <pascal@borreli.com>
 Paul Bowsher <pbowsher@globalpersonals.co.uk>
 Paul Hammond <paul@paulhammond.org>
-Paul Nasrat <pnasrat@gmail.com>
 Phil Spitler <pspitler@gmail.com>
 Piotr Bogdan <ppbogdan@gmail.com>
 pysqz <randomq@126.com>
@@ -157,7 +151,6 @@ Roberto Hashioka <roberto_hashioka@hotmail.com>
 Ryan Fowler <rwfowler@gmail.com>
 Sam Alba <sam.alba@gmail.com>
 Sam J Sharpe <sam.sharpe@digital.cabinet-office.gov.uk>
-Scott Bessler <scottbessler@gmail.com>
 Sean P. Kane <skane@newrelic.com>
 Shawn Siefkas <shawn.siefkas@meredith.com>
 Shih-Yuan Lee <fourdollars@gmail.com>
@@ -168,7 +161,6 @@ Sridatta Thatipamala <sthatipamala@gmail.com>
 Sridhar Ratnakumar <sridharr@activestate.com>
 Steeve Morin <steeve.morin@gmail.com>
 Stefan Praszalowicz <stefan@greplin.com>
-Sven Dowideit <SvenDowideit@home.org.au>
 Thatcher Peskens <thatcher@dotcloud.com>
 Thermionix <bond711@gmail.com>
 Thijs Terlouw <thijsterlouw@gmail.com>
@@ -193,5 +185,4 @@ Vladimir Kirillov <proger@wilab.org.ua>
 Walter Stanish <walter@pratyeka.org>
 Wes Morgan <cap10morgan@gmail.com>
 Will Dietz <w@wdtz.org>
-Yang Bai <hamo.by@gmail.com>
 Zaiste! <oh@zaiste.net>

CONTRIBUTING.md

@@ -1,21 +1,11 @@
 # Contributing to Docker
 
-Want to hack on Docker? Awesome! Here are instructions to get you
-started. They are probably not perfect, please let us know if anything
-feels wrong or incomplete.
-
-## Reporting Issues
-
-When reporting [issues](https://github.com/dotcloud/docker/issues) 
-on Github please include your host OS ( Ubuntu 12.04, Fedora 19, etc... )
-and the output of `docker version` along with the output of `docker info` if possible.  
-This information will help us review and fix your issue faster.
+Want to hack on Docker? Awesome! Here are instructions to get you started. They are probably not perfect, please let us know if anything feels
+wrong or incomplete.
 
 ## Build Environment
 
-For instructions on setting up your development environment, please
-see our dedicated [dev environment setup
-docs](http://docs.docker.io/en/latest/contributing/devenvironment/).
+For instructions on setting up your development environment, please see our dedicated [dev environment setup docs](http://docs.docker.io/en/latest/contributing/devenvironment/).
 
 ## Contribution guidelines
 
@@ -71,7 +61,7 @@ your branch before submitting a pull request.
 
 Update the documentation when creating or modifying features. Test
 your documentation changes for clarity, concision, and correctness, as
-well as a clean documentation build. See ``docs/README.md`` for more
+well as a clean docmuent build. See ``docs/README.md`` for more
 information on building the docs and how docs get released.
 
 Write clean code. Universally formatted code promotes ease of writing, reading,
@@ -122,7 +112,6 @@ For more details see [MAINTAINERS.md](hack/MAINTAINERS.md)
 * Step 1: learn the component inside out
 * Step 2: make yourself useful by contributing code, bugfixes, support etc.
 * Step 3: volunteer on the irc channel (#docker@freenode)
-* Step 4: propose yourself at a scheduled #docker-meeting
 
 Don't forget: being a maintainer is a time investment. Make sure you will have time to make yourself available.
 You don't have to be a maintainer to make a difference on the project!

Dockerfile

@@ -4,75 +4,78 @@
 #
 # # Assemble the full dev environment. This is slow the first time.
 # docker build -t docker .
+# # Apparmor messes with privileged mode: disable it
+# /etc/init.d/apparmor stop ; /etc/init.d/apparmor teardown
 #
 # # Mount your source in an interactive container for quick testing:
-# docker run -v `pwd`:/go/src/github.com/dotcloud/docker -privileged -i -t docker bash
+# docker run -v `pwd`:/go/src/github.com/dotcloud/docker -privileged -lxc-conf=lxc.aa_profile=unconfined -i -t docker bash
+#
 #
 # # Run the test suite:
-# docker run -privileged docker hack/make.sh test
+# docker run -privileged -lxc-conf=lxc.aa_profile=unconfined docker hack/make.sh test
 #
 # # Publish a release:
-# docker run -privileged \
+# docker run -privileged -lxc-conf=lxc.aa_profile=unconfined \
 #  -e AWS_S3_BUCKET=baz \
 #  -e AWS_ACCESS_KEY=foo \
 #  -e AWS_SECRET_KEY=bar \
 #  -e GPG_PASSPHRASE=gloubiboulga \
 #  docker hack/release.sh
 #
-# Note: Apparmor used to mess with privileged mode, but this is no longer
-# the case. Therefore, you don't have to disable it anymore.
-#
 
-docker-version	0.6.1
-FROM	ubuntu:12.04
-MAINTAINER	Solomon Hykes <solomon@dotcloud.com>
+docker-version 0.6.1
+from    ubuntu:12.10
+maintainer      Solomon Hykes <solomon@dotcloud.com>
 
 # Build dependencies
-RUN	echo 'deb http://archive.ubuntu.com/ubuntu precise main universe' > /etc/apt/sources.list
-RUN	apt-get update
-RUN	apt-get install -y -q curl
-RUN	apt-get install -y -q git
-RUN	apt-get install -y -q mercurial
-RUN	apt-get install -y -q build-essential libsqlite3-dev
+run     apt-get update
+run     apt-get install -y -q curl
+run     apt-get install -y -q git
+run     apt-get install -y -q mercurial
+run     apt-get install -y -q build-essential
+run     apt-get install -y -q libsqlite3-dev
 
-# Install Go
-RUN	curl -s https://go.googlecode.com/files/go1.2.src.tar.gz | tar -v -C /usr/local -xz
-ENV	PATH	/usr/local/go/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
-ENV	GOPATH	/go:/go/src/github.com/dotcloud/docker/vendor
-RUN	cd /usr/local/go/src && ./make.bash && go install -ldflags '-w -linkmode external -extldflags "-static -Wl,--unresolved-symbols=ignore-in-shared-libs"' -tags netgo -a std
+# Install Go from source (for eventual cross-compiling)
+run     curl -s https://go.googlecode.com/files/go1.2rc1.src.tar.gz | tar -v -C / -xz && mv /go /goroot
+run     cd /goroot/src && ./make.bash
+env     GOROOT  /goroot
+env     PATH    $PATH:/goroot/bin
+env     GOPATH  /go:/go/src/github.com/dotcloud/docker/vendor
 
-# Ubuntu stuff
-RUN	apt-get install -y -q ruby1.9.3 rubygems libffi-dev
-RUN	gem install --no-rdoc --no-ri fpm
-RUN	apt-get install -y -q reprepro dpkg-sig
-
-RUN	apt-get install -y -q python-pip
-RUN	pip install s3cmd==1.1.0-beta3
-RUN	pip install python-magic==0.4.6
-RUN	/bin/echo -e '[default]\naccess_key=$AWS_ACCESS_KEY\nsecret_key=$AWS_SECRET_KEY\n' > /.s3cfg
-
-# Runtime dependencies
-RUN	apt-get install -y -q iptables
-RUN	apt-get install -y -q lxc
-RUN	apt-get install -y -q aufs-tools
+# Create Go cache with tag netgo (for static compilation of Go while preserving CGO support)
+run     go install -ldflags '-w -linkmode external -extldflags "-static -Wl,--unresolved-symbols=ignore-in-shared-libs"' -tags netgo -a std
 
 # Get lvm2 source for compiling statically
-RUN	git clone https://git.fedorahosted.org/git/lvm2.git /usr/local/lvm2 && cd /usr/local/lvm2 && git checkout v2_02_103
+run     git clone git://git.fedorahosted.org/git/lvm2.git /lvm2
+run     cd /lvm2 && git checkout v2_02_102
+
+# can't use git clone -b because it's not supported by git versions before 1.7.10
+run	cd /lvm2 && ./configure --enable-static_link && make && make install_device-mapper
 # see https://git.fedorahosted.org/cgit/lvm2.git/refs/tags for release tags
-# note: we can't use "git clone -b" above because it requires at least git 1.7.10 to be able to use that on a tag instead of a branch and we only have 1.7.9.5
 
-# Compile and install lvm2
-RUN	cd /usr/local/lvm2 && ./configure --enable-static_link && make device-mapper && make install_device-mapper
-# see https://git.fedorahosted.org/cgit/lvm2.git/tree/INSTALL
+# Ubuntu stuff
+run     apt-get install -y -q ruby1.9.3 rubygems libffi-dev
+run     gem install --no-rdoc --no-ri fpm
+run     apt-get install -y -q reprepro dpkg-sig
 
-# Grab Go's cover tool for dead-simple code coverage testing
-RUN	go get code.google.com/p/go.tools/cmd/cover
+# Install s3cmd 1.0.1 (earlier versions don't support env variables in the config)
+run     apt-get install -y -q python-pip
+run     pip install s3cmd
+run     pip install python-magic
+run     /bin/echo -e '[default]\naccess_key=$AWS_ACCESS_KEY\nsecret_key=$AWS_SECRET_KEY\n' > /.s3cfg
 
-VOLUME	/var/lib/docker
-WORKDIR	/go/src/github.com/dotcloud/docker
+# Runtime dependencies
+run     apt-get install -y -q iptables
+run     dpkg-divert --local --rename --add /sbin/initctl && \
+        ln -s /bin/true /sbin/initctl && \
+        apt-get install -y -q lxc
+
+volume  /var/lib/docker
+workdir /go/src/github.com/dotcloud/docker
 
 # Wrap all commands in the "docker-in-docker" script to allow nested containers
-ENTRYPOINT	["hack/dind"]
+entrypoint ["hack/dind"]
 
 # Upload docker source
-ADD	.	/go/src/github.com/dotcloud/docker
+add     .		/go/src/github.com/dotcloud/docker
+

Makefile

@@ -1,26 +0,0 @@
-.PHONY: all binary build default docs shell test
-
-DOCKER_RUN_DOCKER := docker run -rm -i -t -privileged -e TESTFLAGS -v $(CURDIR)/bundles:/go/src/github.com/dotcloud/docker/bundles docker
-
-default: binary
-
-all: build
-	$(DOCKER_RUN_DOCKER) hack/make.sh
-
-binary: build
-	$(DOCKER_RUN_DOCKER) hack/make.sh binary
-
-docs:
-	docker build -t docker-docs docs && docker run -p 8000:8000 docker-docs
-
-test: build
-	$(DOCKER_RUN_DOCKER) hack/make.sh test test-integration
-
-shell: build
-	$(DOCKER_RUN_DOCKER) bash
-
-build: bundles
-	docker build -t docker .
-
-bundles:
-	mkdir bundles

NOTICE

@@ -1,19 +1,43 @@
 Docker
-Copyright 2012-2013 Docker, Inc.
+Copyright 2012-2013 dotCloud, inc.
 
-This product includes software developed at Docker, Inc. (http://www.docker.com).
+This product includes software developed at dotCloud,
+inc. (http://www.dotcloud.com).
 
 This product contains software (https://github.com/kr/pty) developed
 by Keith Rarick, licensed under the MIT License.
 
 The following is courtesy of our legal counsel:
 
+Transfers of Docker shall be in accordance with applicable export
+controls of any country and all other applicable legal requirements.
+Docker shall not be distributed or downloaded to or in Cuba, Iran,
+North Korea, Sudan or Syria and shall not be distributed or downloaded
+to any person on the Denied Persons List administered by the U.S.
+Department of Commerce.
 
-Use and transfer of Docker may be subject to certain restrictions by the
-United States and other governments.  
-It is your responsibility to ensure that your use and/or transfer does not
-violate applicable laws. 
+What does that mean? 
+Here is a further explanation from our legal counsel:
 
-For more information, please see http://www.bis.doc.gov
+Like all software products that utilize cryptography, the export and
+use of Docker is subject to the U.S. Commerce Department's Export
+Administration Regulations (EAR) because it uses or contains
+cryptography (see
+http://www.bis.doc.gov/index.php/policy-guidance/encryption).  Certain
+free and open source software projects have a lightweight set of
+requirements, which can generally be met by providing email notice to
+the appropriate U.S. government agencies that their source code is
+available on a publicly available repository and making the
+appropriate statements in the README.
 
-See also http://www.apache.org/dev/crypto.html and/or seek legal counsel.
+The restrictions of the EAR apply to certain denied locations
+(currently Iran, Sudan, Syria, North Korea, or Cuba) and those
+individuals on the Denied Persons List, which is available here:
+http://www.bis.doc.gov/index.php/policy-guidance/lists-of-parties-of-concern/denied-persons-list.
+If you are incorporating Docker into a new open source project, the
+EAR restrictions apply to your incorporation of Docker into your
+project in the same manner as other cryptography-enabled projects,
+such as OpenSSL, almost all Linux distributions, etc.
+
+For more information, see http://www.apache.org/dev/crypto.html and/or
+seek legal counsel.

README.md

@@ -193,9 +193,10 @@ wrong or incomplete.
 *Brought to you courtesy of our legal counsel. For more context,
 please see the Notice document.*
 
-Use and transfer of Docker may be subject to certain restrictions by the
-United States and other governments.  
-It is your responsibility to ensure that your use and/or transfer does not
-violate applicable laws. 
+Transfers of Docker shall be in accordance with applicable export
+controls of any country and all other applicable legal requirements.
+Docker shall not be distributed or downloaded to or in Cuba, Iran,
+North Korea, Sudan or Syria and shall not be distributed or downloaded
+to any person on the Denied Persons List administered by the U.S.
+Department of Commerce.
 
-For more information, please see http://www.bis.doc.gov

VERSION

@@ -1 +1 @@
-0.7.2
+0.6.4-dev

Vagrantfile

@@ -4,136 +4,65 @@
 BOX_NAME = ENV['BOX_NAME'] || "ubuntu"
 BOX_URI = ENV['BOX_URI'] || "http://files.vagrantup.com/precise64.box"
 VF_BOX_URI = ENV['BOX_URI'] || "http://files.vagrantup.com/precise64_vmware_fusion.box"
-AWS_BOX_URI = ENV['BOX_URI'] || "https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box"
 AWS_REGION = ENV['AWS_REGION'] || "us-east-1"
-AWS_AMI = ENV['AWS_AMI'] || "ami-69f5a900"
-AWS_INSTANCE_TYPE = ENV['AWS_INSTANCE_TYPE'] || 't1.micro'
-
+AWS_AMI    = ENV['AWS_AMI']    || "ami-d0f89fb9"
 FORWARD_DOCKER_PORTS = ENV['FORWARD_DOCKER_PORTS']
 
-SSH_PRIVKEY_PATH = ENV["SSH_PRIVKEY_PATH"]
-
-# A script to upgrade from the 12.04 kernel to the raring backport kernel (3.8)
-# and install docker.
-$script = <<SCRIPT
-# The username to add to the docker group will be passed as the first argument
-# to the script.  If nothing is passed, default to "vagrant".
-user="$1"
-if [ -z "$user" ]; then
-    user=vagrant
-fi
-
-# Adding an apt gpg key is idempotent.
-wget -q -O - https://get.docker.io/gpg | apt-key add -
-
-# Creating the docker.list file is idempotent, but it may overrite desired
-# settings if it already exists.  This could be solved with md5sum but it
-# doesn't seem worth it.
-echo 'deb http://get.docker.io/ubuntu docker main' > \
-    /etc/apt/sources.list.d/docker.list
-
-# Update remote package metadata.  'apt-get update' is idempotent.
-apt-get update -q
-
-# Install docker.  'apt-get install' is idempotent.
-apt-get install -q -y lxc-docker
-
-usermod -a -G docker "$user"
-
-tmp=`mktemp -q` && {
-    # Only install the backport kernel, don't bother upgrade if the backport is
-    # already installed.  We want parse the output of apt so we need to save it
-    # with 'tee'.  NOTE: The installation of the kernel will trigger dkms to
-    # install vboxguest if needed.
-    apt-get install -q -y --no-upgrade linux-image-generic-lts-raring | \
-        tee "$tmp"
-
-    # Parse the number of installed packages from the output
-    NUM_INST=`awk '$2 == "upgraded," && $4 == "newly" { print $3 }' "$tmp"`
-    rm "$tmp"
-}
-
-# If the number of installed packages is greater than 0, we want to reboot (the
-# backport kernel was installed but is not running).
-if [ "$NUM_INST" -gt 0 ];
-then
-    echo "Rebooting down to activate new kernel."
-    echo "/vagrant will not be mounted.  Use 'vagrant halt' followed by"
-    echo "'vagrant up' to ensure /vagrant is mounted."
-    shutdown -r now
-fi
-SCRIPT
-
-# We need to install the virtualbox guest additions *before* we do the normal
-# docker installation.  As such this script is prepended to the common docker
-# install script above.  This allows the install of the backport kernel to
-# trigger dkms to build the virtualbox guest module install.
-$vbox_script = <<VBOX_SCRIPT + $script
-# Install the VirtualBox guest additions if they aren't already installed.
-if [ ! -d /opt/VBoxGuestAdditions-4.3.4/ ]; then
-    # Update remote package metadata.  'apt-get update' is idempotent.
-    apt-get update -q
-
-    # Kernel Headers and dkms are required to build the vbox guest kernel
-    # modules.
-    apt-get install -q -y linux-headers-generic-lts-raring dkms
-
-    echo 'Downloading VBox Guest Additions...'
-    wget -cq http://dlc.sun.com.edgesuite.net/virtualbox/4.3.4/VBoxGuestAdditions_4.3.4.iso
-    echo "f120793fa35050a8280eacf9c930cf8d9b88795161520f6515c0cc5edda2fe8a  VBoxGuestAdditions_4.3.4.iso" | sha256sum --check || exit 1
-
-    mount -o loop,ro /home/vagrant/VBoxGuestAdditions_4.3.4.iso /mnt
-    /mnt/VBoxLinuxAdditions.run --nox11
-    umount /mnt
-fi
-VBOX_SCRIPT
-
 Vagrant::Config.run do |config|
   # Setup virtual machine box. This VM configuration code is always executed.
   config.vm.box = BOX_NAME
   config.vm.box_url = BOX_URI
 
-  # Use the specified private key path if it is specified and not empty.
-  if SSH_PRIVKEY_PATH
-      config.ssh.private_key_path = SSH_PRIVKEY_PATH
-  end
-
   config.ssh.forward_agent = true
+
+  # Provision docker and new kernel if deployment was not done.
+  # It is assumed Vagrant can successfully launch the provider instance.
+  if Dir.glob("#{File.dirname(__FILE__)}/.vagrant/machines/default/*/id").empty?
+    # Add lxc-docker package
+    pkg_cmd = "wget -q -O - https://get.docker.io/gpg | apt-key add -;" \
+      "echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list;" \
+      "apt-get update -qq; apt-get install -q -y --force-yes lxc-docker; "
+    # Add Ubuntu raring backported kernel
+    pkg_cmd << "apt-get update -qq; apt-get install -q -y linux-image-generic-lts-raring; "
+    # Add guest additions if local vbox VM. As virtualbox is the default provider,
+    # it is assumed it won't be explicitly stated.
+    if ENV["VAGRANT_DEFAULT_PROVIDER"].nil? && ARGV.none? { |arg| arg.downcase.start_with?("--provider") }
+      pkg_cmd << "apt-get install -q -y linux-headers-generic-lts-raring dkms; " \
+        "echo 'Downloading VBox Guest Additions...'; " \
+        "wget -q http://dlc.sun.com.edgesuite.net/virtualbox/4.2.12/VBoxGuestAdditions_4.2.12.iso; "
+      # Prepare the VM to add guest additions after reboot
+      pkg_cmd << "echo -e 'mount -o loop,ro /home/vagrant/VBoxGuestAdditions_4.2.12.iso /mnt\n" \
+        "echo yes | /mnt/VBoxLinuxAdditions.run\numount /mnt\n" \
+          "rm /root/guest_additions.sh; ' > /root/guest_additions.sh; " \
+        "chmod 700 /root/guest_additions.sh; " \
+        "sed -i -E 's#^exit 0#[ -x /root/guest_additions.sh ] \\&\\& /root/guest_additions.sh#' /etc/rc.local; " \
+        "echo 'Installation of VBox Guest Additions is proceeding in the background.'; " \
+        "echo '\"vagrant reload\" can be used in about 2 minutes to activate the new guest additions.'; "
+    end
+    # Add vagrant user to the docker group
+    pkg_cmd << "usermod -a -G docker vagrant; "
+    # Activate new kernel
+    pkg_cmd << "shutdown -r +1; "
+    config.vm.provision :shell, :inline => pkg_cmd
+  end
 end
 
+
 # Providers were added on Vagrant >= 1.1.0
-#
-# NOTE: The vagrant "vm.provision" appends its arguments to a list and executes
-# them in order.  If you invoke "vm.provision :shell, :inline => $script"
-# twice then vagrant will run the script two times.  Unfortunately when you use
-# providers and the override argument to set up provisioners (like the vbox
-# guest extensions) they 1) don't replace the other provisioners (they append
-# to the end of the list) and 2) you can't control the order the provisioners
-# are executed (you can only append to the list).  If you want the virtualbox
-# only script to run before the other script, you have to jump through a lot of
-# hoops.
-#
-# Here is my only repeatable solution: make one script that is common ($script)
-# and another script that is the virtual box guest *prepended* to the common
-# script.  Only ever use "vm.provision" *one time* per provider.  That means
-# every single provider has an override, and every single one configures
-# "vm.provision".  Much saddness, but such is life.
 Vagrant::VERSION >= "1.1.0" and Vagrant.configure("2") do |config|
   config.vm.provider :aws do |aws, override|
-    username = "ubuntu"
-    override.vm.box_url = AWS_BOX_URI
-    override.vm.provision :shell, :inline => $script, :args => username
-    aws.access_key_id = ENV["AWS_ACCESS_KEY"]
-    aws.secret_access_key = ENV["AWS_SECRET_KEY"]
+    aws.access_key_id = ENV["AWS_ACCESS_KEY_ID"]
+    aws.secret_access_key = ENV["AWS_SECRET_ACCESS_KEY"]
     aws.keypair_name = ENV["AWS_KEYPAIR_NAME"]
-    override.ssh.username = username
+    override.ssh.private_key_path = ENV["AWS_SSH_PRIVKEY"]
+    override.ssh.username = "ubuntu"
     aws.region = AWS_REGION
     aws.ami    = AWS_AMI
-    aws.instance_type = AWS_INSTANCE_TYPE
+    aws.instance_type = "t1.micro"
   end
 
-  config.vm.provider :rackspace do |rs, override|
-    override.vm.provision :shell, :inline => $script
+  config.vm.provider :rackspace do |rs|
+    config.ssh.private_key_path = ENV["RS_PRIVATE_KEY"]
     rs.username = ENV["RS_USERNAME"]
     rs.api_key  = ENV["RS_API_KEY"]
     rs.public_key_path = ENV["RS_PUBLIC_KEY"]
@@ -142,25 +71,20 @@ Vagrant::VERSION >= "1.1.0" and Vagrant.configure("2") do |config|
   end
 
   config.vm.provider :vmware_fusion do |f, override|
+    override.vm.box = BOX_NAME
     override.vm.box_url = VF_BOX_URI
     override.vm.synced_folder ".", "/vagrant", disabled: true
-    override.vm.provision :shell, :inline => $script
     f.vmx["displayName"] = "docker"
   end
 
-  config.vm.provider :virtualbox do |vb, override|
-    override.vm.provision :shell, :inline => $vbox_script
+  config.vm.provider :virtualbox do |vb|
+    config.vm.box = BOX_NAME
+    config.vm.box_url = BOX_URI
     vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
     vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
   end
 end
 
-# If this is a version 1 config, virtualbox is the only option.  A version 2
-# config would have already been set in the above provider section.
-Vagrant::VERSION < "1.1.0" and Vagrant::Config.run do |config|
-  config.vm.provision :shell, :inline => $vbox_script
-end
-
 if !FORWARD_DOCKER_PORTS.nil?
   Vagrant::VERSION < "1.1.0" and Vagrant::Config.run do |config|
     (49000..49900).each do |port|

api.go

@@ -1,16 +1,12 @@
 package docker
 
 import (
-	"bufio"
-	"bytes"
 	"code.google.com/p/go.net/websocket"
 	"encoding/base64"
 	"encoding/json"
-	"expvar"
 	"fmt"
-	"github.com/dotcloud/docker/archive"
 	"github.com/dotcloud/docker/auth"
-	"github.com/dotcloud/docker/systemd"
+	"github.com/dotcloud/docker/gograph"
 	"github.com/dotcloud/docker/utils"
 	"github.com/gorilla/mux"
 	"io"
@@ -19,7 +15,6 @@ import (
 	"mime"
 	"net"
 	"net/http"
-	"net/http/pprof"
 	"os"
 	"os/exec"
 	"regexp"
@@ -28,7 +23,7 @@ import (
 )
 
 const (
-	APIVERSION        = 1.8
+	APIVERSION        = 1.6
 	DEFAULTHTTPHOST   = "127.0.0.1"
 	DEFAULTHTTPPORT   = 4243
 	DEFAULTUNIXSOCKET = "/var/run/docker.sock"
@@ -48,9 +43,6 @@ func hijackServer(w http.ResponseWriter) (io.ReadCloser, io.Writer, error) {
 
 //If we don't do this, POST method without Content-type (even with empty body) will fail
 func parseForm(r *http.Request) error {
-	if r == nil {
-		return nil
-	}
 	if err := r.ParseForm(); err != nil && !strings.HasPrefix(err.Error(), "mime:") {
 		return err
 	}
@@ -66,10 +58,7 @@ func parseMultipartForm(r *http.Request) error {
 
 func httpError(w http.ResponseWriter, err error) {
 	statusCode := http.StatusInternalServerError
-	// FIXME: this is brittle and should not be necessary.
-	// If we need to differentiate between different possible error types, we should
-	// create appropriate error types with clearly defined meaning.
-	if strings.Contains(err.Error(), "No such") {
+	if strings.HasPrefix(err.Error(), "No such") {
 		statusCode = http.StatusNotFound
 	} else if strings.HasPrefix(err.Error(), "Bad parameter") {
 		statusCode = http.StatusBadRequest
@@ -140,22 +129,15 @@ func postAuth(srv *Server, version float64, w http.ResponseWriter, r *http.Reque
 }
 
 func getVersion(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
-	srv.Eng.ServeHTTP(w, r)
-	return nil
+	return writeJSON(w, http.StatusOK, srv.DockerVersion())
 }
 
 func postContainersKill(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
 	if vars == nil {
 		return fmt.Errorf("Missing parameter")
 	}
-	if err := parseForm(r); err != nil {
-		return err
-	}
-	job := srv.Eng.Job("kill", vars["name"])
-	if sig := r.Form.Get("signal"); sig != "" {
-		job.Args = append(job.Args, sig)
-	}
-	if err := job.Run(); err != nil {
+	name := vars["name"]
+	if err := srv.ContainerKill(name); err != nil {
 		return err
 	}
 	w.WriteHeader(http.StatusNoContent)
@@ -166,11 +148,10 @@ func getContainersExport(srv *Server, version float64, w http.ResponseWriter, r
 	if vars == nil {
 		return fmt.Errorf("Missing parameter")
 	}
-	job := srv.Eng.Job("export", vars["name"])
-	if err := job.Stdout.Add(w); err != nil {
-		return err
-	}
-	if err := job.Run(); err != nil {
+	name := vars["name"]
+
+	if err := srv.ContainerExport(name, w); err != nil {
+		utils.Errorf("%s", err)
 		return err
 	}
 	return nil
@@ -192,23 +173,10 @@ func getImagesJSON(srv *Server, version float64, w http.ResponseWriter, r *http.
 		return err
 	}
 
-	if version < 1.7 {
-		outs2 := []APIImagesOld{}
-		for _, ctnr := range outs {
-			outs2 = append(outs2, ctnr.ToLegacy()...)
-		}
-
-		return writeJSON(w, http.StatusOK, outs2)
-	}
 	return writeJSON(w, http.StatusOK, outs)
 }
 
 func getImagesViz(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
-	if version > 1.6 {
-		w.WriteHeader(http.StatusNotFound)
-		return fmt.Errorf("This is now implemented in the client.")
-	}
-
 	if err := srv.ImagesViz(w); err != nil {
 		return err
 	}
@@ -216,8 +184,7 @@ func getImagesViz(srv *Server, version float64, w http.ResponseWriter, r *http.R
 }
 
 func getInfo(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
-	srv.Eng.ServeHTTP(w, r)
-	return nil
+	return writeJSON(w, http.StatusOK, srv.DockerInfo())
 }
 
 func getEvents(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -251,10 +218,9 @@ func getEvents(srv *Server, version float64, w http.ResponseWriter, r *http.Requ
 	}
 	w.Header().Set("Content-Type", "application/json")
 	wf := utils.NewWriteFlusher(w)
-	wf.Flush()
 	if since != 0 {
 		// If since, send previous events that happened after the timestamp
-		for _, event := range srv.GetEvents() {
+		for _, event := range srv.events {
 			if event.Time >= since {
 				err := sendEvent(wf, &event)
 				if err != nil && err.Error() == "JSON error" {
@@ -314,10 +280,13 @@ func getContainersTop(srv *Server, version float64, w http.ResponseWriter, r *ht
 	if err := parseForm(r); err != nil {
 		return err
 	}
-	procsStr, err := srv.ContainerTop(vars["name"], r.Form.Get("ps_args"))
+	name := vars["name"]
+	ps_args := r.Form.Get("ps_args")
+	procsStr, err := srv.ContainerTop(name, ps_args)
 	if err != nil {
 		return err
 	}
+
 	return writeJSON(w, http.StatusOK, procsStr)
 }
 
@@ -345,25 +314,31 @@ func getContainersJSON(srv *Server, version float64, w http.ResponseWriter, r *h
 	if version < 1.5 {
 		outs2 := []APIContainersOld{}
 		for _, ctnr := range outs {
-			outs2 = append(outs2, *ctnr.ToLegacy())
+			outs2 = append(outs2, ctnr.ToLegacy())
 		}
 
 		return writeJSON(w, http.StatusOK, outs2)
+	} else {
+		return writeJSON(w, http.StatusOK, outs)
 	}
-	return writeJSON(w, http.StatusOK, outs)
 }
 
 func postImagesTag(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
 	if err := parseForm(r); err != nil {
 		return err
 	}
+	repo := r.Form.Get("repo")
+	tag := r.Form.Get("tag")
 	if vars == nil {
 		return fmt.Errorf("Missing parameter")
 	}
+	name := vars["name"]
+	force, err := getBoolParam(r.Form.Get("force"))
+	if err != nil {
+		return err
+	}
 
-	job := srv.Eng.Job("tag", vars["name"], r.Form.Get("repo"), r.Form.Get("tag"))
-	job.Setenv("force", r.Form.Get("force"))
-	if err := job.Run(); err != nil {
+	if err := srv.ContainerTag(name, repo, tag, force); err != nil {
 		return err
 	}
 	w.WriteHeader(http.StatusCreated)
@@ -378,17 +353,13 @@ func postCommit(srv *Server, version float64, w http.ResponseWriter, r *http.Req
 	if err := json.NewDecoder(r.Body).Decode(config); err != nil && err != io.EOF {
 		utils.Errorf("%s", err)
 	}
-
-	job := srv.Eng.Job("commit", r.Form.Get("container"))
-	job.Setenv("repo", r.Form.Get("repo"))
-	job.Setenv("tag", r.Form.Get("tag"))
-	job.Setenv("author", r.Form.Get("author"))
-	job.Setenv("comment", r.Form.Get("comment"))
-	job.SetenvJson("config", config)
-
-	var id string
-	job.Stdout.AddString(&id)
-	if err := job.Run(); err != nil {
+	repo := r.Form.Get("repo")
+	tag := r.Form.Get("tag")
+	container := r.Form.Get("container")
+	author := r.Form.Get("author")
+	comment := r.Form.Get("comment")
+	id, err := srv.ContainerCommit(container, repo, tag, author, comment, config)
+	if err != nil {
 		return err
 	}
 
@@ -475,16 +446,15 @@ func postImagesInsert(srv *Server, version float64, w http.ResponseWriter, r *ht
 		w.Header().Set("Content-Type", "application/json")
 	}
 	sf := utils.NewStreamFormatter(version > 1.0)
-	err := srv.ImageInsert(name, url, path, w, sf)
+	imgID, err := srv.ImageInsert(name, url, path, w, sf)
 	if err != nil {
 		if sf.Used() {
 			w.Write(sf.FormatError(err))
 			return nil
 		}
-		return err
 	}
 
-	return nil
+	return writeJSON(w, http.StatusOK, &APIID{ID: imgID})
 }
 
 func postImagesPush(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -533,58 +503,43 @@ func postImagesPush(srv *Server, version float64, w http.ResponseWriter, r *http
 	return nil
 }
 
-func getImagesGet(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
-	name := vars["name"]
-	if version > 1.0 {
-		w.Header().Set("Content-Type", "application/x-tar")
-	}
-	return srv.ImageExport(name, w)
-}
-
-func postImagesLoad(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
-	return srv.ImageLoad(r.Body)
-}
-
 func postContainersCreate(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
-	if err := parseForm(r); err != nil {
-		return nil
-	}
+	config := &Config{}
 	out := &APIRun{}
-	job := srv.Eng.Job("create", r.Form.Get("name"))
-	if err := job.DecodeEnv(r.Body); err != nil {
+
+	if err := json.NewDecoder(r.Body).Decode(config); err != nil {
 		return err
 	}
+
 	resolvConf, err := utils.GetResolvConf()
 	if err != nil {
 		return err
 	}
-	if !job.GetenvBool("NetworkDisabled") && len(job.Getenv("Dns")) == 0 && len(srv.runtime.config.Dns) == 0 && utils.CheckLocalDns(resolvConf) {
+
+	if !config.NetworkDisabled && len(config.Dns) == 0 && len(srv.runtime.config.Dns) == 0 && utils.CheckLocalDns(resolvConf) {
 		out.Warnings = append(out.Warnings, fmt.Sprintf("Docker detected local DNS server on resolv.conf. Using default external servers: %v", defaultDns))
-		job.SetenvList("Dns", defaultDns)
+		config.Dns = defaultDns
 	}
-	// Read container ID from the first line of stdout
-	job.Stdout.AddString(&out.ID)
-	// Read warnings from stderr
-	warnings := &bytes.Buffer{}
-	job.Stderr.Add(warnings)
-	if err := job.Run(); err != nil {
+
+	id, warnings, err := srv.ContainerCreate(config)
+	if err != nil {
 		return err
 	}
-	// Parse warnings from stderr
-	scanner := bufio.NewScanner(warnings)
-	for scanner.Scan() {
-		out.Warnings = append(out.Warnings, scanner.Text())
+	out.ID = id
+	for _, warning := range warnings {
+		out.Warnings = append(out.Warnings, warning)
 	}
-	if job.GetenvInt("Memory") > 0 && !srv.runtime.capabilities.MemoryLimit {
+
+	if config.Memory > 0 && !srv.runtime.capabilities.MemoryLimit {
 		log.Println("WARNING: Your kernel does not support memory limit capabilities. Limitation discarded.")
 		out.Warnings = append(out.Warnings, "Your kernel does not support memory limit capabilities. Limitation discarded.")
 	}
-	if job.GetenvInt("Memory") > 0 && !srv.runtime.capabilities.SwapLimit {
+	if config.Memory > 0 && !srv.runtime.capabilities.SwapLimit {
 		log.Println("WARNING: Your kernel does not support swap limit capabilities. Limitation discarded.")
 		out.Warnings = append(out.Warnings, "Your kernel does not support memory swap capabilities. Limitation discarded.")
 	}
 
-	if !job.GetenvBool("NetworkDisabled") && srv.runtime.capabilities.IPv4ForwardingDisabled {
+	if !config.NetworkDisabled && srv.runtime.capabilities.IPv4ForwardingDisabled {
 		log.Println("Warning: IPv4 forwarding is disabled.")
 		out.Warnings = append(out.Warnings, "IPv4 forwarding is disabled.")
 	}
@@ -651,28 +606,33 @@ func deleteImages(srv *Server, version float64, w http.ResponseWriter, r *http.R
 	if imgs != nil {
 		if len(imgs) != 0 {
 			return writeJSON(w, http.StatusOK, imgs)
+		} else {
+			return fmt.Errorf("Conflict, %s wasn't deleted", name)
 		}
-		return fmt.Errorf("Conflict, %s wasn't deleted", name)
+	} else {
+		w.WriteHeader(http.StatusNoContent)
 	}
-	w.WriteHeader(http.StatusNoContent)
 	return nil
 }
 
 func postContainersStart(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
-	if vars == nil {
-		return fmt.Errorf("Missing parameter")
-	}
-	name := vars["name"]
-	job := srv.Eng.Job("start", name)
+	var hostConfig *HostConfig
 	// allow a nil body for backwards compatibility
 	if r.Body != nil {
 		if matchesContentType(r.Header.Get("Content-Type"), "application/json") {
-			if err := job.DecodeEnv(r.Body); err != nil {
+			hostConfig = &HostConfig{}
+			if err := json.NewDecoder(r.Body).Decode(hostConfig); err != nil {
 				return err
 			}
 		}
 	}
-	if err := job.Run(); err != nil {
+
+	if vars == nil {
+		return fmt.Errorf("Missing parameter")
+	}
+	name := vars["name"]
+	if err := srv.ContainerStart(name, hostConfig); err != nil {
+		utils.Debugf("error ContainerStart: %s", err)
 		return err
 	}
 	w.WriteHeader(http.StatusNoContent)
@@ -683,12 +643,17 @@ func postContainersStop(srv *Server, version float64, w http.ResponseWriter, r *
 	if err := parseForm(r); err != nil {
 		return err
 	}
+	t, err := strconv.Atoi(r.Form.Get("t"))
+	if err != nil || t < 0 {
+		t = 10
+	}
+
 	if vars == nil {
 		return fmt.Errorf("Missing parameter")
 	}
-	job := srv.Eng.Job("stop", vars["name"])
-	job.Setenv("t", r.Form.Get("t"))
-	if err := job.Run(); err != nil {
+	name := vars["name"]
+
+	if err := srv.ContainerStop(name, t); err != nil {
 		return err
 	}
 	w.WriteHeader(http.StatusNoContent)
@@ -699,28 +664,33 @@ func postContainersWait(srv *Server, version float64, w http.ResponseWriter, r *
 	if vars == nil {
 		return fmt.Errorf("Missing parameter")
 	}
-	job := srv.Eng.Job("wait", vars["name"])
-	var statusStr string
-	job.Stdout.AddString(&statusStr)
-	if err := job.Run(); err != nil {
-		return err
-	}
-	// Parse a 16-bit encoded integer to map typical unix exit status.
-	status, err := strconv.ParseInt(statusStr, 10, 16)
+	name := vars["name"]
+
+	status, err := srv.ContainerWait(name)
 	if err != nil {
 		return err
 	}
-	return writeJSON(w, http.StatusOK, &APIWait{StatusCode: int(status)})
+
+	return writeJSON(w, http.StatusOK, &APIWait{StatusCode: status})
 }
 
 func postContainersResize(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
 	if err := parseForm(r); err != nil {
 		return err
 	}
+	height, err := strconv.Atoi(r.Form.Get("h"))
+	if err != nil {
+		return err
+	}
+	width, err := strconv.Atoi(r.Form.Get("w"))
+	if err != nil {
+		return err
+	}
 	if vars == nil {
 		return fmt.Errorf("Missing parameter")
 	}
-	if err := srv.Eng.Job("resize", vars["name"], r.Form.Get("h"), r.Form.Get("w")).Run(); err != nil {
+	name := vars["name"]
+	if err := srv.ContainerResize(name, height, width); err != nil {
 		return err
 	}
 	return nil
@@ -860,10 +830,7 @@ func getContainersByName(srv *Server, version float64, w http.ResponseWriter, r
 		return fmt.Errorf("Conflict between containers and images")
 	}
 
-	container.readHostConfig()
-	c := APIContainer{container, container.hostConfig}
-
-	return writeJSON(w, http.StatusOK, c)
+	return writeJSON(w, http.StatusOK, container)
 }
 
 func getImagesByName(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -889,25 +856,12 @@ func postBuild(srv *Server, version float64, w http.ResponseWriter, r *http.Requ
 	if version < 1.3 {
 		return fmt.Errorf("Multipart upload for build is no longer supported. Please upgrade your docker client.")
 	}
-	var (
-		remoteURL         = r.FormValue("remote")
-		repoName          = r.FormValue("t")
-		rawSuppressOutput = r.FormValue("q")
-		rawNoCache        = r.FormValue("nocache")
-		rawRm             = r.FormValue("rm")
-		authEncoded       = r.Header.Get("X-Registry-Auth")
-		authConfig        = &auth.AuthConfig{}
-		tag               string
-	)
-	repoName, tag = utils.ParseRepositoryTag(repoName)
-	if authEncoded != "" {
-		authJson := base64.NewDecoder(base64.URLEncoding, strings.NewReader(authEncoded))
-		if err := json.NewDecoder(authJson).Decode(authConfig); err != nil {
-			// for a pull it is not an error if no auth was given
-			// to increase compatibility with the existing api it is defaulting to be empty
-			authConfig = &auth.AuthConfig{}
-		}
-	}
+	remoteURL := r.FormValue("remote")
+	repoName := r.FormValue("t")
+	rawSuppressOutput := r.FormValue("q")
+	rawNoCache := r.FormValue("nocache")
+	rawRm := r.FormValue("rm")
+	repoName, tag := utils.ParseRepositoryTag(repoName)
 
 	var context io.Reader
 
@@ -927,13 +881,13 @@ func postBuild(srv *Server, version float64, w http.ResponseWriter, r *http.Requ
 			return fmt.Errorf("Error trying to use git: %s (%s)", err, output)
 		}
 
-		c, err := archive.Tar(root, archive.Bzip2)
+		c, err := Tar(root, Bzip2)
 		if err != nil {
 			return err
 		}
 		context = c
 	} else if utils.IsURL(remoteURL) {
-		f, err := utils.Download(remoteURL)
+		f, err := utils.Download(remoteURL, ioutil.Discard)
 		if err != nil {
 			return err
 		}
@@ -942,7 +896,7 @@ func postBuild(srv *Server, version float64, w http.ResponseWriter, r *http.Requ
 		if err != nil {
 			return err
 		}
-		c, err := MkBuildContext(string(dockerFile), nil)
+		c, err := mkBuildContext(string(dockerFile), nil)
 		if err != nil {
 			return err
 		}
@@ -962,26 +916,9 @@ func postBuild(srv *Server, version float64, w http.ResponseWriter, r *http.Requ
 		return err
 	}
 
-	if version >= 1.8 {
-		w.Header().Set("Content-Type", "application/json")
-	}
-	sf := utils.NewStreamFormatter(version >= 1.8)
-	b := NewBuildFile(srv,
-		&StdoutFormater{
-			Writer:          utils.NewWriteFlusher(w),
-			StreamFormatter: sf,
-		},
-		&StderrFormater{
-			Writer:          utils.NewWriteFlusher(w),
-			StreamFormatter: sf,
-		},
-		!suppressOutput, !noCache, rm, utils.NewWriteFlusher(w), sf, authConfig)
+	b := NewBuildFile(srv, utils.NewWriteFlusher(w), !suppressOutput, !noCache, rm)
 	id, err := b.Build(context)
 	if err != nil {
-		if sf.Used() {
-			w.Write(sf.FormatError(err))
-			return nil
-		}
 		return fmt.Errorf("Error build: %s", err)
 	}
 	if repoName != "" {
@@ -1007,7 +944,7 @@ func postContainersCopy(srv *Server, version float64, w http.ResponseWriter, r *
 	}
 
 	if copyData.Resource == "" {
-		return fmt.Errorf("Path cannot be empty")
+		return fmt.Errorf("Resource cannot be empty")
 	}
 	if copyData.Resource[0] == '/' {
 		copyData.Resource = copyData.Resource[1:]
@@ -1065,37 +1002,76 @@ func makeHttpHandler(srv *Server, logging bool, localMethod string, localRoute s
 	}
 }
 
-// Replicated from expvar.go as not public.
-func expvarHandler(w http.ResponseWriter, r *http.Request) {
-	w.Header().Set("Content-Type", "application/json; charset=utf-8")
-	fmt.Fprintf(w, "{\n")
-	first := true
-	expvar.Do(func(kv expvar.KeyValue) {
-		if !first {
-			fmt.Fprintf(w, ",\n")
+func getContainersLinks(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
+	if err := parseForm(r); err != nil {
+		return err
+	}
+
+	runtime := srv.runtime
+	all, err := getBoolParam(r.Form.Get("all"))
+	if err != nil {
+		return err
+	}
+
+	out := []APILink{}
+	err = runtime.containerGraph.Walk("/", func(p string, e *gograph.Entity) error {
+		if container := runtime.Get(e.ID()); container != nil {
+			if !all && strings.Contains(p, container.ID) {
+				return nil
+			}
+			out = append(out, APILink{
+				Path:        p,
+				ContainerID: container.ID,
+				Image:       runtime.repositories.ImageName(container.Image),
+			})
 		}
-		first = false
-		fmt.Fprintf(w, "%q: %s", kv.Key, kv.Value)
-	})
-	fmt.Fprintf(w, "\n}\n")
+		return nil
+	}, -1)
+
+	if err != nil {
+		return err
+	}
+	return writeJSON(w, http.StatusOK, out)
 }
 
-func AttachProfiler(router *mux.Router) {
-	router.HandleFunc("/debug/vars", expvarHandler)
-	router.HandleFunc("/debug/pprof/", pprof.Index)
-	router.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
-	router.HandleFunc("/debug/pprof/profile", pprof.Profile)
-	router.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
-	router.HandleFunc("/debug/pprof/heap", pprof.Handler("heap").ServeHTTP)
-	router.HandleFunc("/debug/pprof/goroutine", pprof.Handler("goroutine").ServeHTTP)
-	router.HandleFunc("/debug/pprof/threadcreate", pprof.Handler("threadcreate").ServeHTTP)
+func postContainerLink(srv *Server, version float64, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
+	if vars == nil {
+		return fmt.Errorf("Missing parameter")
+	}
+	values := make(map[string]string)
+	if matchesContentType(r.Header.Get("Content-Type"), "application/json") && r.Body != nil {
+		defer r.Body.Close()
+
+		dec := json.NewDecoder(r.Body)
+		if err := dec.Decode(&values); err != nil {
+			return err
+		}
+	} else {
+		return fmt.Errorf("Invalid json body")
+	}
+	currentName := values["currentName"]
+	newName := values["newName"]
+
+	if currentName == "" {
+		return fmt.Errorf("currentName cannot be empty")
+	}
+	if newName == "" {
+		return fmt.Errorf("newName cannot be empty")
+	}
+
+	if err := srv.runtime.RenameLink(currentName, newName); err != nil {
+		if strings.HasSuffix(err.Error(), "name are not unique") {
+			return fmt.Errorf("Conflict, %s already exists", newName)
+		}
+		return err
+	}
+
+	return nil
 }
 
 func createRouter(srv *Server, logging bool) (*mux.Router, error) {
 	r := mux.NewRouter()
-	if os.Getenv("DEBUG") != "" {
-		AttachProfiler(r)
-	}
+
 	m := map[string]map[string]HttpApiFunc{
 		"GET": {
 			"/events":                         getEvents,
@@ -1104,7 +1080,6 @@ func createRouter(srv *Server, logging bool) (*mux.Router, error) {
 			"/images/json":                    getImagesJSON,
 			"/images/viz":                     getImagesViz,
 			"/images/search":                  getImagesSearch,
-			"/images/{name:.*}/get":           getImagesGet,
 			"/images/{name:.*}/history":       getImagesHistory,
 			"/images/{name:.*}/json":          getImagesByName,
 			"/containers/ps":                  getContainersJSON,
@@ -1114,6 +1089,7 @@ func createRouter(srv *Server, logging bool) (*mux.Router, error) {
 			"/containers/{name:.*}/json":      getContainersByName,
 			"/containers/{name:.*}/top":       getContainersTop,
 			"/containers/{name:.*}/attach/ws": wsContainersAttach,
+			"/containers/links":               getContainersLinks,
 		},
 		"POST": {
 			"/auth":                         postAuth,
@@ -1121,7 +1097,6 @@ func createRouter(srv *Server, logging bool) (*mux.Router, error) {
 			"/build":                        postBuild,
 			"/images/create":                postImagesCreate,
 			"/images/{name:.*}/insert":      postImagesInsert,
-			"/images/load":                  postImagesLoad,
 			"/images/{name:.*}/push":        postImagesPush,
 			"/images/{name:.*}/tag":         postImagesTag,
 			"/containers/create":            postContainersCreate,
@@ -1133,6 +1108,7 @@ func createRouter(srv *Server, logging bool) (*mux.Router, error) {
 			"/containers/{name:.*}/resize":  postContainersResize,
 			"/containers/{name:.*}/attach":  postContainersAttach,
 			"/containers/{name:.*}/copy":    postContainersCopy,
+			"/containers/link":              postContainerLink,
 		},
 		"DELETE": {
 			"/containers/{name:.*}": deleteContainers,
@@ -1167,21 +1143,9 @@ func createRouter(srv *Server, logging bool) (*mux.Router, error) {
 	return r, nil
 }
 
-// ServeRequest processes a single http request to the docker remote api.
-// FIXME: refactor this to be part of Server and not require re-creating a new
-// router each time. This requires first moving ListenAndServe into Server.
-func ServeRequest(srv *Server, apiversion float64, w http.ResponseWriter, req *http.Request) error {
-	router, err := createRouter(srv, false)
-	if err != nil {
-		return err
-	}
-	// Insert APIVERSION into the request as a convenience
-	req.URL.Path = fmt.Sprintf("/v%g%s", apiversion, req.URL.Path)
-	router.ServeHTTP(w, req)
-	return nil
-}
-
 func ListenAndServe(proto, addr string, srv *Server, logging bool) error {
+	log.Printf("Listening for HTTP on %s (%s)\n", addr, proto)
+
 	r, err := createRouter(srv, logging)
 	if err != nil {
 		return err
@@ -1212,9 +1176,5 @@ func ListenAndServe(proto, addr string, srv *Server, logging bool) error {
 		}
 	}
 	httpSrv := http.Server{Addr: addr, Handler: r}
-
-	log.Printf("Listening for HTTP on %s (%s)\n", addr, proto)
-	// Tell the init daemon we are accepting requests
-	go systemd.SdNotify("READY=1")
 	return httpSrv.Serve(l)
 }

api_params.go

@@ -1,131 +1,134 @@
 package docker
 
-import "strings"
-
-type (
-	APIHistory struct {
-		ID        string   `json:"Id"`
-		Tags      []string `json:",omitempty"`
-		Created   int64
-		CreatedBy string `json:",omitempty"`
-		Size      int64
-	}
-
-	APIImages struct {
-		ID          string   `json:"Id"`
-		RepoTags    []string `json:",omitempty"`
-		Created     int64
-		Size        int64
-		VirtualSize int64
-		ParentId    string `json:",omitempty"`
-	}
-
-	APIImagesOld struct {
-		Repository  string `json:",omitempty"`
-		Tag         string `json:",omitempty"`
-		ID          string `json:"Id"`
-		Created     int64
-		Size        int64
-		VirtualSize int64
-	}
-
-	APITop struct {
-		Titles    []string
-		Processes [][]string
-	}
-
-	APIRmi struct {
-		Deleted  string `json:",omitempty"`
-		Untagged string `json:",omitempty"`
-	}
-
-	APIContainers struct {
-		ID         string `json:"Id"`
-		Image      string
-		Command    string
-		Created    int64
-		Status     string
-		Ports      []APIPort
-		SizeRw     int64
-		SizeRootFs int64
-		Names      []string
-	}
-
-	APIContainersOld struct {
-		ID         string `json:"Id"`
-		Image      string
-		Command    string
-		Created    int64
-		Status     string
-		Ports      string
-		SizeRw     int64
-		SizeRootFs int64
-	}
-
-	APIID struct {
-		ID string `json:"Id"`
-	}
-
-	APIRun struct {
-		ID       string   `json:"Id"`
-		Warnings []string `json:",omitempty"`
-	}
-
-	APIPort struct {
-		PrivatePort int64
-		PublicPort  int64
-		Type        string
-		IP          string
-	}
-
-	APIWait struct {
-		StatusCode int
-	}
-
-	APIAuth struct {
-		Status string
-	}
-
-	APIImageConfig struct {
-		ID string `json:"Id"`
-		*Config
-	}
-
-	APICopy struct {
-		Resource string
-		HostPath string
-	}
-	APIContainer struct {
-		*Container
-		HostConfig *HostConfig
-	}
-)
-
-func (api APIImages) ToLegacy() []APIImagesOld {
-	outs := []APIImagesOld{}
-	for _, repotag := range api.RepoTags {
-		components := strings.SplitN(repotag, ":", 2)
-		outs = append(outs, APIImagesOld{
-			ID:          api.ID,
-			Repository:  components[0],
-			Tag:         components[1],
-			Created:     api.Created,
-			Size:        api.Size,
-			VirtualSize: api.VirtualSize,
-		})
-	}
-	return outs
+type APIHistory struct {
+	ID        string   `json:"Id"`
+	Tags      []string `json:",omitempty"`
+	Created   int64
+	CreatedBy string `json:",omitempty"`
 }
 
-func (api APIContainers) ToLegacy() *APIContainersOld {
-	return &APIContainersOld{
-		ID:         api.ID,
-		Image:      api.Image,
-		Command:    api.Command,
-		Created:    api.Created,
-		Status:     api.Status,
-		Ports:      displayablePorts(api.Ports),
-		SizeRw:     api.SizeRw,
-		SizeRootFs: api.SizeRootFs,
+type APIImages struct {
+	Repository  string `json:",omitempty"`
+	Tag         string `json:",omitempty"`
+	ID          string `json:"Id"`
+	Created     int64
+	Size        int64
+	VirtualSize int64
+}
+
+type APIInfo struct {
+	Debug                  bool
+	Containers             int
+	Images                 int
+	NFd                    int    `json:",omitempty"`
+	NGoroutines            int    `json:",omitempty"`
+	MemoryLimit            bool   `json:",omitempty"`
+	SwapLimit              bool   `json:",omitempty"`
+	IPv4Forwarding         bool   `json:",omitempty"`
+	LXCVersion             string `json:",omitempty"`
+	NEventsListener        int    `json:",omitempty"`
+	KernelVersion          string `json:",omitempty"`
+	IndexServerAddress     string `json:",omitempty"`
+	DevmapperPool          string `json:",omitempty"`
+	DevmapperDataUsed      uint64 `json:",omitempty"`
+	DevmapperDataTotal     uint64 `json:",omitempty"`
+	DevmapperMetadataUsed  uint64 `json:",omitempty"`
+	DevmapperMetadataTotal uint64 `json:",omitempty"`
+}
+
+type APITop struct {
+	Titles    []string
+	Processes [][]string
+}
+
+type APIRmi struct {
+	Deleted  string `json:",omitempty"`
+	Untagged string `json:",omitempty"`
+}
+
+type APIContainers struct {
+	ID         string `json:"Id"`
+	Image      string
+	Command    string
+	Created    int64
+	Status     string
+	Ports      []APIPort
+	SizeRw     int64
+	SizeRootFs int64
+	Names      []string
+}
+
+func (self *APIContainers) ToLegacy() APIContainersOld {
+	return APIContainersOld{
+		ID:         self.ID,
+		Image:      self.Image,
+		Command:    self.Command,
+		Created:    self.Created,
+		Status:     self.Status,
+		Ports:      displayablePorts(self.Ports),
+		SizeRw:     self.SizeRw,
+		SizeRootFs: self.SizeRootFs,
 	}
 }
+
+type APIContainersOld struct {
+	ID         string `json:"Id"`
+	Image      string
+	Command    string
+	Created    int64
+	Status     string
+	Ports      string
+	SizeRw     int64
+	SizeRootFs int64
+}
+
+type APISearch struct {
+	Name        string
+	Description string
+}
+
+type APIID struct {
+	ID string `json:"Id"`
+}
+
+type APIRun struct {
+	ID       string   `json:"Id"`
+	Warnings []string `json:",omitempty"`
+}
+
+type APIPort struct {
+	PrivatePort int64
+	PublicPort  int64
+	Type        string
+	IP          string
+}
+
+type APIVersion struct {
+	Version   string
+	GitCommit string `json:",omitempty"`
+	GoVersion string `json:",omitempty"`
+}
+
+type APIWait struct {
+	StatusCode int
+}
+
+type APIAuth struct {
+	Status string
+}
+
+type APIImageConfig struct {
+	ID string `json:"Id"`
+	*Config
+}
+
+type APICopy struct {
+	Resource string
+	HostPath string
+}
+
+type APILink struct {
+	Path        string
+	ContainerID string
+	Image       string
+}

api_unit_test.go

@@ -1,19 +0,0 @@
-package docker
-
-import (
-	"testing"
-)
-
-func TestJsonContentType(t *testing.T) {
-	if !matchesContentType("application/json", "application/json") {
-		t.Fail()
-	}
-
-	if !matchesContentType("application/json; charset=utf-8", "application/json") {
-		t.Fail()
-	}
-
-	if matchesContentType("dockerapplication/json", "application/json") {
-		t.Fail()
-	}
-}

archive.go

@@ -1,4 +1,4 @@
-package archive
+package docker
 
 import (
 	"archive/tar"
@@ -15,15 +15,7 @@ import (
 
 type Archive io.Reader
 
-type Compression int
-
-type TarOptions struct {
-	Includes    []string
-	Excludes    []string
-	Recursive   bool
-	Compression Compression
-	CreateFiles []string
-}
+type Compression uint32
 
 const (
 	Uncompressed Compression = iota
@@ -88,7 +80,7 @@ func (compression *Compression) Extension() string {
 // Tar creates an archive from the directory at `path`, and returns it as a
 // stream of bytes.
 func Tar(path string, compression Compression) (io.Reader, error) {
-	return TarFilter(path, &TarOptions{Recursive: true, Compression: compression})
+	return TarFilter(path, compression, nil, true, nil)
 }
 
 func escapeName(name string) string {
@@ -109,29 +101,25 @@ func escapeName(name string) string {
 
 // Tar creates an archive from the directory at `path`, only including files whose relative
 // paths are included in `filter`. If `filter` is nil, then all files are included.
-func TarFilter(path string, options *TarOptions) (io.Reader, error) {
+func TarFilter(path string, compression Compression, filter []string, recursive bool, createFiles []string) (io.Reader, error) {
 	args := []string{"tar", "--numeric-owner", "-f", "-", "-C", path, "-T", "-"}
-	if options.Includes == nil {
-		options.Includes = []string{"."}
+	if filter == nil {
+		filter = []string{"."}
 	}
-	args = append(args, "-c"+options.Compression.Flag())
+	args = append(args, "-c"+compression.Flag())
 
-	for _, exclude := range options.Excludes {
-		args = append(args, fmt.Sprintf("--exclude=%s", exclude))
-	}
-
-	if !options.Recursive {
+	if !recursive {
 		args = append(args, "--no-recursion")
 	}
 
 	files := ""
-	for _, f := range options.Includes {
+	for _, f := range filter {
 		files = files + escapeName(f) + "\n"
 	}
 
 	tmpDir := ""
 
-	if options.CreateFiles != nil {
+	if createFiles != nil {
 		var err error // Can't use := here or we override the outer tmpDir
 		tmpDir, err = ioutil.TempDir("", "docker-tar")
 		if err != nil {
@@ -139,7 +127,7 @@ func TarFilter(path string, options *TarOptions) (io.Reader, error) {
 		}
 
 		files = files + "-C" + tmpDir + "\n"
-		for _, f := range options.CreateFiles {
+		for _, f := range createFiles {
 			path := filepath.Join(tmpDir, f)
 			err := os.MkdirAll(filepath.Dir(path), 0600)
 			if err != nil {
@@ -167,7 +155,7 @@ func TarFilter(path string, options *TarOptions) (io.Reader, error) {
 // The archive may be compressed with one of the following algorithms:
 //  identity (uncompressed), gzip, bzip2, xz.
 // FIXME: specify behavior when target path exists vs. doesn't exist.
-func Untar(archive io.Reader, path string, options *TarOptions) error {
+func Untar(archive io.Reader, path string) error {
 	if archive == nil {
 		return fmt.Errorf("Empty archive")
 	}
@@ -175,29 +163,21 @@ func Untar(archive io.Reader, path string, options *TarOptions) error {
 	buf := make([]byte, 10)
 	totalN := 0
 	for totalN < 10 {
-		n, err := archive.Read(buf[totalN:])
-		if err != nil {
+		if n, err := archive.Read(buf[totalN:]); err != nil {
 			if err == io.EOF {
 				return fmt.Errorf("Tarball too short")
 			}
 			return err
+		} else {
+			totalN += n
+			utils.Debugf("[tar autodetect] n: %d", n)
 		}
-		totalN += n
-		utils.Debugf("[tar autodetect] n: %d", n)
 	}
-
 	compression := DetectCompression(buf)
 
 	utils.Debugf("Archive compression detected: %s", compression.Extension())
-	args := []string{"--numeric-owner", "-f", "-", "-C", path, "-x" + compression.Flag()}
 
-	if options != nil {
-		for _, exclude := range options.Excludes {
-			args = append(args, fmt.Sprintf("--exclude=%s", exclude))
-		}
-	}
-
-	cmd := exec.Command("tar", args...)
+	cmd := exec.Command("tar", "--numeric-owner", "-f", "-", "-C", path, "-x"+compression.Flag())
 	cmd.Stdin = io.MultiReader(bytes.NewReader(buf), archive)
 	// Hardcode locale environment for predictable outcome regardless of host configuration.
 	//   (see https://github.com/dotcloud/docker/issues/355)
@@ -214,11 +194,11 @@ func Untar(archive io.Reader, path string, options *TarOptions) error {
 // TarUntar aborts and returns the error.
 func TarUntar(src string, filter []string, dst string) error {
 	utils.Debugf("TarUntar(%s %s %s)", src, filter, dst)
-	archive, err := TarFilter(src, &TarOptions{Compression: Uncompressed, Includes: filter, Recursive: true})
+	archive, err := TarFilter(src, Uncompressed, filter, true, nil)
 	if err != nil {
 		return err
 	}
-	return Untar(archive, dst, nil)
+	return Untar(archive, dst)
 }
 
 // UntarPath is a convenience function which looks for an archive
@@ -226,7 +206,7 @@ func TarUntar(src string, filter []string, dst string) error {
 func UntarPath(src, dst string) error {
 	if archive, err := os.Open(src); err != nil {
 		return err
-	} else if err := Untar(archive, dst, nil); err != nil {
+	} else if err := Untar(archive, dst); err != nil {
 		return err
 	}
 	return nil
@@ -295,7 +275,7 @@ func CopyFileWithTar(src, dst string) error {
 		return err
 	}
 	tw.Close()
-	return Untar(buf, filepath.Dir(dst), nil)
+	return Untar(buf, filepath.Dir(dst))
 }
 
 // CmdStream executes a command, and returns its stdout as a stream.

archive/changes_test.go

@@ -1,305 +0,0 @@
-package archive
-
-import (
-	"io/ioutil"
-	"os"
-	"os/exec"
-	"path"
-	"sort"
-	"testing"
-	"time"
-)
-
-func max(x, y int) int {
-	if x >= y {
-		return x
-	}
-	return y
-}
-
-func copyDir(src, dst string) error {
-	cmd := exec.Command("cp", "-a", src, dst)
-	if err := cmd.Run(); err != nil {
-		return err
-	}
-	return nil
-}
-
-// Helper to sort []Change by path
-type byPath struct{ changes []Change }
-
-func (b byPath) Less(i, j int) bool { return b.changes[i].Path < b.changes[j].Path }
-func (b byPath) Len() int           { return len(b.changes) }
-func (b byPath) Swap(i, j int)      { b.changes[i], b.changes[j] = b.changes[j], b.changes[i] }
-
-type FileType uint32
-
-const (
-	Regular FileType = iota
-	Dir
-	Symlink
-)
-
-type FileData struct {
-	filetype    FileType
-	path        string
-	contents    string
-	permissions os.FileMode
-}
-
-func createSampleDir(t *testing.T, root string) {
-	files := []FileData{
-		{Regular, "file1", "file1\n", 0600},
-		{Regular, "file2", "file2\n", 0666},
-		{Regular, "file3", "file3\n", 0404},
-		{Regular, "file4", "file4\n", 0600},
-		{Regular, "file5", "file5\n", 0600},
-		{Regular, "file6", "file6\n", 0600},
-		{Regular, "file7", "file7\n", 0600},
-		{Dir, "dir1", "", 0740},
-		{Regular, "dir1/file1-1", "file1-1\n", 01444},
-		{Regular, "dir1/file1-2", "file1-2\n", 0666},
-		{Dir, "dir2", "", 0700},
-		{Regular, "dir2/file2-1", "file2-1\n", 0666},
-		{Regular, "dir2/file2-2", "file2-2\n", 0666},
-		{Dir, "dir3", "", 0700},
-		{Regular, "dir3/file3-1", "file3-1\n", 0666},
-		{Regular, "dir3/file3-2", "file3-2\n", 0666},
-		{Dir, "dir4", "", 0700},
-		{Regular, "dir4/file3-1", "file4-1\n", 0666},
-		{Regular, "dir4/file3-2", "file4-2\n", 0666},
-		{Symlink, "symlink1", "target1", 0666},
-		{Symlink, "symlink2", "target2", 0666},
-	}
-
-	now := time.Now()
-	for _, info := range files {
-		p := path.Join(root, info.path)
-		if info.filetype == Dir {
-			if err := os.MkdirAll(p, info.permissions); err != nil {
-				t.Fatal(err)
-			}
-		} else if info.filetype == Regular {
-			if err := ioutil.WriteFile(p, []byte(info.contents), info.permissions); err != nil {
-				t.Fatal(err)
-			}
-		} else if info.filetype == Symlink {
-			if err := os.Symlink(info.contents, p); err != nil {
-				t.Fatal(err)
-			}
-		}
-
-		if info.filetype != Symlink {
-			// Set a consistent ctime, atime for all files and dirs
-			if err := os.Chtimes(p, now, now); err != nil {
-				t.Fatal(err)
-			}
-		}
-	}
-}
-
-// Create an directory, copy it, make sure we report no changes between the two
-func TestChangesDirsEmpty(t *testing.T) {
-	src, err := ioutil.TempDir("", "docker-changes-test")
-	if err != nil {
-		t.Fatal(err)
-	}
-	createSampleDir(t, src)
-	dst := src + "-copy"
-	if err := copyDir(src, dst); err != nil {
-		t.Fatal(err)
-	}
-	changes, err := ChangesDirs(dst, src)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	if len(changes) != 0 {
-		t.Fatalf("Reported changes for identical dirs: %v", changes)
-	}
-	os.RemoveAll(src)
-	os.RemoveAll(dst)
-}
-
-func mutateSampleDir(t *testing.T, root string) {
-	// Remove a regular file
-	if err := os.RemoveAll(path.Join(root, "file1")); err != nil {
-		t.Fatal(err)
-	}
-
-	// Remove a directory
-	if err := os.RemoveAll(path.Join(root, "dir1")); err != nil {
-		t.Fatal(err)
-	}
-
-	// Remove a symlink
-	if err := os.RemoveAll(path.Join(root, "symlink1")); err != nil {
-		t.Fatal(err)
-	}
-
-	// Rewrite a file
-	if err := ioutil.WriteFile(path.Join(root, "file2"), []byte("fileN\n"), 0777); err != nil {
-		t.Fatal(err)
-	}
-
-	// Replace a file
-	if err := os.RemoveAll(path.Join(root, "file3")); err != nil {
-		t.Fatal(err)
-	}
-	if err := ioutil.WriteFile(path.Join(root, "file3"), []byte("fileM\n"), 0404); err != nil {
-		t.Fatal(err)
-	}
-
-	// Touch file
-	if err := os.Chtimes(path.Join(root, "file4"), time.Now(), time.Now()); err != nil {
-		t.Fatal(err)
-	}
-
-	// Replace file with dir
-	if err := os.RemoveAll(path.Join(root, "file5")); err != nil {
-		t.Fatal(err)
-	}
-	if err := os.MkdirAll(path.Join(root, "file5"), 0666); err != nil {
-		t.Fatal(err)
-	}
-
-	// Create new file
-	if err := ioutil.WriteFile(path.Join(root, "filenew"), []byte("filenew\n"), 0777); err != nil {
-		t.Fatal(err)
-	}
-
-	// Create new dir
-	if err := os.MkdirAll(path.Join(root, "dirnew"), 0766); err != nil {
-		t.Fatal(err)
-	}
-
-	// Create a new symlink
-	if err := os.Symlink("targetnew", path.Join(root, "symlinknew")); err != nil {
-		t.Fatal(err)
-	}
-
-	// Change a symlink
-	if err := os.RemoveAll(path.Join(root, "symlink2")); err != nil {
-		t.Fatal(err)
-	}
-	if err := os.Symlink("target2change", path.Join(root, "symlink2")); err != nil {
-		t.Fatal(err)
-	}
-
-	// Replace dir with file
-	if err := os.RemoveAll(path.Join(root, "dir2")); err != nil {
-		t.Fatal(err)
-	}
-	if err := ioutil.WriteFile(path.Join(root, "dir2"), []byte("dir2\n"), 0777); err != nil {
-		t.Fatal(err)
-	}
-
-	// Touch dir
-	if err := os.Chtimes(path.Join(root, "dir3"), time.Now(), time.Now()); err != nil {
-		t.Fatal(err)
-	}
-}
-
-func TestChangesDirsMutated(t *testing.T) {
-	src, err := ioutil.TempDir("", "docker-changes-test")
-	if err != nil {
-		t.Fatal(err)
-	}
-	createSampleDir(t, src)
-	dst := src + "-copy"
-	if err := copyDir(src, dst); err != nil {
-		t.Fatal(err)
-	}
-	defer os.RemoveAll(src)
-	defer os.RemoveAll(dst)
-
-	mutateSampleDir(t, dst)
-
-	changes, err := ChangesDirs(dst, src)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	sort.Sort(byPath{changes})
-
-	expectedChanges := []Change{
-		{"/dir1", ChangeDelete},
-		{"/dir2", ChangeModify},
-		{"/dir3", ChangeModify},
-		{"/dirnew", ChangeAdd},
-		{"/file1", ChangeDelete},
-		{"/file2", ChangeModify},
-		{"/file3", ChangeModify},
-		{"/file4", ChangeModify},
-		{"/file5", ChangeModify},
-		{"/filenew", ChangeAdd},
-		{"/symlink1", ChangeDelete},
-		{"/symlink2", ChangeModify},
-		{"/symlinknew", ChangeAdd},
-	}
-
-	for i := 0; i < max(len(changes), len(expectedChanges)); i++ {
-		if i >= len(expectedChanges) {
-			t.Fatalf("unexpected change %s\n", changes[i].String())
-		}
-		if i >= len(changes) {
-			t.Fatalf("no change for expected change %s\n", expectedChanges[i].String())
-		}
-		if changes[i].Path == expectedChanges[i].Path {
-			if changes[i] != expectedChanges[i] {
-				t.Fatalf("Wrong change for %s, expected %s, got %s\n", changes[i].Path, changes[i].String(), expectedChanges[i].String())
-			}
-		} else if changes[i].Path < expectedChanges[i].Path {
-			t.Fatalf("unexpected change %s\n", changes[i].String())
-		} else {
-			t.Fatalf("no change for expected change %s != %s\n", expectedChanges[i].String(), changes[i].String())
-		}
-	}
-}
-
-func TestApplyLayer(t *testing.T) {
-	t.Skip("Skipping TestApplyLayer due to known failures") // Disable this for now as it is broken
-	return
-
-	// src, err := ioutil.TempDir("", "docker-changes-test")
-	// if err != nil {
-	// 	t.Fatal(err)
-	// }
-	// createSampleDir(t, src)
-	// dst := src + "-copy"
-	// if err := copyDir(src, dst); err != nil {
-	// 	t.Fatal(err)
-	// }
-	// mutateSampleDir(t, dst)
-
-	// changes, err := ChangesDirs(dst, src)
-	// if err != nil {
-	// 	t.Fatal(err)
-	// }
-
-	// layer, err := ExportChanges(dst, changes)
-	// if err != nil {
-	// 	t.Fatal(err)
-	// }
-
-	// layerCopy, err := NewTempArchive(layer, "")
-	// if err != nil {
-	// 	t.Fatal(err)
-	// }
-
-	// if err := ApplyLayer(src, layerCopy); err != nil {
-	// 	t.Fatal(err)
-	// }
-
-	// changes2, err := ChangesDirs(src, dst)
-	// if err != nil {
-	// 	t.Fatal(err)
-	// }
-
-	// if len(changes2) != 0 {
-	// 	t.Fatalf("Unexpected differences after re applying mutation: %v", changes)
-	// }
-
-	// os.RemoveAll(src)
-	// os.RemoveAll(dst)
-}

archive/diff.go

@@ -1,97 +0,0 @@
-package archive
-
-import (
-	"os"
-	"path/filepath"
-	"strings"
-	"syscall"
-	"time"
-)
-
-// ApplyLayer parses a diff in the standard layer format from `layer`, and
-// applies it to the directory `dest`.
-func ApplyLayer(dest string, layer Archive) error {
-	// Poor man's diff applyer in 2 steps:
-
-	// Step 1: untar everything in place
-	if err := Untar(layer, dest, nil); err != nil {
-		return err
-	}
-
-	modifiedDirs := make(map[string]*syscall.Stat_t)
-	addDir := func(file string) {
-		d := filepath.Dir(file)
-		if _, exists := modifiedDirs[d]; !exists {
-			if s, err := os.Lstat(d); err == nil {
-				if sys := s.Sys(); sys != nil {
-					if stat, ok := sys.(*syscall.Stat_t); ok {
-						modifiedDirs[d] = stat
-					}
-				}
-			}
-		}
-	}
-
-	// Step 2: walk for whiteouts and apply them, removing them in the process
-	err := filepath.Walk(dest, func(fullPath string, f os.FileInfo, err error) error {
-		if err != nil {
-			if os.IsNotExist(err) {
-				// This happens in the case of whiteouts in parent dir removing a directory
-				// We just ignore it
-				return filepath.SkipDir
-			}
-			return err
-		}
-
-		// Rebase path
-		path, err := filepath.Rel(dest, fullPath)
-		if err != nil {
-			return err
-		}
-		path = filepath.Join("/", path)
-
-		// Skip AUFS metadata
-		if matched, err := filepath.Match("/.wh..wh.*", path); err != nil {
-			return err
-		} else if matched {
-			addDir(fullPath)
-			if err := os.RemoveAll(fullPath); err != nil {
-				return err
-			}
-		}
-
-		filename := filepath.Base(path)
-		if strings.HasPrefix(filename, ".wh.") {
-			rmTargetName := filename[len(".wh."):]
-			rmTargetPath := filepath.Join(filepath.Dir(fullPath), rmTargetName)
-
-			// Remove the file targeted by the whiteout
-			addDir(rmTargetPath)
-			if err := os.RemoveAll(rmTargetPath); err != nil {
-				return err
-			}
-			// Remove the whiteout itself
-			addDir(fullPath)
-			if err := os.RemoveAll(fullPath); err != nil {
-				return err
-			}
-		}
-		return nil
-	})
-	if err != nil {
-		return err
-	}
-
-	for k, v := range modifiedDirs {
-		lastAccess := getLastAccess(v)
-		lastModification := getLastModification(v)
-		aTime := time.Unix(lastAccess.Unix())
-		mTime := time.Unix(lastModification.Unix())
-
-		if err := os.Chtimes(k, aTime, mTime); err != nil {
-			return err
-		}
-	}
-
-	return nil
-}

archive/stat_darwin.go

@@ -1,11 +0,0 @@
-package archive
-
-import "syscall"
-
-func getLastAccess(stat *syscall.Stat_t) syscall.Timespec {
-	return stat.Atimespec
-}
-
-func getLastModification(stat *syscall.Stat_t) syscall.Timespec {
-	return stat.Mtimespec
-}

archive/stat_linux.go

@@ -1,11 +0,0 @@
-package archive
-
-import "syscall"
-
-func getLastAccess(stat *syscall.Stat_t) syscall.Timespec {
-	return stat.Atim
-}
-
-func getLastModification(stat *syscall.Stat_t) syscall.Timespec {
-	return stat.Mtim
-}

archive_test.go

@@ -1,4 +1,4 @@
-package archive
+package docker
 
 import (
 	"bytes"
@@ -83,7 +83,7 @@ func tarUntar(t *testing.T, origin string, compression Compression) error {
 		return err
 	}
 	defer os.RemoveAll(tmp)
-	if err := Untar(archive, tmp, nil); err != nil {
+	if err := Untar(archive, tmp); err != nil {
 		return err
 	}
 	if _, err := os.Stat(tmp); err != nil {

auth/auth.go

@@ -63,7 +63,7 @@ func decodeAuth(authStr string) (string, string, error) {
 	if n > decLen {
 		return "", "", fmt.Errorf("Something went wrong decoding auth config")
 	}
-	arr := strings.SplitN(string(decoded), ":", 2)
+	arr := strings.Split(string(decoded), ":")
 	if len(arr) != 2 {
 		return "", "", fmt.Errorf("Invalid auth configuration file")
 	}
@@ -192,6 +192,14 @@ func Login(authConfig *AuthConfig, factory *utils.HTTPRequestFactory) (string, e
 		} else {
 			status = "Account created. Please see the documentation of the registry " + serverAddress + " for instructions how to activate it."
 		}
+	} else if reqStatusCode == 403 {
+		if loginAgainstOfficialIndex {
+			return "", fmt.Errorf("Login: Your account hasn't been activated. " +
+				"Please check your e-mail for a confirmation link.")
+		} else {
+			return "", fmt.Errorf("Login: Your account hasn't been activated. " +
+				"Please see the documentation of the registry " + serverAddress + " for instructions how to activate it.")
+		}
 	} else if reqStatusCode == 400 {
 		if string(reqBody) == "\"Username or email already exists\"" {
 			req, err := factory.NewRequest("GET", serverAddress+"users/", nil)
@@ -209,39 +217,13 @@ func Login(authConfig *AuthConfig, factory *utils.HTTPRequestFactory) (string, e
 				status = "Login Succeeded"
 			} else if resp.StatusCode == 401 {
 				return "", fmt.Errorf("Wrong login/password, please try again")
-			} else if resp.StatusCode == 403 {
-				if loginAgainstOfficialIndex {
-					return "", fmt.Errorf("Login: Account is not Active. Please check your e-mail for a confirmation link.")
-				}
-				return "", fmt.Errorf("Login: Account is not Active. Please see the documentation of the registry %s for instructions how to activate it.", serverAddress)
 			} else {
-				return "", fmt.Errorf("Login: %s (Code: %d; Headers: %s)", body, resp.StatusCode, resp.Header)
+				return "", fmt.Errorf("Login: %s (Code: %d; Headers: %s)", body,
+					resp.StatusCode, resp.Header)
 			}
 		} else {
 			return "", fmt.Errorf("Registration: %s", reqBody)
 		}
-	} else if reqStatusCode == 401 {
-		// This case would happen with private registries where /v1/users is
-		// protected, so people can use `docker login` as an auth check.
-		req, err := factory.NewRequest("GET", serverAddress+"users/", nil)
-		req.SetBasicAuth(authConfig.Username, authConfig.Password)
-		resp, err := client.Do(req)
-		if err != nil {
-			return "", err
-		}
-		defer resp.Body.Close()
-		body, err := ioutil.ReadAll(resp.Body)
-		if err != nil {
-			return "", err
-		}
-		if resp.StatusCode == 200 {
-			status = "Login Succeeded"
-		} else if resp.StatusCode == 401 {
-			return "", fmt.Errorf("Wrong login/password, please try again")
-		} else {
-			return "", fmt.Errorf("Login: %s (Code: %d; Headers: %s)", body,
-				resp.StatusCode, resp.Header)
-		}
 	} else {
 		return "", fmt.Errorf("Unexpected status code [%d] : %s", reqStatusCode, reqBody)
 	}

auth/auth_test.go

@@ -1,8 +1,11 @@
 package auth
 
 import (
+	"crypto/rand"
+	"encoding/hex"
 	"io/ioutil"
 	"os"
+	"strings"
 	"testing"
 )
 
@@ -26,6 +29,52 @@ func TestEncodeAuth(t *testing.T) {
 	}
 }
 
+func TestLogin(t *testing.T) {
+	os.Setenv("DOCKER_INDEX_URL", "https://indexstaging-docker.dotcloud.com")
+	defer os.Setenv("DOCKER_INDEX_URL", "")
+	authConfig := &AuthConfig{Username: "unittester", Password: "surlautrerivejetattendrai", Email: "noise+unittester@dotcloud.com"}
+	status, err := Login(authConfig, nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if status != "Login Succeeded" {
+		t.Fatalf("Expected status \"Login Succeeded\", found \"%s\" instead", status)
+	}
+}
+
+func TestCreateAccount(t *testing.T) {
+	os.Setenv("DOCKER_INDEX_URL", "https://indexstaging-docker.dotcloud.com")
+	defer os.Setenv("DOCKER_INDEX_URL", "")
+	tokenBuffer := make([]byte, 16)
+	_, err := rand.Read(tokenBuffer)
+	if err != nil {
+		t.Fatal(err)
+	}
+	token := hex.EncodeToString(tokenBuffer)[:12]
+	username := "ut" + token
+	authConfig := &AuthConfig{Username: username, Password: "test42", Email: "docker-ut+" + token + "@example.com"}
+	status, err := Login(authConfig, nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	expectedStatus := "Account created. Please use the confirmation link we sent" +
+		" to your e-mail to activate it."
+	if status != expectedStatus {
+		t.Fatalf("Expected status: \"%s\", found \"%s\" instead.", expectedStatus, status)
+	}
+
+	status, err = Login(authConfig, nil)
+	if err == nil {
+		t.Fatalf("Expected error but found nil instead")
+	}
+
+	expectedError := "Login: Account is not Active"
+
+	if !strings.Contains(err.Error(), expectedError) {
+		t.Fatalf("Expected message \"%s\" but found \"%s\" instead", expectedError, err)
+	}
+}
+
 func setupTempConfigFile() (*ConfigFile, error) {
 	root, err := ioutil.TempDir("", "docker-test-auth")
 	if err != nil {

buildfile.go

@@ -3,8 +3,6 @@ package docker
 import (
 	"encoding/json"
 	"fmt"
-	"github.com/dotcloud/docker/archive"
-	"github.com/dotcloud/docker/auth"
 	"github.com/dotcloud/docker/utils"
 	"io"
 	"io/ioutil"
@@ -34,24 +32,17 @@ type buildFile struct {
 	utilizeCache bool
 	rm           bool
 
-	authConfig *auth.AuthConfig
-
 	tmpContainers map[string]struct{}
 	tmpImages     map[string]struct{}
 
-	outStream io.Writer
-	errStream io.Writer
-
-	// Deprecated, original writer used for ImagePull. To be removed.
-	outOld io.Writer
-	sf     *utils.StreamFormatter
+	out io.Writer
 }
 
 func (b *buildFile) clearTmp(containers map[string]struct{}) {
 	for c := range containers {
 		tmp := b.runtime.Get(c)
 		b.runtime.Destroy(tmp)
-		fmt.Fprintf(b.outStream, "Removing intermediate container %s\n", utils.TruncateID(c))
+		fmt.Fprintf(b.out, "Removing intermediate container %s\n", utils.TruncateID(c))
 	}
 }
 
@@ -60,7 +51,7 @@ func (b *buildFile) CmdFrom(name string) error {
 	if err != nil {
 		if b.runtime.graph.IsNotExist(err) {
 			remote, tag := utils.ParseRepositoryTag(name)
-			if err := b.srv.ImagePull(remote, tag, b.outOld, b.sf, b.authConfig, nil, true); err != nil {
+			if err := b.srv.ImagePull(remote, tag, b.out, utils.NewStreamFormatter(false), nil, nil, true); err != nil {
 				return err
 			}
 			image, err = b.runtime.repositories.LookupImage(name)
@@ -73,9 +64,6 @@ func (b *buildFile) CmdFrom(name string) error {
 	}
 	b.image = image.ID
 	b.config = &Config{}
-	if image.Config != nil {
-		b.config = image.Config
-	}
 	if b.config.Env == nil || len(b.config.Env) == 0 {
 		b.config.Env = append(b.config.Env, "HOME=/", "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin")
 	}
@@ -108,7 +96,7 @@ func (b *buildFile) CmdRun(args string) error {
 		if cache, err := b.srv.ImageGetCached(b.image, b.config); err != nil {
 			return err
 		} else if cache != nil {
-			fmt.Fprintf(b.outStream, " ---> Using cache\n")
+			fmt.Fprintf(b.out, " ---> Using cache\n")
 			utils.Debugf("[BUILDER] Use cached version")
 			b.image = cache.ID
 			return nil
@@ -188,6 +176,8 @@ func (b *buildFile) CmdEnv(args string) error {
 func (b *buildFile) CmdCmd(args string) error {
 	var cmd []string
 	if err := json.Unmarshal([]byte(args), &cmd); err != nil {
+		// If the unmarshal fails, it is not an error, we just use the
+		// args as a string.
 		utils.Debugf("Error unmarshalling: %s, setting cmd to /bin/sh -c", err)
 		cmd = []string{"/bin/sh", "-c", args}
 	}
@@ -199,6 +189,9 @@ func (b *buildFile) CmdCmd(args string) error {
 }
 
 func (b *buildFile) CmdExpose(args string) error {
+	if strings.Contains(args, ":") {
+		return fmt.Errorf("EXPOSE cannot be used to bind to a host ip or port")
+	}
 	ports := strings.Split(args, " ")
 	b.config.PortSpecs = append(ports, b.config.PortSpecs...)
 	return b.commit("", b.config.Cmd, fmt.Sprintf("EXPOSE %v", ports))
@@ -249,7 +242,7 @@ func (b *buildFile) CmdVolume(args string) error {
 		volume = []string{args}
 	}
 	if b.config.Volumes == nil {
-		b.config.Volumes = map[string]struct{}{}
+		b.config.Volumes = NewPathOpts()
 	}
 	for _, v := range volume {
 		b.config.Volumes[v] = struct{}{}
@@ -261,7 +254,7 @@ func (b *buildFile) CmdVolume(args string) error {
 }
 
 func (b *buildFile) addRemote(container *Container, orig, dest string) error {
-	file, err := utils.Download(orig)
+	file, err := utils.Download(orig, ioutil.Discard)
 	if err != nil {
 		return err
 	}
@@ -296,24 +289,24 @@ func (b *buildFile) addContext(container *Container, orig, dest string) error {
 		destPath = destPath + "/"
 	}
 	if !strings.HasPrefix(origPath, b.context) {
-		return fmt.Errorf("Forbidden path outside the build context: %s (%s)", orig, origPath)
+		return fmt.Errorf("Forbidden path: %s", origPath)
 	}
 	fi, err := os.Stat(origPath)
 	if err != nil {
 		return fmt.Errorf("%s: no such file or directory", orig)
 	}
 	if fi.IsDir() {
-		if err := archive.CopyWithTar(origPath, destPath); err != nil {
+		if err := CopyWithTar(origPath, destPath); err != nil {
 			return err
 		}
 		// First try to unpack the source as an archive
-	} else if err := archive.UntarPath(origPath, destPath); err != nil {
-		utils.Debugf("Couldn't untar %s to %s: %s", origPath, destPath, err)
+	} else if err := UntarPath(origPath, destPath); err != nil {
+		utils.Debugf("[tar] Not a directory nor a tar archive. Copying as a file. Untar error from %s to %s: %s", origPath, destPath, err)
 		// If that fails, just copy it as a regular file
 		if err := os.MkdirAll(path.Dir(destPath), 0755); err != nil {
 			return err
 		}
-		if err := archive.CopyWithTar(origPath, destPath); err != nil {
+		if err := CopyWithTar(origPath, destPath); err != nil {
 			return err
 		}
 	}
@@ -344,7 +337,7 @@ func (b *buildFile) CmdAdd(args string) error {
 
 	b.config.Image = b.image
 	// Create the container and start it
-	container, _, err := b.runtime.Create(b.config, "")
+	container, _, err := b.runtime.Create(b.config)
 	if err != nil {
 		return err
 	}
@@ -372,34 +365,6 @@ func (b *buildFile) CmdAdd(args string) error {
 	return nil
 }
 
-type StdoutFormater struct {
-	io.Writer
-	*utils.StreamFormatter
-}
-
-func (sf *StdoutFormater) Write(buf []byte) (int, error) {
-	formattedBuf := sf.StreamFormatter.FormatStream(string(buf))
-	n, err := sf.Writer.Write(formattedBuf)
-	if n != len(formattedBuf) {
-		return n, io.ErrShortWrite
-	}
-	return len(buf), err
-}
-
-type StderrFormater struct {
-	io.Writer
-	*utils.StreamFormatter
-}
-
-func (sf *StderrFormater) Write(buf []byte) (int, error) {
-	formattedBuf := sf.StreamFormatter.FormatStream("\033[91m" + string(buf) + "\033[0m")
-	n, err := sf.Writer.Write(formattedBuf)
-	if n != len(formattedBuf) {
-		return n, io.ErrShortWrite
-	}
-	return len(buf), err
-}
-
 func (b *buildFile) run() (string, error) {
 	if b.image == "" {
 		return "", fmt.Errorf("Please provide a source image with `from` prior to run")
@@ -407,43 +372,33 @@ func (b *buildFile) run() (string, error) {
 	b.config.Image = b.image
 
 	// Create the container and start it
-	c, _, err := b.runtime.Create(b.config, "")
+	c, _, err := b.runtime.Create(b.config)
 	if err != nil {
 		return "", err
 	}
 	b.tmpContainers[c.ID] = struct{}{}
-	fmt.Fprintf(b.outStream, " ---> Running in %s\n", utils.TruncateID(c.ID))
+	fmt.Fprintf(b.out, " ---> Running in %s\n", utils.TruncateID(c.ID))
 
 	// override the entry point that may have been picked up from the base image
 	c.Path = b.config.Cmd[0]
 	c.Args = b.config.Cmd[1:]
 
-	var errCh chan error
-
-	if b.verbose {
-		errCh = utils.Go(func() error {
-			return <-c.Attach(nil, nil, b.outStream, b.errStream)
-		})
-	}
-
 	//start the container
-	if err := c.Start(); err != nil {
+	hostConfig := &HostConfig{}
+	if err := c.Start(hostConfig); err != nil {
 		return "", err
 	}
 
-	if errCh != nil {
-		if err := <-errCh; err != nil {
+	if b.verbose {
+		err = <-c.Attach(nil, nil, b.out, b.out)
+		if err != nil {
 			return "", err
 		}
 	}
 
 	// Wait for it to finish
 	if ret := c.Wait(); ret != 0 {
-		err := &utils.JSONError{
-			Message: fmt.Sprintf("The command %v returned a non-zero code: %d", b.config.Cmd, ret),
-			Code:    ret,
-		}
-		return "", err
+		return "", fmt.Errorf("The command %v returned a non-zero code: %d", b.config.Cmd, ret)
 	}
 
 	return c.ID, nil
@@ -464,7 +419,7 @@ func (b *buildFile) commit(id string, autoCmd []string, comment string) error {
 			if cache, err := b.srv.ImageGetCached(b.image, b.config); err != nil {
 				return err
 			} else if cache != nil {
-				fmt.Fprintf(b.outStream, " ---> Using cache\n")
+				fmt.Fprintf(b.out, " ---> Using cache\n")
 				utils.Debugf("[BUILDER] Use cached version")
 				b.image = cache.ID
 				return nil
@@ -473,15 +428,12 @@ func (b *buildFile) commit(id string, autoCmd []string, comment string) error {
 			}
 		}
 
-		container, warnings, err := b.runtime.Create(b.config, "")
+		container, _, err := b.runtime.Create(b.config)
 		if err != nil {
 			return err
 		}
-		for _, warning := range warnings {
-			fmt.Fprintf(b.outStream, " ---> [Warning] %s\n", warning)
-		}
 		b.tmpContainers[container.ID] = struct{}{}
-		fmt.Fprintf(b.outStream, " ---> Running in %s\n", utils.TruncateID(container.ID))
+		fmt.Fprintf(b.out, " ---> Running in %s\n", utils.TruncateID(container.ID))
 		id = container.ID
 		if err := container.EnsureMounted(); err != nil {
 			return err
@@ -516,7 +468,7 @@ func (b *buildFile) Build(context io.Reader) (string, error) {
 	if err != nil {
 		return "", err
 	}
-	if err := archive.Untar(context, name, nil); err != nil {
+	if err := Untar(context, name); err != nil {
 		return "", err
 	}
 	defer os.RemoveAll(name)
@@ -547,22 +499,22 @@ func (b *buildFile) Build(context io.Reader) (string, error) {
 
 		method, exists := reflect.TypeOf(b).MethodByName("Cmd" + strings.ToUpper(instruction[:1]) + strings.ToLower(instruction[1:]))
 		if !exists {
-			fmt.Fprintf(b.errStream, "# Skipping unknown instruction %s\n", strings.ToUpper(instruction))
+			fmt.Fprintf(b.out, "# Skipping unknown instruction %s\n", strings.ToUpper(instruction))
 			continue
 		}
 
 		stepN += 1
-		fmt.Fprintf(b.outStream, "Step %d : %s %s\n", stepN, strings.ToUpper(instruction), arguments)
+		fmt.Fprintf(b.out, "Step %d : %s %s\n", stepN, strings.ToUpper(instruction), arguments)
 
 		ret := method.Func.Call([]reflect.Value{reflect.ValueOf(b), reflect.ValueOf(arguments)})[0].Interface()
 		if ret != nil {
 			return "", ret.(error)
 		}
 
-		fmt.Fprintf(b.outStream, " ---> %s\n", utils.TruncateID(b.image))
+		fmt.Fprintf(b.out, " ---> %v\n", utils.TruncateID(b.image))
 	}
 	if b.image != "" {
-		fmt.Fprintf(b.outStream, "Successfully built %s\n", utils.TruncateID(b.image))
+		fmt.Fprintf(b.out, "Successfully built %s\n", utils.TruncateID(b.image))
 		if b.rm {
 			b.clearTmp(b.tmpContainers)
 		}
@@ -571,20 +523,16 @@ func (b *buildFile) Build(context io.Reader) (string, error) {
 	return "", fmt.Errorf("An error occurred during the build\n")
 }
 
-func NewBuildFile(srv *Server, outStream, errStream io.Writer, verbose, utilizeCache, rm bool, outOld io.Writer, sf *utils.StreamFormatter, auth *auth.AuthConfig) BuildFile {
+func NewBuildFile(srv *Server, out io.Writer, verbose, utilizeCache, rm bool) BuildFile {
 	return &buildFile{
 		runtime:       srv.runtime,
 		srv:           srv,
 		config:        &Config{},
-		outStream:     outStream,
-		errStream:     errStream,
+		out:           out,
 		tmpContainers: make(map[string]struct{}),
 		tmpImages:     make(map[string]struct{}),
 		verbose:       verbose,
 		utilizeCache:  utilizeCache,
 		rm:            rm,
-		sf:            sf,
-		authConfig:    auth,
-		outOld:        outOld,
 	}
 }

buildfile_test.go

@@ -2,10 +2,6 @@ package docker
 
 import (
 	"fmt"
-	"github.com/dotcloud/docker"
-	"github.com/dotcloud/docker/archive"
-	"github.com/dotcloud/docker/engine"
-	"github.com/dotcloud/docker/utils"
 	"io/ioutil"
 	"net"
 	"net/http"
@@ -16,8 +12,8 @@ import (
 
 // mkTestContext generates a build context from the contents of the provided dockerfile.
 // This context is suitable for use as an argument to BuildFile.Build()
-func mkTestContext(dockerfile string, files [][2]string, t *testing.T) archive.Archive {
-	context, err := docker.MkBuildContext(dockerfile, files)
+func mkTestContext(dockerfile string, files [][2]string, t *testing.T) Archive {
+	context, err := mkBuildContext(dockerfile, files)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -227,22 +223,21 @@ func mkTestingFileServer(files [][2]string) (*httptest.Server, error) {
 
 func TestBuild(t *testing.T) {
 	for _, ctx := range testContexts {
-		_, err := buildImage(ctx, t, nil, true)
-		if err != nil {
-			t.Fatal(err)
-		}
+		buildImage(ctx, t, nil, true)
 	}
 }
 
-func buildImage(context testContextTemplate, t *testing.T, eng *engine.Engine, useCache bool) (*docker.Image, error) {
-	if eng == nil {
-		eng = NewTestEngine(t)
-		runtime := mkRuntimeFromEngine(eng, t)
-		// FIXME: we might not need runtime, why not simply nuke
-		// the engine?
+func buildImage(context testContextTemplate, t *testing.T, srv *Server, useCache bool) *Image {
+	if srv == nil {
+		runtime := mkRuntime(t)
 		defer nuke(runtime)
+
+		srv = &Server{
+			runtime:     runtime,
+			pullingPool: make(map[string]struct{}),
+			pushingPool: make(map[string]struct{}),
+		}
 	}
-	srv := mkServerFromEngine(eng, t)
 
 	httpServer, err := mkTestingFileServer(context.remoteFiles)
 	if err != nil {
@@ -256,34 +251,28 @@ func buildImage(context testContextTemplate, t *testing.T, eng *engine.Engine, u
 	}
 	port := httpServer.URL[idx+1:]
 
-	iIP := eng.Hack_GetGlobalVar("httpapi.bridgeIP")
-	if iIP == nil {
-		t.Fatal("Legacy bridgeIP field not set in engine")
-	}
-	ip, ok := iIP.(net.IP)
-	if !ok {
-		panic("Legacy bridgeIP field in engine does not cast to net.IP")
-	}
+	ip := srv.runtime.networkManager.bridgeNetwork.IP
 	dockerfile := constructDockerfile(context.dockerfile, ip, port)
 
-	buildfile := docker.NewBuildFile(srv, ioutil.Discard, ioutil.Discard, false, useCache, false, ioutil.Discard, utils.NewStreamFormatter(false), nil)
+	buildfile := NewBuildFile(srv, ioutil.Discard, false, useCache, false)
 	id, err := buildfile.Build(mkTestContext(dockerfile, context.files, t))
 	if err != nil {
-		return nil, err
+		t.Fatal(err)
 	}
 
-	return srv.ImageInspect(id)
+	img, err := srv.ImageInspect(id)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return img
 }
 
 func TestVolume(t *testing.T) {
-	img, err := buildImage(testContextTemplate{`
+	img := buildImage(testContextTemplate{`
         from {IMAGE}
         volume /test
         cmd Hello world
     `, nil, nil}, t, nil, true)
-	if err != nil {
-		t.Fatal(err)
-	}
 
 	if len(img.Config.Volumes) == 0 {
 		t.Fail()
@@ -296,13 +285,10 @@ func TestVolume(t *testing.T) {
 }
 
 func TestBuildMaintainer(t *testing.T) {
-	img, err := buildImage(testContextTemplate{`
+	img := buildImage(testContextTemplate{`
         from {IMAGE}
         maintainer dockerio
     `, nil, nil}, t, nil, true)
-	if err != nil {
-		t.Fatal(err)
-	}
 
 	if img.Author != "dockerio" {
 		t.Fail()
@@ -310,13 +296,10 @@ func TestBuildMaintainer(t *testing.T) {
 }
 
 func TestBuildUser(t *testing.T) {
-	img, err := buildImage(testContextTemplate{`
+	img := buildImage(testContextTemplate{`
         from {IMAGE}
         user dockerio
     `, nil, nil}, t, nil, true)
-	if err != nil {
-		t.Fatal(err)
-	}
 
 	if img.Config.User != "dockerio" {
 		t.Fail()
@@ -324,15 +307,11 @@ func TestBuildUser(t *testing.T) {
 }
 
 func TestBuildEnv(t *testing.T) {
-	img, err := buildImage(testContextTemplate{`
+	img := buildImage(testContextTemplate{`
         from {IMAGE}
         env port 4243
         `,
 		nil, nil}, t, nil, true)
-	if err != nil {
-		t.Fatal(err)
-	}
-
 	hasEnv := false
 	for _, envVar := range img.Config.Env {
 		if envVar == "port=4243" {
@@ -346,14 +325,11 @@ func TestBuildEnv(t *testing.T) {
 }
 
 func TestBuildCmd(t *testing.T) {
-	img, err := buildImage(testContextTemplate{`
+	img := buildImage(testContextTemplate{`
         from {IMAGE}
         cmd ["/bin/echo", "Hello World"]
         `,
 		nil, nil}, t, nil, true)
-	if err != nil {
-		t.Fatal(err)
-	}
 
 	if img.Config.Cmd[0] != "/bin/echo" {
 		t.Log(img.Config.Cmd[0])
@@ -366,14 +342,11 @@ func TestBuildCmd(t *testing.T) {
 }
 
 func TestBuildExpose(t *testing.T) {
-	img, err := buildImage(testContextTemplate{`
+	img := buildImage(testContextTemplate{`
         from {IMAGE}
         expose 4243
         `,
 		nil, nil}, t, nil, true)
-	if err != nil {
-		t.Fatal(err)
-	}
 
 	if img.Config.PortSpecs[0] != "4243" {
 		t.Fail()
@@ -381,14 +354,11 @@ func TestBuildExpose(t *testing.T) {
 }
 
 func TestBuildEntrypoint(t *testing.T) {
-	img, err := buildImage(testContextTemplate{`
+	img := buildImage(testContextTemplate{`
         from {IMAGE}
         entrypoint ["/bin/echo"]
         `,
 		nil, nil}, t, nil, true)
-	if err != nil {
-		t.Fatal(err)
-	}
 
 	if img.Config.Entrypoint[0] != "/bin/echo" {
 	}
@@ -397,28 +367,28 @@ func TestBuildEntrypoint(t *testing.T) {
 // testing #1405 - config.Cmd does not get cleaned up if
 // utilizing cache
 func TestBuildEntrypointRunCleanup(t *testing.T) {
-	eng := NewTestEngine(t)
-	defer nuke(mkRuntimeFromEngine(eng, t))
+	runtime := mkRuntime(t)
+	defer nuke(runtime)
 
-	img, err := buildImage(testContextTemplate{`
+	srv := &Server{
+		runtime:     runtime,
+		pullingPool: make(map[string]struct{}),
+		pushingPool: make(map[string]struct{}),
+	}
+
+	img := buildImage(testContextTemplate{`
         from {IMAGE}
         run echo "hello"
         `,
-		nil, nil}, t, eng, true)
-	if err != nil {
-		t.Fatal(err)
-	}
+		nil, nil}, t, srv, true)
 
-	img, err = buildImage(testContextTemplate{`
+	img = buildImage(testContextTemplate{`
         from {IMAGE}
         run echo "hello"
         add foo /foo
         entrypoint ["/bin/echo"]
         `,
-		[][2]string{{"foo", "HEYO"}}, nil}, t, eng, true)
-	if err != nil {
-		t.Fatal(err)
-	}
+		[][2]string{{"foo", "HEYO"}}, nil}, t, srv, true)
 
 	if len(img.Config.Cmd) != 0 {
 		t.Fail()
@@ -426,8 +396,14 @@ func TestBuildEntrypointRunCleanup(t *testing.T) {
 }
 
 func TestBuildImageWithCache(t *testing.T) {
-	eng := NewTestEngine(t)
-	defer nuke(mkRuntimeFromEngine(eng, t))
+	runtime := mkRuntime(t)
+	defer nuke(runtime)
+
+	srv := &Server{
+		runtime:     runtime,
+		pullingPool: make(map[string]struct{}),
+		pushingPool: make(map[string]struct{}),
+	}
 
 	template := testContextTemplate{`
         from {IMAGE}
@@ -435,18 +411,11 @@ func TestBuildImageWithCache(t *testing.T) {
         `,
 		nil, nil}
 
-	img, err := buildImage(template, t, eng, true)
-	if err != nil {
-		t.Fatal(err)
-	}
-
+	img := buildImage(template, t, srv, true)
 	imageId := img.ID
 
 	img = nil
-	img, err = buildImage(template, t, eng, true)
-	if err != nil {
-		t.Fatal(err)
-	}
+	img = buildImage(template, t, srv, true)
 
 	if imageId != img.ID {
 		t.Logf("Image ids should match: %s != %s", imageId, img.ID)
@@ -455,8 +424,14 @@ func TestBuildImageWithCache(t *testing.T) {
 }
 
 func TestBuildImageWithoutCache(t *testing.T) {
-	eng := NewTestEngine(t)
-	defer nuke(mkRuntimeFromEngine(eng, t))
+	runtime := mkRuntime(t)
+	defer nuke(runtime)
+
+	srv := &Server{
+		runtime:     runtime,
+		pullingPool: make(map[string]struct{}),
+		pushingPool: make(map[string]struct{}),
+	}
 
 	template := testContextTemplate{`
         from {IMAGE}
@@ -464,17 +439,11 @@ func TestBuildImageWithoutCache(t *testing.T) {
         `,
 		nil, nil}
 
-	img, err := buildImage(template, t, eng, true)
-	if err != nil {
-		t.Fatal(err)
-	}
+	img := buildImage(template, t, srv, true)
 	imageId := img.ID
 
 	img = nil
-	img, err = buildImage(template, t, eng, false)
-	if err != nil {
-		t.Fatal(err)
-	}
+	img = buildImage(template, t, srv, false)
 
 	if imageId == img.ID {
 		t.Logf("Image ids should not match: %s == %s", imageId, img.ID)
@@ -483,9 +452,14 @@ func TestBuildImageWithoutCache(t *testing.T) {
 }
 
 func TestForbiddenContextPath(t *testing.T) {
-	eng := NewTestEngine(t)
-	defer nuke(mkRuntimeFromEngine(eng, t))
-	srv := mkServerFromEngine(eng, t)
+	runtime := mkRuntime(t)
+	defer nuke(runtime)
+
+	srv := &Server{
+		runtime:     runtime,
+		pullingPool: make(map[string]struct{}),
+		pushingPool: make(map[string]struct{}),
+	}
 
 	context := testContextTemplate{`
         from {IMAGE}
@@ -506,17 +480,10 @@ func TestForbiddenContextPath(t *testing.T) {
 	}
 	port := httpServer.URL[idx+1:]
 
-	iIP := eng.Hack_GetGlobalVar("httpapi.bridgeIP")
-	if iIP == nil {
-		t.Fatal("Legacy bridgeIP field not set in engine")
-	}
-	ip, ok := iIP.(net.IP)
-	if !ok {
-		panic("Legacy bridgeIP field in engine does not cast to net.IP")
-	}
+	ip := srv.runtime.networkManager.bridgeNetwork.IP
 	dockerfile := constructDockerfile(context.dockerfile, ip, port)
 
-	buildfile := docker.NewBuildFile(srv, ioutil.Discard, ioutil.Discard, false, true, false, ioutil.Discard, utils.NewStreamFormatter(false), nil)
+	buildfile := NewBuildFile(srv, ioutil.Discard, false, true, false)
 	_, err = buildfile.Build(mkTestContext(dockerfile, context.files, t))
 
 	if err == nil {
@@ -524,15 +491,21 @@ func TestForbiddenContextPath(t *testing.T) {
 		t.Fail()
 	}
 
-	if err.Error() != "Forbidden path outside the build context: ../../ (/)" {
+	if err.Error() != "Forbidden path: /" {
 		t.Logf("Error message is not expected: %s", err.Error())
 		t.Fail()
 	}
 }
 
 func TestBuildADDFileNotFound(t *testing.T) {
-	eng := NewTestEngine(t)
-	defer nuke(mkRuntimeFromEngine(eng, t))
+	runtime := mkRuntime(t)
+	defer nuke(runtime)
+
+	srv := &Server{
+		runtime:     runtime,
+		pullingPool: make(map[string]struct{}),
+		pushingPool: make(map[string]struct{}),
+	}
 
 	context := testContextTemplate{`
         from {IMAGE}
@@ -552,17 +525,10 @@ func TestBuildADDFileNotFound(t *testing.T) {
 	}
 	port := httpServer.URL[idx+1:]
 
-	iIP := eng.Hack_GetGlobalVar("httpapi.bridgeIP")
-	if iIP == nil {
-		t.Fatal("Legacy bridgeIP field not set in engine")
-	}
-	ip, ok := iIP.(net.IP)
-	if !ok {
-		panic("Legacy bridgeIP field in engine does not cast to net.IP")
-	}
+	ip := srv.runtime.networkManager.bridgeNetwork.IP
 	dockerfile := constructDockerfile(context.dockerfile, ip, port)
 
-	buildfile := docker.NewBuildFile(mkServerFromEngine(eng, t), ioutil.Discard, ioutil.Discard, false, true, false, ioutil.Discard, utils.NewStreamFormatter(false), nil)
+	buildfile := NewBuildFile(srv, ioutil.Discard, false, true, false)
 	_, err = buildfile.Build(mkTestContext(dockerfile, context.files, t))
 
 	if err == nil {
@@ -575,58 +541,3 @@ func TestBuildADDFileNotFound(t *testing.T) {
 		t.Fail()
 	}
 }
-
-func TestBuildInheritance(t *testing.T) {
-	eng := NewTestEngine(t)
-	defer nuke(mkRuntimeFromEngine(eng, t))
-
-	img, err := buildImage(testContextTemplate{`
-            from {IMAGE}
-            expose 4243
-            `,
-		nil, nil}, t, eng, true)
-
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	img2, _ := buildImage(testContextTemplate{fmt.Sprintf(`
-            from %s
-            entrypoint ["/bin/echo"]
-            `, img.ID),
-		nil, nil}, t, eng, true)
-
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	// from child
-	if img2.Config.Entrypoint[0] != "/bin/echo" {
-		t.Fail()
-	}
-
-	// from parent
-	if img.Config.PortSpecs[0] != "4243" {
-		t.Fail()
-	}
-}
-
-func TestBuildFails(t *testing.T) {
-	_, err := buildImage(testContextTemplate{`
-        from {IMAGE}
-        run sh -c "exit 23"
-        `,
-		nil, nil}, t, nil, true)
-
-	if err == nil {
-		t.Fatal("Error should not be nil")
-	}
-
-	sterr, ok := err.(*utils.JSONError)
-	if !ok {
-		t.Fatalf("Error should be utils.JSONError")
-	}
-	if sterr.Code != 23 {
-		t.Fatalf("StatusCode %d unexpected, should be 23", sterr.Code)
-	}
-}

changes.go

@@ -1,4 +1,4 @@
-package archive
+package docker
 
 import (
 	"fmt"
@@ -34,78 +34,6 @@ func (change *Change) String() string {
 	return fmt.Sprintf("%s %s", kind, change.Path)
 }
 
-func Changes(layers []string, rw string) ([]Change, error) {
-	var changes []Change
-	err := filepath.Walk(rw, func(path string, f os.FileInfo, err error) error {
-		if err != nil {
-			return err
-		}
-
-		// Rebase path
-		path, err = filepath.Rel(rw, path)
-		if err != nil {
-			return err
-		}
-		path = filepath.Join("/", path)
-
-		// Skip root
-		if path == "/" {
-			return nil
-		}
-
-		// Skip AUFS metadata
-		if matched, err := filepath.Match("/.wh..wh.*", path); err != nil || matched {
-			return err
-		}
-
-		change := Change{
-			Path: path,
-		}
-
-		// Find out what kind of modification happened
-		file := filepath.Base(path)
-		// If there is a whiteout, then the file was removed
-		if strings.HasPrefix(file, ".wh.") {
-			originalFile := file[len(".wh."):]
-			change.Path = filepath.Join(filepath.Dir(path), originalFile)
-			change.Kind = ChangeDelete
-		} else {
-			// Otherwise, the file was added
-			change.Kind = ChangeAdd
-
-			// ...Unless it already existed in a top layer, in which case, it's a modification
-			for _, layer := range layers {
-				stat, err := os.Stat(filepath.Join(layer, path))
-				if err != nil && !os.IsNotExist(err) {
-					return err
-				}
-				if err == nil {
-					// The file existed in the top layer, so that's a modification
-
-					// However, if it's a directory, maybe it wasn't actually modified.
-					// If you modify /foo/bar/baz, then /foo will be part of the changed files only because it's the parent of bar
-					if stat.IsDir() && f.IsDir() {
-						if f.Size() == stat.Size() && f.Mode() == stat.Mode() && f.ModTime() == stat.ModTime() {
-							// Both directories are the same, don't record the change
-							return nil
-						}
-					}
-					change.Kind = ChangeModify
-					break
-				}
-			}
-		}
-
-		// Record change
-		changes = append(changes, change)
-		return nil
-	})
-	if err != nil && !os.IsNotExist(err) {
-		return nil, err
-	}
-	return changes, nil
-}
-
 type FileInfo struct {
 	parent   *FileInfo
 	name     string
@@ -139,6 +67,21 @@ func (info *FileInfo) path() string {
 	return filepath.Join(info.parent.path(), info.name)
 }
 
+func (info *FileInfo) unlink() {
+	if info.parent != nil {
+		delete(info.parent.children, info.name)
+	}
+}
+
+func (info *FileInfo) Remove(path string) bool {
+	child := info.LookUp(path)
+	if child != nil {
+		child.unlink()
+		return true
+	}
+	return false
+}
+
 func (info *FileInfo) isDir() bool {
 	return info.parent == nil || info.stat.Mode&syscall.S_IFDIR == syscall.S_IFDIR
 }
@@ -175,13 +118,16 @@ func (info *FileInfo) addChanges(oldInfo *FileInfo, changes *[]Change) {
 			// be visible when actually comparing the stat fields. The only time this
 			// breaks down is if some code intentionally hides a change by setting
 			// back mtime
+			oldMtime := syscall.NsecToTimeval(oldStat.Mtim.Nano())
+			newMtime := syscall.NsecToTimeval(oldStat.Mtim.Nano())
 			if oldStat.Mode != newStat.Mode ||
 				oldStat.Uid != newStat.Uid ||
 				oldStat.Gid != newStat.Gid ||
 				oldStat.Rdev != newStat.Rdev ||
 				// Don't look at size for dirs, its not a good measure of change
 				(oldStat.Size != newStat.Size && oldStat.Mode&syscall.S_IFDIR != syscall.S_IFDIR) ||
-				getLastModification(oldStat) != getLastModification(newStat) {
+				oldMtime.Sec != newMtime.Sec ||
+				oldMtime.Usec != newMtime.Usec {
 				change := Change{
 					Path: newChild.path(),
 					Kind: ChangeModify,
@@ -222,6 +168,77 @@ func newRootFileInfo() *FileInfo {
 	return root
 }
 
+func applyLayer(root *FileInfo, layer string) error {
+	err := filepath.Walk(layer, func(layerPath string, f os.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+
+		// Skip root
+		if layerPath == layer {
+			return nil
+		}
+
+		// rebase path
+		relPath, err := filepath.Rel(layer, layerPath)
+		if err != nil {
+			return err
+		}
+		relPath = filepath.Join("/", relPath)
+
+		// Skip AUFS metadata
+		if matched, err := filepath.Match("/.wh..wh.*", relPath); err != nil || matched {
+			if err != nil || !f.IsDir() {
+				return err
+			}
+			return filepath.SkipDir
+		}
+
+		var layerStat syscall.Stat_t
+		err = syscall.Lstat(layerPath, &layerStat)
+		if err != nil {
+			return err
+		}
+
+		file := filepath.Base(relPath)
+		// If there is a whiteout, then the file was removed
+		if strings.HasPrefix(file, ".wh.") {
+			originalFile := file[len(".wh."):]
+			deletePath := filepath.Join(filepath.Dir(relPath), originalFile)
+
+			root.Remove(deletePath)
+		} else {
+			// Added or changed file
+			existing := root.LookUp(relPath)
+			if existing != nil {
+				// Changed file
+				existing.stat = layerStat
+				if !existing.isDir() {
+					// Changed from dir to non-dir, delete all previous files
+					existing.children = make(map[string]*FileInfo)
+				}
+			} else {
+				// Added file
+				parent := root.LookUp(filepath.Dir(relPath))
+				if parent == nil {
+					return fmt.Errorf("collectFileInfo: Unexpectedly no parent for %s", relPath)
+				}
+
+				info := &FileInfo{
+					name:     filepath.Base(relPath),
+					children: make(map[string]*FileInfo),
+					parent:   parent,
+					stat:     layerStat,
+				}
+
+				parent.children[info.name] = info
+			}
+		}
+		return nil
+	})
+	return err
+}
+
 func collectFileInfo(sourceDir string) (*FileInfo, error) {
 	root := newRootFileInfo()
 
@@ -266,6 +283,24 @@ func collectFileInfo(sourceDir string) (*FileInfo, error) {
 	return root, nil
 }
 
+// Compare a directory with an array of layer directories it was based on and
+// generate an array of Change objects describing the changes
+func ChangesLayers(newDir string, layers []string) ([]Change, error) {
+	newRoot, err := collectFileInfo(newDir)
+	if err != nil {
+		return nil, err
+	}
+	oldRoot := newRootFileInfo()
+	for i := len(layers) - 1; i >= 0; i-- {
+		layer := layers[i]
+		if err = applyLayer(oldRoot, layer); err != nil {
+			return nil, err
+		}
+	}
+
+	return newRoot.Changes(oldRoot), nil
+}
+
 // Compare two directories and generate an array of Change objects describing the changes
 func ChangesDirs(newDir, oldDir string) ([]Change, error) {
 	oldRoot, err := collectFileInfo(oldDir)
@@ -279,39 +314,3 @@ func ChangesDirs(newDir, oldDir string) ([]Change, error) {
 
 	return newRoot.Changes(oldRoot), nil
 }
-
-func ChangesSize(newDir string, changes []Change) int64 {
-	var size int64
-	for _, change := range changes {
-		if change.Kind == ChangeModify || change.Kind == ChangeAdd {
-			file := filepath.Join(newDir, change.Path)
-			fileInfo, _ := os.Lstat(file)
-			if fileInfo != nil && !fileInfo.IsDir() {
-				size += fileInfo.Size()
-			}
-		}
-	}
-	return size
-}
-
-func ExportChanges(dir string, changes []Change) (Archive, error) {
-	files := make([]string, 0)
-	deletions := make([]string, 0)
-	for _, change := range changes {
-		if change.Kind == ChangeModify || change.Kind == ChangeAdd {
-			files = append(files, change.Path)
-		}
-		if change.Kind == ChangeDelete {
-			base := filepath.Base(change.Path)
-			dir := filepath.Dir(change.Path)
-			deletions = append(deletions, filepath.Join(dir, ".wh."+base))
-		}
-	}
-	// FIXME: Why do we create whiteout files inside Tar code ?
-	return TarFilter(dir, &TarOptions{
-		Compression: Uncompressed,
-		Includes:    files,
-		Recursive:   false,
-		CreateFiles: deletions,
-	})
-}

commands_test.go

@@ -3,15 +3,9 @@ package docker
 import (
 	"bufio"
 	"fmt"
-	"github.com/dotcloud/docker"
-	"github.com/dotcloud/docker/engine"
-	"github.com/dotcloud/docker/term"
 	"github.com/dotcloud/docker/utils"
 	"io"
 	"io/ioutil"
-	"os"
-	"path"
-	"regexp"
 	"strings"
 	"testing"
 	"time"
@@ -32,47 +26,6 @@ func closeWrap(args ...io.Closer) error {
 	return nil
 }
 
-func setRaw(t *testing.T, c *docker.Container) *term.State {
-	pty, err := c.GetPtyMaster()
-	if err != nil {
-		t.Fatal(err)
-	}
-	state, err := term.MakeRaw(pty.Fd())
-	if err != nil {
-		t.Fatal(err)
-	}
-	return state
-}
-
-func unsetRaw(t *testing.T, c *docker.Container, state *term.State) {
-	pty, err := c.GetPtyMaster()
-	if err != nil {
-		t.Fatal(err)
-	}
-	term.RestoreTerminal(pty.Fd(), state)
-}
-
-func waitContainerStart(t *testing.T, timeout time.Duration) *docker.Container {
-	var container *docker.Container
-
-	setTimeout(t, "Waiting for the container to be started timed out", timeout, func() {
-		for {
-			l := globalRuntime.List()
-			if len(l) == 1 && l[0].State.IsRunning() {
-				container = l[0]
-				break
-			}
-			time.Sleep(10 * time.Millisecond)
-		}
-	})
-
-	if container == nil {
-		t.Fatal("An error occured while waiting for the container to start")
-	}
-
-	return container
-}
-
 func setTimeout(t *testing.T, msg string, d time.Duration, f func()) {
 	c := make(chan bool)
 
@@ -110,8 +63,8 @@ func assertPipe(input, output string, r io.Reader, w io.Writer, count int) error
 func TestRunHostname(t *testing.T) {
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	c := make(chan struct{})
 	go func() {
@@ -131,32 +84,18 @@ func TestRunHostname(t *testing.T) {
 		}
 	})
 
-	container := globalRuntime.List()[0]
-
-	setTimeout(t, "CmdRun timed out", 10*time.Second, func() {
+	setTimeout(t, "CmdRun timed out", 5*time.Second, func() {
 		<-c
-
-		go func() {
-			cli.CmdWait(container.ID)
-		}()
-
-		if _, err := bufio.NewReader(stdout).ReadString('\n'); err != nil {
-			t.Fatal(err)
-		}
 	})
 
-	// Cleanup pipes
-	if err := closeWrap(stdout, stdoutPipe); err != nil {
-		t.Fatal(err)
-	}
 }
 
 // TestRunWorkdir checks that 'docker run -w' correctly sets a custom working directory
 func TestRunWorkdir(t *testing.T) {
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	c := make(chan struct{})
 	go func() {
@@ -176,32 +115,18 @@ func TestRunWorkdir(t *testing.T) {
 		}
 	})
 
-	container := globalRuntime.List()[0]
-
-	setTimeout(t, "CmdRun timed out", 10*time.Second, func() {
+	setTimeout(t, "CmdRun timed out", 5*time.Second, func() {
 		<-c
-
-		go func() {
-			cli.CmdWait(container.ID)
-		}()
-
-		if _, err := bufio.NewReader(stdout).ReadString('\n'); err != nil {
-			t.Fatal(err)
-		}
 	})
 
-	// Cleanup pipes
-	if err := closeWrap(stdout, stdoutPipe); err != nil {
-		t.Fatal(err)
-	}
 }
 
 // TestRunWorkdirExists checks that 'docker run -w' correctly sets a custom working directory, even if it exists
 func TestRunWorkdirExists(t *testing.T) {
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	c := make(chan struct{})
 	go func() {
@@ -221,32 +146,18 @@ func TestRunWorkdirExists(t *testing.T) {
 		}
 	})
 
-	container := globalRuntime.List()[0]
-
 	setTimeout(t, "CmdRun timed out", 5*time.Second, func() {
 		<-c
-
-		go func() {
-			cli.CmdWait(container.ID)
-		}()
-
-		if _, err := bufio.NewReader(stdout).ReadString('\n'); err != nil {
-			t.Fatal(err)
-		}
 	})
 
-	// Cleanup pipes
-	if err := closeWrap(stdout, stdoutPipe); err != nil {
-		t.Fatal(err)
-	}
 }
 
 func TestRunExit(t *testing.T) {
 	stdin, stdinPipe := io.Pipe()
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	c1 := make(chan struct{})
 	go func() {
@@ -255,7 +166,7 @@ func TestRunExit(t *testing.T) {
 	}()
 
 	setTimeout(t, "Read/Write assertion timed out", 2*time.Second, func() {
-		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 150); err != nil {
+		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 15); err != nil {
 			t.Fatal(err)
 		}
 	})
@@ -298,8 +209,8 @@ func TestRunDisconnect(t *testing.T) {
 	stdin, stdinPipe := io.Pipe()
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	c1 := make(chan struct{})
 	go func() {
@@ -310,7 +221,7 @@ func TestRunDisconnect(t *testing.T) {
 	}()
 
 	setTimeout(t, "Read/Write assertion timed out", 2*time.Second, func() {
-		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 150); err != nil {
+		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 15); err != nil {
 			t.Fatal(err)
 		}
 	})
@@ -331,40 +242,48 @@ func TestRunDisconnect(t *testing.T) {
 	setTimeout(t, "Waiting for /bin/cat to exit timed out", 2*time.Second, func() {
 		container := globalRuntime.List()[0]
 		container.Wait()
-		if container.State.IsRunning() {
+		if container.State.Running {
 			t.Fatalf("/bin/cat is still running after closing stdin")
 		}
 	})
 }
 
-// Expected behaviour: the process stay alive when the client disconnects
-// but the client detaches.
+// Expected behaviour: the process dies when the client disconnects
 func TestRunDisconnectTty(t *testing.T) {
 
 	stdin, stdinPipe := io.Pipe()
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	c1 := make(chan struct{})
 	go func() {
-		defer close(c1)
 		// We're simulating a disconnect so the return value doesn't matter. What matters is the
 		// fact that CmdRun returns.
 		if err := cli.CmdRun("-i", "-t", unitTestImageID, "/bin/cat"); err != nil {
-			utils.Debugf("Error CmdRun: %s", err)
+			utils.Debugf("Error CmdRun: %s\n", err)
 		}
+
+		close(c1)
 	}()
 
-	container := waitContainerStart(t, 10*time.Second)
-
-	state := setRaw(t, container)
-	defer unsetRaw(t, container, state)
+	setTimeout(t, "Waiting for the container to be started timed out", 10*time.Second, func() {
+		for {
+			// Client disconnect after run -i should keep stdin out in TTY mode
+			l := globalRuntime.List()
+			if len(l) == 1 && l[0].State.Running {
+				break
+			}
+			time.Sleep(10 * time.Millisecond)
+		}
+	})
 
 	// Client disconnect after run -i should keep stdin out in TTY mode
-	setTimeout(t, "Read/Write assertion timed out", 2*time.Second, func() {
-		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 150); err != nil {
+	container := globalRuntime.List()[0]
+
+	setTimeout(t, "Read/Write assertion timed out", 2000*time.Second, func() {
+		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 15); err != nil {
 			t.Fatal(err)
 		}
 	})
@@ -374,16 +293,12 @@ func TestRunDisconnectTty(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	// wait for CmdRun to return
-	setTimeout(t, "Waiting for CmdRun timed out", 5*time.Second, func() {
-		<-c1
-	})
-
 	// In tty mode, we expect the process to stay alive even after client's stdin closes.
+	// Do not wait for run to finish
 
 	// Give some time to monitor to do his thing
 	container.WaitTimeout(500 * time.Millisecond)
-	if !container.State.IsRunning() {
+	if !container.State.Running {
 		t.Fatalf("/bin/cat should  still be running after closing stdin (tty mode)")
 	}
 }
@@ -396,8 +311,8 @@ func TestRunAttachStdin(t *testing.T) {
 	stdin, stdinPipe := io.Pipe()
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	ch := make(chan struct{})
 	go func() {
@@ -423,8 +338,8 @@ func TestRunAttachStdin(t *testing.T) {
 		if err != nil {
 			t.Fatal(err)
 		}
-		if cmdOutput != container.ID+"\n" {
-			t.Fatalf("Wrong output: should be '%s', not '%s'\n", container.ID+"\n", cmdOutput)
+		if cmdOutput != container.ShortID()+"\n" {
+			t.Fatalf("Wrong output: should be '%s', not '%s'\n", container.ShortID()+"\n", cmdOutput)
 		}
 	})
 
@@ -460,8 +375,8 @@ func TestRunDetach(t *testing.T) {
 	stdin, stdinPipe := io.Pipe()
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	ch := make(chan struct{})
 	go func() {
@@ -469,129 +384,45 @@ func TestRunDetach(t *testing.T) {
 		cli.CmdRun("-i", "-t", unitTestImageID, "cat")
 	}()
 
-	container := waitContainerStart(t, 10*time.Second)
-
-	state := setRaw(t, container)
-	defer unsetRaw(t, container, state)
-
 	setTimeout(t, "First read/write assertion timed out", 2*time.Second, func() {
-		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 150); err != nil {
+		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 15); err != nil {
 			t.Fatal(err)
 		}
 	})
 
+	container := globalRuntime.List()[0]
+
 	setTimeout(t, "Escape sequence timeout", 5*time.Second, func() {
-		stdinPipe.Write([]byte{16})
-		time.Sleep(100 * time.Millisecond)
-		stdinPipe.Write([]byte{17})
+		stdinPipe.Write([]byte{'', ''})
+		if err := stdinPipe.Close(); err != nil {
+			t.Fatal(err)
+		}
 	})
 
 	// wait for CmdRun to return
 	setTimeout(t, "Waiting for CmdRun timed out", 15*time.Second, func() {
 		<-ch
 	})
-	closeWrap(stdin, stdinPipe, stdout, stdoutPipe)
 
 	time.Sleep(500 * time.Millisecond)
-	if !container.State.IsRunning() {
+	if !container.State.Running {
 		t.Fatal("The detached container should be still running")
 	}
 
 	setTimeout(t, "Waiting for container to die timed out", 20*time.Second, func() {
 		container.Kill()
+		container.Wait()
 	})
 }
 
-// TestAttachDetach checks that attach in tty mode can be detached using the long container ID
+// TestAttachDetach checks that attach in tty mode can be detached
 func TestAttachDetach(t *testing.T) {
 	stdin, stdinPipe := io.Pipe()
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
-	ch := make(chan struct{})
-	go func() {
-		defer close(ch)
-		if err := cli.CmdRun("-i", "-t", "-d", unitTestImageID, "cat"); err != nil {
-			t.Fatal(err)
-		}
-	}()
-
-	container := waitContainerStart(t, 10*time.Second)
-
-	setTimeout(t, "Reading container's id timed out", 10*time.Second, func() {
-		buf := make([]byte, 1024)
-		n, err := stdout.Read(buf)
-		if err != nil {
-			t.Fatal(err)
-		}
-
-		if strings.Trim(string(buf[:n]), " \r\n") != container.ID {
-			t.Fatalf("Wrong ID received. Expect %s, received %s", container.ID, buf[:n])
-		}
-	})
-	setTimeout(t, "Starting container timed out", 10*time.Second, func() {
-		<-ch
-	})
-
-	state := setRaw(t, container)
-	defer unsetRaw(t, container, state)
-
-	stdin, stdinPipe = io.Pipe()
-	stdout, stdoutPipe = io.Pipe()
-	cli = docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-
-	ch = make(chan struct{})
-	go func() {
-		defer close(ch)
-		if err := cli.CmdAttach(container.ID); err != nil {
-			if err != io.ErrClosedPipe {
-				t.Fatal(err)
-			}
-		}
-	}()
-
-	setTimeout(t, "First read/write assertion timed out", 2*time.Second, func() {
-		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 150); err != nil {
-			if err != io.ErrClosedPipe {
-				t.Fatal(err)
-			}
-		}
-	})
-
-	setTimeout(t, "Escape sequence timeout", 5*time.Second, func() {
-		stdinPipe.Write([]byte{16})
-		time.Sleep(100 * time.Millisecond)
-		stdinPipe.Write([]byte{17})
-	})
-
-	// wait for CmdRun to return
-	setTimeout(t, "Waiting for CmdAttach timed out", 15*time.Second, func() {
-		<-ch
-	})
-
-	closeWrap(stdin, stdinPipe, stdout, stdoutPipe)
-
-	time.Sleep(500 * time.Millisecond)
-	if !container.State.IsRunning() {
-		t.Fatal("The detached container should be still running")
-	}
-
-	setTimeout(t, "Waiting for container to die timedout", 5*time.Second, func() {
-		container.Kill()
-	})
-}
-
-// TestAttachDetachTruncatedID checks that attach in tty mode can be detached
-func TestAttachDetachTruncatedID(t *testing.T) {
-	stdin, stdinPipe := io.Pipe()
-	stdout, stdoutPipe := io.Pipe()
-
-	cli := docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
-
-	// Discard the CmdRun output
 	go stdout.Read(make([]byte, 1024))
 	setTimeout(t, "Starting container timed out", 2*time.Second, func() {
 		if err := cli.CmdRun("-i", "-t", "-d", unitTestImageID, "cat"); err != nil {
@@ -599,52 +430,46 @@ func TestAttachDetachTruncatedID(t *testing.T) {
 		}
 	})
 
-	container := waitContainerStart(t, 10*time.Second)
-
-	state := setRaw(t, container)
-	defer unsetRaw(t, container, state)
+	container := globalRuntime.List()[0]
 
 	stdin, stdinPipe = io.Pipe()
 	stdout, stdoutPipe = io.Pipe()
-	cli = docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	cli = NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
 
 	ch := make(chan struct{})
 	go func() {
 		defer close(ch)
-		if err := cli.CmdAttach(utils.TruncateID(container.ID)); err != nil {
-			if err != io.ErrClosedPipe {
-				t.Fatal(err)
-			}
+		if err := cli.CmdAttach(container.ShortID()); err != nil {
+			t.Fatal(err)
 		}
 	}()
 
 	setTimeout(t, "First read/write assertion timed out", 2*time.Second, func() {
-		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 150); err != nil {
-			if err != io.ErrClosedPipe {
-				t.Fatal(err)
-			}
+		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 15); err != nil {
+			t.Fatal(err)
 		}
 	})
 
 	setTimeout(t, "Escape sequence timeout", 5*time.Second, func() {
-		stdinPipe.Write([]byte{16})
-		time.Sleep(100 * time.Millisecond)
-		stdinPipe.Write([]byte{17})
+		stdinPipe.Write([]byte{'', ''})
+		if err := stdinPipe.Close(); err != nil {
+			t.Fatal(err)
+		}
 	})
 
 	// wait for CmdRun to return
 	setTimeout(t, "Waiting for CmdAttach timed out", 15*time.Second, func() {
 		<-ch
 	})
-	closeWrap(stdin, stdinPipe, stdout, stdoutPipe)
 
 	time.Sleep(500 * time.Millisecond)
-	if !container.State.IsRunning() {
+	if !container.State.Running {
 		t.Fatal("The detached container should be still running")
 	}
 
 	setTimeout(t, "Waiting for container to die timedout", 5*time.Second, func() {
 		container.Kill()
+		container.Wait()
 	})
 }
 
@@ -653,13 +478,13 @@ func TestAttachDisconnect(t *testing.T) {
 	stdin, stdinPipe := io.Pipe()
 	stdout, stdoutPipe := io.Pipe()
 
-	cli := docker.NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(stdin, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	go func() {
 		// Start a process in daemon mode
 		if err := cli.CmdRun("-d", "-i", unitTestImageID, "/bin/cat"); err != nil {
-			utils.Debugf("Error CmdRun: %s", err)
+			utils.Debugf("Error CmdRun: %s\n", err)
 		}
 	}()
 
@@ -672,7 +497,7 @@ func TestAttachDisconnect(t *testing.T) {
 	setTimeout(t, "Waiting for the container to be started timed out", 10*time.Second, func() {
 		for {
 			l := globalRuntime.List()
-			if len(l) == 1 && l[0].State.IsRunning() {
+			if len(l) == 1 && l[0].State.Running {
 				break
 			}
 			time.Sleep(10 * time.Millisecond)
@@ -691,7 +516,7 @@ func TestAttachDisconnect(t *testing.T) {
 	}()
 
 	setTimeout(t, "First read/write assertion timed out", 2*time.Second, func() {
-		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 150); err != nil {
+		if err := assertPipe("hello\n", "hello", stdout, stdinPipe, 15); err != nil {
 			t.Fatal(err)
 		}
 	})
@@ -708,7 +533,7 @@ func TestAttachDisconnect(t *testing.T) {
 	// We closed stdin, expect /bin/cat to still be running
 	// Wait a little bit to make sure container.monitor() did his thing
 	err := container.WaitTimeout(500 * time.Millisecond)
-	if err == nil || !container.State.IsRunning() {
+	if err == nil || !container.State.Running {
 		t.Fatalf("/bin/cat is not running after closing stdin")
 	}
 
@@ -722,8 +547,8 @@ func TestAttachDisconnect(t *testing.T) {
 func TestRunAutoRemove(t *testing.T) {
 	t.Skip("Fixme. Skipping test for now, race condition")
 	stdout, stdoutPipe := io.Pipe()
-	cli := docker.NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
+	cli := NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
+	defer cleanup(globalRuntime)
 
 	c := make(chan struct{})
 	go func() {
@@ -745,7 +570,7 @@ func TestRunAutoRemove(t *testing.T) {
 		}
 	})
 
-	setTimeout(t, "CmdRun timed out", 10*time.Second, func() {
+	setTimeout(t, "CmdRun timed out", 5*time.Second, func() {
 		<-c
 	})
 
@@ -755,216 +580,3 @@ func TestRunAutoRemove(t *testing.T) {
 		t.Fatalf("failed to remove container automatically: container %s still exists", temporaryContainerID)
 	}
 }
-
-func TestCmdLogs(t *testing.T) {
-	t.Skip("Test not impemented")
-	cli := docker.NewDockerCli(nil, ioutil.Discard, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
-
-	if err := cli.CmdRun(unitTestImageID, "sh", "-c", "ls -l"); err != nil {
-		t.Fatal(err)
-	}
-	if err := cli.CmdRun("-t", unitTestImageID, "sh", "-c", "ls -l"); err != nil {
-		t.Fatal(err)
-	}
-
-	if err := cli.CmdLogs(globalRuntime.List()[0].ID); err != nil {
-		t.Fatal(err)
-	}
-}
-
-// Expected behaviour: error out when attempting to bind mount non-existing source paths
-func TestRunErrorBindNonExistingSource(t *testing.T) {
-
-	cli := docker.NewDockerCli(nil, nil, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
-
-	c := make(chan struct{})
-	go func() {
-		defer close(c)
-		// This check is made at runtime, can't be "unit tested"
-		if err := cli.CmdRun("-v", "/i/dont/exist:/tmp", unitTestImageID, "echo 'should fail'"); err == nil {
-			t.Fatal("should have failed to run when using /i/dont/exist as a source for the bind mount")
-		}
-	}()
-
-	setTimeout(t, "CmdRun timed out", 5*time.Second, func() {
-		<-c
-	})
-}
-
-func TestImagesViz(t *testing.T) {
-	stdout, stdoutPipe := io.Pipe()
-
-	cli := docker.NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
-
-	image := buildTestImages(t, globalEngine)
-
-	c := make(chan struct{})
-	go func() {
-		defer close(c)
-		if err := cli.CmdImages("-viz"); err != nil {
-			t.Fatal(err)
-		}
-		stdoutPipe.Close()
-	}()
-
-	setTimeout(t, "Reading command output time out", 2*time.Second, func() {
-		cmdOutputBytes, err := ioutil.ReadAll(bufio.NewReader(stdout))
-		if err != nil {
-			t.Fatal(err)
-		}
-		cmdOutput := string(cmdOutputBytes)
-
-		regexpStrings := []string{
-			"digraph docker {",
-			fmt.Sprintf("base -> \"%s\" \\[style=invis]", unitTestImageIDShort),
-			fmt.Sprintf("label=\"%s\\\\n%s:latest\"", unitTestImageIDShort, unitTestImageName),
-			fmt.Sprintf("label=\"%s\\\\n%s:%s\"", utils.TruncateID(image.ID), "test", "latest"),
-			"base \\[style=invisible]",
-		}
-
-		compiledRegexps := []*regexp.Regexp{}
-		for _, regexpString := range regexpStrings {
-			regexp, err := regexp.Compile(regexpString)
-			if err != nil {
-				fmt.Println("Error in regex string: ", err)
-				return
-			}
-			compiledRegexps = append(compiledRegexps, regexp)
-		}
-
-		for _, regexp := range compiledRegexps {
-			if !regexp.MatchString(cmdOutput) {
-				t.Fatalf("images -viz content '%s' did not match regexp '%s'", cmdOutput, regexp)
-			}
-		}
-	})
-}
-
-func TestImagesTree(t *testing.T) {
-	stdout, stdoutPipe := io.Pipe()
-
-	cli := docker.NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
-
-	image := buildTestImages(t, globalEngine)
-
-	c := make(chan struct{})
-	go func() {
-		defer close(c)
-		if err := cli.CmdImages("-tree"); err != nil {
-			t.Fatal(err)
-		}
-		stdoutPipe.Close()
-	}()
-
-	setTimeout(t, "Reading command output time out", 2*time.Second, func() {
-		cmdOutputBytes, err := ioutil.ReadAll(bufio.NewReader(stdout))
-		if err != nil {
-			t.Fatal(err)
-		}
-		cmdOutput := string(cmdOutputBytes)
-		regexpStrings := []string{
-			fmt.Sprintf("└─%s Virtual Size: \\d+.\\d+ MB Tags: %s:latest", unitTestImageIDShort, unitTestImageName),
-			"(?m)   └─[0-9a-f]+.*",
-			"(?m)    └─[0-9a-f]+.*",
-			"(?m)      └─[0-9a-f]+.*",
-			fmt.Sprintf("(?m)^        └─%s Virtual Size: \\d+.\\d+ MB Tags: test:latest", utils.TruncateID(image.ID)),
-		}
-
-		compiledRegexps := []*regexp.Regexp{}
-		for _, regexpString := range regexpStrings {
-			regexp, err := regexp.Compile(regexpString)
-			if err != nil {
-				fmt.Println("Error in regex string: ", err)
-				return
-			}
-			compiledRegexps = append(compiledRegexps, regexp)
-		}
-
-		for _, regexp := range compiledRegexps {
-			if !regexp.MatchString(cmdOutput) {
-				t.Fatalf("images -tree content '%s' did not match regexp '%s'", cmdOutput, regexp)
-			}
-		}
-	})
-}
-
-func buildTestImages(t *testing.T, eng *engine.Engine) *docker.Image {
-
-	var testBuilder = testContextTemplate{
-		`
-from   {IMAGE}
-run    sh -c 'echo root:testpass > /tmp/passwd'
-run    mkdir -p /var/run/sshd
-run    [ "$(cat /tmp/passwd)" = "root:testpass" ]
-run    [ "$(ls -d /var/run/sshd)" = "/var/run/sshd" ]
-`,
-		nil,
-		nil,
-	}
-	image, err := buildImage(testBuilder, t, eng, true)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	if err := eng.Job("tag", image.ID, "test").Run(); err != nil {
-		t.Fatal(err)
-	}
-
-	return image
-}
-
-// #2098 - Docker cidFiles only contain short version of the containerId
-//sudo docker run -cidfile /tmp/docker_test.cid ubuntu echo "test"
-// TestRunCidFile tests that run -cidfile returns the longid
-func TestRunCidFile(t *testing.T) {
-	stdout, stdoutPipe := io.Pipe()
-
-	tmpDir, err := ioutil.TempDir("", "TestRunCidFile")
-	if err != nil {
-		t.Fatal(err)
-	}
-	tmpCidFile := path.Join(tmpDir, "cid")
-
-	cli := docker.NewDockerCli(nil, stdoutPipe, ioutil.Discard, testDaemonProto, testDaemonAddr)
-	defer cleanup(globalEngine, t)
-
-	c := make(chan struct{})
-	go func() {
-		defer close(c)
-		if err := cli.CmdRun("-cidfile", tmpCidFile, unitTestImageID, "ls"); err != nil {
-			t.Fatal(err)
-		}
-	}()
-
-	defer os.RemoveAll(tmpDir)
-	setTimeout(t, "Reading command output time out", 2*time.Second, func() {
-		cmdOutput, err := bufio.NewReader(stdout).ReadString('\n')
-		if err != nil {
-			t.Fatal(err)
-		}
-		if len(cmdOutput) < 1 {
-			t.Fatalf("'ls' should return something , not '%s'", cmdOutput)
-		}
-		//read the tmpCidFile
-		buffer, err := ioutil.ReadFile(tmpCidFile)
-		if err != nil {
-			t.Fatal(err)
-		}
-		id := string(buffer)
-
-		if len(id) != len("2bf44ea18873287bd9ace8a4cb536a7cbe134bed67e805fdf2f58a57f69b320c") {
-			t.Fatalf("-cidfile should be a long id, not '%s'", id)
-		}
-		//test that its a valid cid? (though the container is gone..)
-		//remove the file and dir.
-	})
-
-	setTimeout(t, "CmdRun timed out", 5*time.Second, func() {
-		<-c
-	})
-
-}

commands_unit_test.go

@@ -1,157 +0,0 @@
-package docker
-
-import (
-	"strings"
-	"testing"
-)
-
-func parse(t *testing.T, args string) (*Config, *HostConfig, error) {
-	config, hostConfig, _, err := ParseRun(strings.Split(args+" ubuntu bash", " "), nil)
-	return config, hostConfig, err
-}
-
-func mustParse(t *testing.T, args string) (*Config, *HostConfig) {
-	config, hostConfig, err := parse(t, args)
-	if err != nil {
-		t.Fatal(err)
-	}
-	return config, hostConfig
-}
-
-func TestParseRunLinks(t *testing.T) {
-	if _, hostConfig := mustParse(t, "-link a:b"); len(hostConfig.Links) == 0 || hostConfig.Links[0] != "a:b" {
-		t.Fatalf("Error parsing links. Expected []string{\"a:b\"}, received: %v", hostConfig.Links)
-	}
-	if _, hostConfig := mustParse(t, "-link a:b -link c:d"); len(hostConfig.Links) < 2 || hostConfig.Links[0] != "a:b" || hostConfig.Links[1] != "c:d" {
-		t.Fatalf("Error parsing links. Expected []string{\"a:b\", \"c:d\"}, received: %v", hostConfig.Links)
-	}
-	if _, hostConfig := mustParse(t, ""); len(hostConfig.Links) != 0 {
-		t.Fatalf("Error parsing links. No link expected, received: %v", hostConfig.Links)
-	}
-
-	if _, _, err := parse(t, "-link a"); err == nil {
-		t.Fatalf("Error parsing links. `-link a` should be an error but is not")
-	}
-	if _, _, err := parse(t, "-link"); err == nil {
-		t.Fatalf("Error parsing links. `-link` should be an error but is not")
-	}
-}
-
-func TestParseRunAttach(t *testing.T) {
-	if config, _ := mustParse(t, "-a stdin"); !config.AttachStdin || config.AttachStdout || config.AttachStderr {
-		t.Fatalf("Error parsing attach flags. Expect only Stdin enabled. Received: in: %v, out: %v, err: %v", config.AttachStdin, config.AttachStdout, config.AttachStderr)
-	}
-	if config, _ := mustParse(t, "-a stdin -a stdout"); !config.AttachStdin || !config.AttachStdout || config.AttachStderr {
-		t.Fatalf("Error parsing attach flags. Expect only Stdin and Stdout enabled. Received: in: %v, out: %v, err: %v", config.AttachStdin, config.AttachStdout, config.AttachStderr)
-	}
-	if config, _ := mustParse(t, "-a stdin -a stdout -a stderr"); !config.AttachStdin || !config.AttachStdout || !config.AttachStderr {
-		t.Fatalf("Error parsing attach flags. Expect all attach enabled. Received: in: %v, out: %v, err: %v", config.AttachStdin, config.AttachStdout, config.AttachStderr)
-	}
-	if config, _ := mustParse(t, ""); config.AttachStdin || !config.AttachStdout || !config.AttachStderr {
-		t.Fatalf("Error parsing attach flags. Expect Stdin disabled. Received: in: %v, out: %v, err: %v", config.AttachStdin, config.AttachStdout, config.AttachStderr)
-	}
-
-	if _, _, err := parse(t, "-a"); err == nil {
-		t.Fatalf("Error parsing attach flags, `-a` should be an error but is not")
-	}
-	if _, _, err := parse(t, "-a invalid"); err == nil {
-		t.Fatalf("Error parsing attach flags, `-a invalid` should be an error but is not")
-	}
-	if _, _, err := parse(t, "-a invalid -a stdout"); err == nil {
-		t.Fatalf("Error parsing attach flags, `-a stdout -a invalid` should be an error but is not")
-	}
-	if _, _, err := parse(t, "-a stdout -a stderr -d"); err == nil {
-		t.Fatalf("Error parsing attach flags, `-a stdout -a stderr -d` should be an error but is not")
-	}
-	if _, _, err := parse(t, "-a stdin -d"); err == nil {
-		t.Fatalf("Error parsing attach flags, `-a stdin -d` should be an error but is not")
-	}
-	if _, _, err := parse(t, "-a stdout -d"); err == nil {
-		t.Fatalf("Error parsing attach flags, `-a stdout -d` should be an error but is not")
-	}
-	if _, _, err := parse(t, "-a stderr -d"); err == nil {
-		t.Fatalf("Error parsing attach flags, `-a stderr -d` should be an error but is not")
-	}
-	if _, _, err := parse(t, "-d -rm"); err == nil {
-		t.Fatalf("Error parsing attach flags, `-d -rm` should be an error but is not")
-	}
-}
-
-func TestParseRunVolumes(t *testing.T) {
-	if config, hostConfig := mustParse(t, "-v /tmp"); hostConfig.Binds != nil {
-		t.Fatalf("Error parsing volume flags, `-v /tmp` should not mount-bind anything. Received %v", hostConfig.Binds)
-	} else if _, exists := config.Volumes["/tmp"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /tmp` is missing from volumes. Received %v", config.Volumes)
-	}
-
-	if config, hostConfig := mustParse(t, "-v /tmp -v /var"); hostConfig.Binds != nil {
-		t.Fatalf("Error parsing volume flags, `-v /tmp -v /var` should not mount-bind anything. Received %v", hostConfig.Binds)
-	} else if _, exists := config.Volumes["/tmp"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /tmp` is missing from volumes. Recevied %v", config.Volumes)
-	} else if _, exists := config.Volumes["/var"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /var` is missing from volumes. Received %v", config.Volumes)
-	}
-
-	if config, hostConfig := mustParse(t, "-v /hostTmp:/containerTmp"); hostConfig.Binds == nil || hostConfig.Binds[0] != "/hostTmp:/containerTmp" {
-		t.Fatalf("Error parsing volume flags, `-v /hostTmp:/containerTmp` should mount-bind /hostTmp into /containeTmp. Received %v", hostConfig.Binds)
-	} else if _, exists := config.Volumes["/containerTmp"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /tmp` is missing from volumes. Received %v", config.Volumes)
-	}
-
-	if config, hostConfig := mustParse(t, "-v /hostTmp:/containerTmp -v /hostVar:/containerVar"); hostConfig.Binds == nil || hostConfig.Binds[0] != "/hostTmp:/containerTmp" || hostConfig.Binds[1] != "/hostVar:/containerVar" {
-		t.Fatalf("Error parsing volume flags, `-v /hostTmp:/containerTmp -v /hostVar:/containerVar` should mount-bind /hostTmp into /containeTmp and /hostVar into /hostContainer. Received %v", hostConfig.Binds)
-	} else if _, exists := config.Volumes["/containerTmp"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /containerTmp` is missing from volumes. Received %v", config.Volumes)
-	} else if _, exists := config.Volumes["/containerVar"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /containerVar` is missing from volumes. Received %v", config.Volumes)
-	}
-
-	if config, hostConfig := mustParse(t, "-v /hostTmp:/containerTmp:ro -v /hostVar:/containerVar:rw"); hostConfig.Binds == nil || hostConfig.Binds[0] != "/hostTmp:/containerTmp:ro" || hostConfig.Binds[1] != "/hostVar:/containerVar:rw" {
-		t.Fatalf("Error parsing volume flags, `-v /hostTmp:/containerTmp:ro -v /hostVar:/containerVar:rw` should mount-bind /hostTmp into /containeTmp and /hostVar into /hostContainer. Received %v", hostConfig.Binds)
-	} else if _, exists := config.Volumes["/containerTmp"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /containerTmp` is missing from volumes. Received %v", config.Volumes)
-	} else if _, exists := config.Volumes["/containerVar"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /containerVar` is missing from volumes. Received %v", config.Volumes)
-	}
-
-	if config, hostConfig := mustParse(t, "-v /hostTmp:/containerTmp -v /containerVar"); hostConfig.Binds == nil || len(hostConfig.Binds) > 1 || hostConfig.Binds[0] != "/hostTmp:/containerTmp" {
-		t.Fatalf("Error parsing volume flags, `-v /hostTmp:/containerTmp -v /containerVar` should mount-bind only /hostTmp into /containeTmp. Received %v", hostConfig.Binds)
-	} else if _, exists := config.Volumes["/containerTmp"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /containerTmp` is missing from volumes. Received %v", config.Volumes)
-	} else if _, exists := config.Volumes["/containerVar"]; !exists {
-		t.Fatalf("Error parsing volume flags, `-v /containerVar` is missing from volumes. Received %v", config.Volumes)
-	}
-
-	if config, hostConfig := mustParse(t, ""); hostConfig.Binds != nil {
-		t.Fatalf("Error parsing volume flags, without volume, nothing should be mount-binded. Received %v", hostConfig.Binds)
-	} else if len(config.Volumes) != 0 {
-		t.Fatalf("Error parsing volume flags, without volume, no volume should be present. Received %v", config.Volumes)
-	}
-
-	mustParse(t, "-v /")
-
-	if _, _, err := parse(t, "-v /:/"); err == nil {
-		t.Fatalf("Error parsing volume flags, `-v /:/` should fail but didn't")
-	}
-	if _, _, err := parse(t, "-v"); err == nil {
-		t.Fatalf("Error parsing volume flags, `-v` should fail but didn't")
-	}
-	if _, _, err := parse(t, "-v /tmp:"); err == nil {
-		t.Fatalf("Error parsing volume flags, `-v /tmp:` should fail but didn't")
-	}
-	if _, _, err := parse(t, "-v /tmp:ro"); err == nil {
-		t.Fatalf("Error parsing volume flags, `-v /tmp:ro` should fail but didn't")
-	}
-	if _, _, err := parse(t, "-v /tmp::"); err == nil {
-		t.Fatalf("Error parsing volume flags, `-v /tmp::` should fail but didn't")
-	}
-	if _, _, err := parse(t, "-v :"); err == nil {
-		t.Fatalf("Error parsing volume flags, `-v :` should fail but didn't")
-	}
-	if _, _, err := parse(t, "-v ::"); err == nil {
-		t.Fatalf("Error parsing volume flags, `-v ::` should fail but didn't")
-	}
-	if _, _, err := parse(t, "-v /tmp:/tmp:/tmp:/tmp"); err == nil {
-		t.Fatalf("Error parsing volume flags, `-v /tmp:/tmp:/tmp:/tmp` should fail but didn't")
-	}
-}

config.go

@@ -1,45 +1,18 @@
 package docker
 
 import (
-	"github.com/dotcloud/docker/engine"
 	"net"
 )
 
-// FIXME: separate runtime configuration from http api configuration
 type DaemonConfig struct {
 	Pidfile                     string
-	Root                        string
+	GraphPath                   string
+	ProtoAddresses              []string
 	AutoRestart                 bool
 	EnableCors                  bool
 	Dns                         []string
 	EnableIptables              bool
 	BridgeIface                 string
-	BridgeIp                    string
 	DefaultIp                   net.IP
 	InterContainerCommunication bool
-	GraphDriver                 string
-}
-
-// ConfigFromJob creates and returns a new DaemonConfig object
-// by parsing the contents of a job's environment.
-func ConfigFromJob(job *engine.Job) *DaemonConfig {
-	var config DaemonConfig
-	config.Pidfile = job.Getenv("Pidfile")
-	config.Root = job.Getenv("Root")
-	config.AutoRestart = job.GetenvBool("AutoRestart")
-	config.EnableCors = job.GetenvBool("EnableCors")
-	if dns := job.GetenvList("Dns"); dns != nil {
-		config.Dns = dns
-	}
-	config.EnableIptables = job.GetenvBool("EnableIptables")
-	if br := job.Getenv("BridgeIface"); br != "" {
-		config.BridgeIface = br
-	} else {
-		config.BridgeIface = DefaultNetworkBridge
-	}
-	config.BridgeIp = job.Getenv("BridgeIp")
-	config.DefaultIp = net.ParseIP(job.Getenv("DefaultIp"))
-	config.InterContainerCommunication = job.GetenvBool("InterContainerCommunication")
-	config.GraphDriver = job.Getenv("GraphDriver")
-	return &config
 }

config_test.go

@@ -1,149 +0,0 @@
-package docker
-
-import (
-	"testing"
-)
-
-func TestCompareConfig(t *testing.T) {
-	volumes1 := make(map[string]struct{})
-	volumes1["/test1"] = struct{}{}
-	config1 := Config{
-		Dns:         []string{"1.1.1.1", "2.2.2.2"},
-		PortSpecs:   []string{"1111:1111", "2222:2222"},
-		Env:         []string{"VAR1=1", "VAR2=2"},
-		VolumesFrom: "11111111",
-		Volumes:     volumes1,
-	}
-	config2 := Config{
-		Dns:         []string{"0.0.0.0", "2.2.2.2"},
-		PortSpecs:   []string{"1111:1111", "2222:2222"},
-		Env:         []string{"VAR1=1", "VAR2=2"},
-		VolumesFrom: "11111111",
-		Volumes:     volumes1,
-	}
-	config3 := Config{
-		Dns:         []string{"1.1.1.1", "2.2.2.2"},
-		PortSpecs:   []string{"0000:0000", "2222:2222"},
-		Env:         []string{"VAR1=1", "VAR2=2"},
-		VolumesFrom: "11111111",
-		Volumes:     volumes1,
-	}
-	config4 := Config{
-		Dns:         []string{"1.1.1.1", "2.2.2.2"},
-		PortSpecs:   []string{"0000:0000", "2222:2222"},
-		Env:         []string{"VAR1=1", "VAR2=2"},
-		VolumesFrom: "22222222",
-		Volumes:     volumes1,
-	}
-	volumes2 := make(map[string]struct{})
-	volumes2["/test2"] = struct{}{}
-	config5 := Config{
-		Dns:         []string{"1.1.1.1", "2.2.2.2"},
-		PortSpecs:   []string{"0000:0000", "2222:2222"},
-		Env:         []string{"VAR1=1", "VAR2=2"},
-		VolumesFrom: "11111111",
-		Volumes:     volumes2,
-	}
-	if CompareConfig(&config1, &config2) {
-		t.Fatalf("CompareConfig should return false, Dns are different")
-	}
-	if CompareConfig(&config1, &config3) {
-		t.Fatalf("CompareConfig should return false, PortSpecs are different")
-	}
-	if CompareConfig(&config1, &config4) {
-		t.Fatalf("CompareConfig should return false, VolumesFrom are different")
-	}
-	if CompareConfig(&config1, &config5) {
-		t.Fatalf("CompareConfig should return false, Volumes are different")
-	}
-	if !CompareConfig(&config1, &config1) {
-		t.Fatalf("CompareConfig should return true")
-	}
-}
-
-func TestMergeConfig(t *testing.T) {
-	volumesImage := make(map[string]struct{})
-	volumesImage["/test1"] = struct{}{}
-	volumesImage["/test2"] = struct{}{}
-	configImage := &Config{
-		Dns:         []string{"1.1.1.1", "2.2.2.2"},
-		PortSpecs:   []string{"1111:1111", "2222:2222"},
-		Env:         []string{"VAR1=1", "VAR2=2"},
-		VolumesFrom: "1111",
-		Volumes:     volumesImage,
-	}
-
-	volumesUser := make(map[string]struct{})
-	volumesUser["/test3"] = struct{}{}
-	configUser := &Config{
-		Dns:       []string{"3.3.3.3"},
-		PortSpecs: []string{"3333:2222", "3333:3333"},
-		Env:       []string{"VAR2=3", "VAR3=3"},
-		Volumes:   volumesUser,
-	}
-
-	if err := MergeConfig(configUser, configImage); err != nil {
-		t.Error(err)
-	}
-
-	if len(configUser.Dns) != 3 {
-		t.Fatalf("Expected 3 dns, 1.1.1.1, 2.2.2.2 and 3.3.3.3, found %d", len(configUser.Dns))
-	}
-	for _, dns := range configUser.Dns {
-		if dns != "1.1.1.1" && dns != "2.2.2.2" && dns != "3.3.3.3" {
-			t.Fatalf("Expected 1.1.1.1 or 2.2.2.2 or 3.3.3.3, found %s", dns)
-		}
-	}
-
-	if len(configUser.ExposedPorts) != 3 {
-		t.Fatalf("Expected 3 ExposedPorts, 1111, 2222 and 3333, found %d", len(configUser.ExposedPorts))
-	}
-	for portSpecs := range configUser.ExposedPorts {
-		if portSpecs.Port() != "1111" && portSpecs.Port() != "2222" && portSpecs.Port() != "3333" {
-			t.Fatalf("Expected 1111 or 2222 or 3333, found %s", portSpecs)
-		}
-	}
-	if len(configUser.Env) != 3 {
-		t.Fatalf("Expected 3 env var, VAR1=1, VAR2=3 and VAR3=3, found %d", len(configUser.Env))
-	}
-	for _, env := range configUser.Env {
-		if env != "VAR1=1" && env != "VAR2=3" && env != "VAR3=3" {
-			t.Fatalf("Expected VAR1=1 or VAR2=3 or VAR3=3, found %s", env)
-		}
-	}
-
-	if len(configUser.Volumes) != 3 {
-		t.Fatalf("Expected 3 volumes, /test1, /test2 and /test3, found %d", len(configUser.Volumes))
-	}
-	for v := range configUser.Volumes {
-		if v != "/test1" && v != "/test2" && v != "/test3" {
-			t.Fatalf("Expected /test1 or /test2 or /test3, found %s", v)
-		}
-	}
-
-	if configUser.VolumesFrom != "1111" {
-		t.Fatalf("Expected VolumesFrom to be 1111, found %s", configUser.VolumesFrom)
-	}
-
-	ports, _, err := parsePortSpecs([]string{"0000"})
-	if err != nil {
-		t.Error(err)
-	}
-	configImage2 := &Config{
-		ExposedPorts: ports,
-	}
-
-	if err := MergeConfig(configUser, configImage2); err != nil {
-		t.Error(err)
-	}
-
-	if len(configUser.ExposedPorts) != 4 {
-		t.Fatalf("Expected 4 ExposedPorts, 0000, 1111, 2222 and 3333, found %d", len(configUser.ExposedPorts))
-	}
-	for portSpecs := range configUser.ExposedPorts {
-		if portSpecs.Port() != "0000" && portSpecs.Port() != "1111" && portSpecs.Port() != "2222" && portSpecs.Port() != "3333" {
-			t.Fatalf("Expected 0000 or 1111 or 2222 or 3333, found %s", portSpecs)
-		}
-	}
-
-}

container_unit_test.go

@@ -1,161 +0,0 @@
-package docker
-
-import (
-	"testing"
-)
-
-func TestParseLxcConfOpt(t *testing.T) {
-	opts := []string{"lxc.utsname=docker", "lxc.utsname = docker "}
-
-	for _, o := range opts {
-		k, v, err := parseLxcOpt(o)
-		if err != nil {
-			t.FailNow()
-		}
-		if k != "lxc.utsname" {
-			t.Fail()
-		}
-		if v != "docker" {
-			t.Fail()
-		}
-	}
-}
-
-func TestParseNetworkOptsPrivateOnly(t *testing.T) {
-	ports, bindings, err := parsePortSpecs([]string{"192.168.1.100::80"})
-	if err != nil {
-		t.Fatal(err)
-	}
-	if len(ports) != 1 {
-		t.Logf("Expected 1 got %d", len(ports))
-		t.FailNow()
-	}
-	if len(bindings) != 1 {
-		t.Logf("Expected 1 got %d", len(bindings))
-		t.FailNow()
-	}
-	for k := range ports {
-		if k.Proto() != "tcp" {
-			t.Logf("Expected tcp got %s", k.Proto())
-			t.Fail()
-		}
-		if k.Port() != "80" {
-			t.Logf("Expected 80 got %s", k.Port())
-			t.Fail()
-		}
-		b, exists := bindings[k]
-		if !exists {
-			t.Log("Binding does not exist")
-			t.FailNow()
-		}
-		if len(b) != 1 {
-			t.Logf("Expected 1 got %d", len(b))
-			t.FailNow()
-		}
-		s := b[0]
-		if s.HostPort != "" {
-			t.Logf("Expected \"\" got %s", s.HostPort)
-			t.Fail()
-		}
-		if s.HostIp != "192.168.1.100" {
-			t.Fail()
-		}
-	}
-}
-
-func TestParseNetworkOptsPublic(t *testing.T) {
-	ports, bindings, err := parsePortSpecs([]string{"192.168.1.100:8080:80"})
-	if err != nil {
-		t.Fatal(err)
-	}
-	if len(ports) != 1 {
-		t.Logf("Expected 1 got %d", len(ports))
-		t.FailNow()
-	}
-	if len(bindings) != 1 {
-		t.Logf("Expected 1 got %d", len(bindings))
-		t.FailNow()
-	}
-	for k := range ports {
-		if k.Proto() != "tcp" {
-			t.Logf("Expected tcp got %s", k.Proto())
-			t.Fail()
-		}
-		if k.Port() != "80" {
-			t.Logf("Expected 80 got %s", k.Port())
-			t.Fail()
-		}
-		b, exists := bindings[k]
-		if !exists {
-			t.Log("Binding does not exist")
-			t.FailNow()
-		}
-		if len(b) != 1 {
-			t.Logf("Expected 1 got %d", len(b))
-			t.FailNow()
-		}
-		s := b[0]
-		if s.HostPort != "8080" {
-			t.Logf("Expected 8080 got %s", s.HostPort)
-			t.Fail()
-		}
-		if s.HostIp != "192.168.1.100" {
-			t.Fail()
-		}
-	}
-}
-
-func TestParseNetworkOptsUdp(t *testing.T) {
-	ports, bindings, err := parsePortSpecs([]string{"192.168.1.100::6000/udp"})
-	if err != nil {
-		t.Fatal(err)
-	}
-	if len(ports) != 1 {
-		t.Logf("Expected 1 got %d", len(ports))
-		t.FailNow()
-	}
-	if len(bindings) != 1 {
-		t.Logf("Expected 1 got %d", len(bindings))
-		t.FailNow()
-	}
-	for k := range ports {
-		if k.Proto() != "udp" {
-			t.Logf("Expected udp got %s", k.Proto())
-			t.Fail()
-		}
-		if k.Port() != "6000" {
-			t.Logf("Expected 6000 got %s", k.Port())
-			t.Fail()
-		}
-		b, exists := bindings[k]
-		if !exists {
-			t.Log("Binding does not exist")
-			t.FailNow()
-		}
-		if len(b) != 1 {
-			t.Logf("Expected 1 got %d", len(b))
-			t.FailNow()
-		}
-		s := b[0]
-		if s.HostPort != "" {
-			t.Logf("Expected \"\" got %s", s.HostPort)
-			t.Fail()
-		}
-		if s.HostIp != "192.168.1.100" {
-			t.Fail()
-		}
-	}
-}
-
-func TestGetFullName(t *testing.T) {
-	name, err := getFullName("testing")
-	if err != nil {
-		t.Fatal(err)
-	}
-	if name != "/testing" {
-		t.Fatalf("Expected /testing got %s", name)
-	}
-	if _, err := getFullName(""); err == nil {
-		t.Fatal("Error should not be nil")
-	}
-}

contrib/brew/.gitignore

@@ -0,0 +1 @@
+*.pyc

contrib/brew/README.md

@@ -0,0 +1,78 @@
+# docker-brew
+
+docker-brew is a command-line tool used to build the docker standard library.
+
+## Install instructions
+
+1. Install python if it isn't already available on your OS of choice
+1. Install the easy_install tool (`sudo apt-get install python-setuptools`
+for Debian)
+1. Install the python package manager, `pip` (`easy_install pip`)
+1. Run the following command: `sudo pip install -r requirements.txt`
+1. You should now be able to use the `docker-brew` script as such.
+
+## Basics
+
+	./docker-brew -h
+
+Display usage and help.
+
+	./docker-brew
+
+Default build from the default repo/branch. Images will be created under the
+`library/` namespace. Does not perform a remote push.
+
+	./docker-brew -n mycorp.com -b stable --push git://github.com/mycorp/docker
+
+Will fetch the library definition files in the `stable` branch of the
+`git://github.com/mycorp/docker` repository and create images under the
+`mycorp.com` namespace (e.g. `mycorp.com/ubuntu`). Created images will then
+be pushed to the official docker repository (pending: support for private
+repositories)
+
+## Library definition files
+
+The library definition files are plain text files found in the `library/`
+subfolder of the docker repository.
+
+### File names
+
+The name of a definition file will determine the name of the image(s) it
+creates. For example, the `library/ubuntu` file will create images in the
+`<namespace>/ubuntu` repository. If multiple instructions are present in
+a single file, all images are expected to be created under a different tag.
+
+### Instruction format
+
+Each line represents a build instruction.
+There are different formats that `docker-brew` is able to parse.
+
+	<git-url>
+	git://github.com/dotcloud/hipache
+	https://github.com/dotcloud/docker.git
+
+The simplest format. `docker-brew` will fetch data from the provided git
+repository from the `HEAD`of its `master` branch. Generated image will be
+tagged as `latest`. Use of this format is discouraged because there is no
+way to ensure stability.
+
+	<docker-tag> <git-url>
+	bleeding-edge git://github.com/dotcloud/docker
+	unstable https://github.com/dotcloud/docker-redis.git
+
+A more advanced format. `docker-brew` will fetch data from the provided git
+repository from the `HEAD`of its `master` branch. Generated image will be
+tagged as `<docker-tag>`. Recommended if we always want to provide a snapshot
+of the latest development. Again, no way to ensure stability.
+
+	<docker-tag>	<git-url>	T:<git-tag>
+	2.4.0 	git://github.com/dotcloud/docker-redis	T:2.4.0
+	<docker-tag>	<git-url>	B:<git-branch>
+	zfs		git://github.com/dotcloud/docker	B:zfs-support
+	<docker-tag>	<git-url>	C:<git-commit-id>
+	2.2.0 	https://github.com/dotcloud/docker-redis.git C:a4bf8923ee4ec566d3ddc212
+
+The most complete format. `docker-brew` will fetch data from the provided git
+repository from the provided reference (if it's a branch, brew will fetch its
+`HEAD`). Generated image will be tagged as `<docker-tag>`. Recommended whenever
+possible.

contrib/brew/brew/__init__.py

@@ -0,0 +1 @@
+from brew import build_library, DEFAULT_REPOSITORY, DEFAULT_BRANCH

contrib/brew/brew/brew.py

@@ -0,0 +1,185 @@
+import os
+import logging
+from shutil import rmtree
+
+import docker
+
+import git
+
+DEFAULT_REPOSITORY = 'git://github.com/dotcloud/docker'
+DEFAULT_BRANCH = 'master'
+
+logger = logging.getLogger(__name__)
+logging.basicConfig(format='%(asctime)s %(levelname)s %(message)s',
+                    level='INFO')
+client = docker.Client()
+processed = {}
+processed_folders = []
+
+
+def build_library(repository=None, branch=None, namespace=None, push=False,
+        debug=False, prefill=True, registry=None):
+    dst_folder = None
+    summary = Summary()
+    if repository is None:
+        repository = DEFAULT_REPOSITORY
+    if branch is None:
+        branch = DEFAULT_BRANCH
+    if debug:
+        logger.setLevel('DEBUG')
+
+    if not (repository.startswith('https://') or repository.startswith('git://')):
+        logger.info('Repository provided assumed to be a local path')
+        dst_folder = repository
+
+    try:
+        client.version()
+    except Exception as e:
+        logger.error('Could not reach the docker daemon. Please make sure it '
+            'is running.')
+        logger.warning('Also make sure you have access to the docker UNIX '
+            'socket (use sudo)')
+        return
+
+    #FIXME: set destination folder and only pull latest changes instead of
+    # cloning the whole repo everytime
+    if not dst_folder:
+        logger.info('Cloning docker repo from {0}, branch: {1}'.format(
+            repository, branch))
+        try:
+            rep, dst_folder = git.clone_branch(repository, branch)
+        except Exception as e:
+            logger.exception(e)
+            logger.error('Source repository could not be fetched. Check '
+                'that the address is correct and the branch exists.')
+            return
+    try:
+        dirlist = os.listdir(os.path.join(dst_folder, 'library'))
+    except OSError as e:
+        logger.error('The path provided ({0}) could not be found or didn\'t'
+            'contain a library/ folder.'.format(dst_folder))
+        return
+    for buildfile in dirlist:
+        if buildfile == 'MAINTAINERS':
+            continue
+        f = open(os.path.join(dst_folder, 'library', buildfile))
+        linecnt = 0
+        for line in f:
+            linecnt = linecnt + 1
+            logger.debug('{0} ---> {1}'.format(buildfile, line))
+            args = line.split()
+            try:
+                if len(args) > 3:
+                    raise RuntimeError('Incorrect line format, '
+                        'please refer to the docs')
+
+                url = None
+                ref = 'refs/heads/master'
+                tag = None
+                if len(args) == 1:  # Just a URL, simple mode
+                    url = args[0]
+                elif len(args) == 2 or len(args) == 3:  # docker-tag   url
+                    url = args[1]
+                    tag = args[0]
+
+                if len(args) == 3:  # docker-tag  url     B:branch or T:tag
+                    ref = None
+                    if args[2].startswith('B:'):
+                        ref = 'refs/heads/' + args[2][2:]
+                    elif args[2].startswith('T:'):
+                        ref = 'refs/tags/' + args[2][2:]
+                    elif args[2].startswith('C:'):
+                        ref = args[2][2:]
+                    else:
+                        raise RuntimeError('Incorrect line format, '
+                            'please refer to the docs')
+                if prefill:
+                    logger.debug('Pulling {0} from official repository (cache '
+                        'fill)'.format(buildfile))
+                    client.pull(buildfile)
+                img = build_repo(url, ref, buildfile, tag, namespace, push,
+                    registry)
+                summary.add_success(buildfile, (linecnt, line), img)
+                processed['{0}@{1}'.format(url, ref)] = img
+            except Exception as e:
+                logger.exception(e)
+                summary.add_exception(buildfile, (linecnt, line), e)
+
+        f.close()
+    if dst_folder != repository:
+        rmtree(dst_folder, True)
+    for d in processed_folders:
+        rmtree(d, True)
+    summary.print_summary(logger)
+
+
+def build_repo(repository, ref, docker_repo, docker_tag, namespace, push, registry):
+    docker_repo = '{0}/{1}'.format(namespace or 'library', docker_repo)
+    img_id = None
+    dst_folder = None
+    if '{0}@{1}'.format(repository, ref) not in processed.keys():
+        logger.info('Cloning {0} (ref: {1})'.format(repository, ref))
+        if repository not in processed:
+            rep, dst_folder = git.clone(repository, ref)
+            processed[repository] = rep
+            processed_folders.append(dst_folder)
+        else:
+            dst_folder = git.checkout(processed[repository], ref)
+        if not 'Dockerfile' in os.listdir(dst_folder):
+            raise RuntimeError('Dockerfile not found in cloned repository')
+        logger.info('Building using dockerfile...')
+        img_id, logs = client.build(path=dst_folder, quiet=True)
+    else:
+        img_id = processed['{0}@{1}'.format(repository, ref)]
+    logger.info('Committing to {0}:{1}'.format(docker_repo,
+        docker_tag or 'latest'))
+    client.tag(img_id, docker_repo, docker_tag)
+    if push:
+        logger.info('Pushing result to registry {0}'.format(
+            registry or "default"))
+        if registry is not None:
+            docker_repo = '{0}/{1}'.format(registry, docker_repo)
+            logger.info('Also tagging {0}'.format(docker_repo))
+            client.tag(img_id, docker_repo, docker_tag)
+        client.push(docker_repo)
+    return img_id
+
+
+class Summary(object):
+    def __init__(self):
+        self._summary = {}
+        self._has_exc = False
+
+    def _add_data(self, image, linestr, data):
+        if image not in self._summary:
+            self._summary[image] = { linestr: data }
+        else:
+            self._summary[image][linestr] = data
+
+    def add_exception(self, image, line, exc):
+        lineno, linestr = line
+        self._add_data(image, linestr, { 'line': lineno, 'exc': str(exc) })
+        self._has_exc = True
+
+    def add_success(self, image, line, img_id):
+        lineno, linestr = line
+        self._add_data(image, linestr, { 'line': lineno, 'id': img_id })
+
+    def print_summary(self, logger=None):
+        linesep = ''.center(61, '-') + '\n'
+        s = 'BREW BUILD SUMMARY\n' + linesep
+        success = 'OVERALL SUCCESS: {}\n'.format(not self._has_exc)
+        details = linesep
+        for image, lines in self._summary.iteritems():
+            details = details + '{}\n{}'.format(image, linesep)
+            for linestr, data in lines.iteritems():
+                details = details + '{0:2} | {1} | {2:50}\n'.format(
+                    data['line'],
+                    'KO' if 'exc' in data else 'OK',
+                    data['exc'] if 'exc' in data else data['id']
+                )
+            details = details + linesep
+        if logger:
+            logger.info(s + success + details)
+        else:
+            print s, success, details

contrib/brew/brew/git.py

@@ -0,0 +1,63 @@
+import tempfile
+import logging
+
+from dulwich import index
+from dulwich.client import get_transport_and_path
+from dulwich.repo import Repo
+
+logger = logging.getLogger(__name__)
+
+
+def clone_branch(repo_url, branch="master", folder=None):
+    return clone(repo_url, 'refs/heads/' + branch, folder)
+
+
+def clone_tag(repo_url, tag, folder=None):
+    return clone(repo_url, 'refs/tags/' + tag, folder)
+
+
+def checkout(rep, ref=None):
+    is_commit = False
+    if ref is None:
+        ref = 'refs/heads/master'
+    elif not ref.startswith('refs/'):
+        is_commit = True
+    if is_commit:
+        rep['HEAD'] = rep.commit(ref)
+    else:
+        rep['HEAD'] = rep.refs[ref]
+    indexfile = rep.index_path()
+    tree = rep["HEAD"].tree
+    index.build_index_from_tree(rep.path, indexfile, rep.object_store, tree)
+    return rep.path
+
+def clone(repo_url, ref=None, folder=None):
+    is_commit = False
+    if ref is None:
+        ref = 'refs/heads/master'
+    elif not ref.startswith('refs/'):
+        is_commit = True
+    logger.debug("clone repo_url={0}, ref={1}".format(repo_url, ref))
+    if folder is None:
+        folder = tempfile.mkdtemp()
+    logger.debug("folder = {0}".format(folder))
+    rep = Repo.init(folder)
+    client, relative_path = get_transport_and_path(repo_url)
+    logger.debug("client={0}".format(client))
+
+    remote_refs = client.fetch(relative_path, rep)
+    for k, v in remote_refs.iteritems():
+        try:
+            rep.refs.add_if_new(k, v)
+        except:
+            pass
+
+    if is_commit:
+        rep['HEAD'] = rep.commit(ref)
+    else:
+        rep['HEAD'] = remote_refs[ref]
+    indexfile = rep.index_path()
+    tree = rep["HEAD"].tree
+    index.build_index_from_tree(rep.path, indexfile, rep.object_store, tree)
+    logger.debug("done")
+    return rep, folder

contrib/brew/docker-brew

@@ -0,0 +1,35 @@
+#!/usr/bin/env python
+
+import argparse
+import sys
+
+try:
+    import brew
+except ImportError as e:
+    print str(e)
+    print 'Please install the required dependencies first'
+    print 'sudo pip install -r requirements.txt'
+    sys.exit(1)
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser('Build the docker standard library')
+    parser.add_argument('--push', action='store_true', default=False,
+        help='Push generated repositories')
+    parser.add_argument('--debug', default=False, action='store_true',
+        help='Enable debugging output')
+    parser.add_argument('--noprefill', default=True, action='store_false',
+        dest='prefill', help='Disable cache prefill')
+    parser.add_argument('-n', metavar='NAMESPACE', default='library',
+        help='Namespace used for generated repositories.'
+        ' Default is library')
+    parser.add_argument('-b', metavar='BRANCH', default=brew.DEFAULT_BRANCH,
+        help='Branch in the repository where the library definition'
+        ' files will be fetched. Default is ' + brew.DEFAULT_BRANCH)
+    parser.add_argument('repository', default=brew.DEFAULT_REPOSITORY,
+        nargs='?', help='git repository containing the library definition'
+        ' files. Default is ' + brew.DEFAULT_REPOSITORY)
+    parser.add_argument('--reg', default=None, help='Registry address to'
+        ' push build results to. Also sets push to true.')
+    args = parser.parse_args()
+    brew.build_library(args.repository, args.b, args.n,
+        args.push or args.reg is not None, args.debug, args.prefill, args.reg)

contrib/brew/requirements.txt

@@ -0,0 +1,2 @@
+dulwich==0.9.0
+-e git://github.com/dotcloud/docker-py.git#egg=docker-py

contrib/brew/setup.py

@@ -0,0 +1,22 @@
+#!/usr/bin/env python
+import os
+from setuptools import setup
+
+ROOT_DIR = os.path.dirname(__file__)
+SOURCE_DIR = os.path.join(ROOT_DIR)
+
+test_requirements = []
+setup(
+    name="docker-brew",
+    version='0.0.1',
+    description="-",
+    packages=['dockerbrew'],
+    install_requires=['dulwich', 'docker'] + test_requirements,
+    zip_safe=False,
+    classifiers=['Development Status :: 3 - Alpha',
+                 'Environment :: Other Environment',
+                 'Intended Audience :: Developers',
+                 'Operating System :: OS Independent',
+                 'Programming Language :: Python',
+                 'Topic :: Utilities'],
+    )

contrib/completion/bash/docker

@@ -426,7 +426,7 @@ _docker_run()
 
 _docker_search()
 {
-	COMPREPLY=( $( compgen -W "-notrunc" "-stars" "-trusted" -- "$cur" ) )
+	COMPREPLY=( $( compgen -W "-notrunc" -- "$cur" ) )
 }
 
 _docker_start()

contrib/desktop-integration/README.txt

@@ -1,11 +0,0 @@
-Desktop Integration
-===================
-
-The ./contrib/desktop-integration contains examples of typical dockerized
-desktop applications.
-
-Examples
-========
-
-* Data container: ./data/Dockerfile creates a data image sharing /data volume
-* Firefox: ./firefox/Dockerfile shows a way to dockerize a common multimedia application

contrib/desktop-integration/data/Dockerfile

@@ -1,38 +0,0 @@
-# VERSION:        0.1
-# DESCRIPTION:    Create data image sharing /data volume
-# AUTHOR:         Daniel Mizyrycki <daniel@dotcloud.com>
-# COMMENTS:
-#   This image is used as base for all data containers.
-#   /data volume is owned by sysadmin.
-# USAGE:
-#   # Download data Dockerfile
-#   wget http://raw.github.com/dotcloud/docker/master/contrib/desktop-integration/data/Dockerfile
-#
-#   # Build data image
-#   docker build -t data -rm .
-#
-#   # Create a data container. (eg: firefox-data)
-#   docker run -name firefox-data data true
-#
-#   # List data from it
-#   docker run -volumes-from firefox-data busybox ls -al /data
-
-docker-version 0.6.5
-
-# Smallest base image, just to launch a container
-from busybox
-maintainer	Daniel Mizyrycki <daniel@docker.com>
-
-# Create a regular user
-run echo 'sysadmin:x:1000:1000::/data:/bin/sh' >> /etc/passwd
-run echo 'sysadmin:x:1000:' >> /etc/group
-
-# Create directory for that user
-run mkdir /data
-run chown sysadmin.sysadmin /data
-
-# Add content to /data. This will keep sysadmin ownership
-run touch /data/init_volume
-
-# Create /data volume
-VOLUME /data

contrib/desktop-integration/firefox/Dockerfile

@@ -1,49 +0,0 @@
-# VERSION:        0.7
-# DESCRIPTION:    Create firefox container with its dependencies
-# AUTHOR:         Daniel Mizyrycki <daniel@dotcloud.com>
-# COMMENTS:
-#   This file describes how to build a Firefox container with all
-#   dependencies installed. It uses native X11 unix socket and alsa
-#   sound devices. Tested on Debian 7.2
-# USAGE:
-#   # Download Firefox Dockerfile
-#   wget http://raw.github.com/dotcloud/docker/master/contrib/desktop-integration/firefox/Dockerfile
-#
-#   # Build firefox image
-#   docker build -t firefox -rm .
-#
-#   # Run stateful data-on-host firefox. For ephemeral, remove -v /data/firefox:/data
-#   docker run -v /data/firefox:/data -v /tmp/.X11-unix:/tmp/.X11-unix \
-#     -v /dev/snd:/dev/snd -lxc-conf='lxc.cgroup.devices.allow = c 116:* rwm' \
-#     -e DISPLAY=unix$DISPLAY firefox
-#
-#   # To run stateful dockerized data containers
-#   docker run -volumes-from firefox-data -v /tmp/.X11-unix:/tmp/.X11-unix \
-#     -v /dev/snd:/dev/snd -lxc-conf='lxc.cgroup.devices.allow = c 116:* rwm' \
-#     -e DISPLAY=unix$DISPLAY firefox
-
-docker-version 0.6.5
-
-# Base docker image
-from tianon/debian:wheezy
-maintainer	Daniel Mizyrycki <daniel@docker.com>
-
-# Install firefox dependencies
-run echo "deb http://ftp.debian.org/debian/ wheezy main contrib" > /etc/apt/sources.list
-run apt-get update
-run DEBIAN_FRONTEND=noninteractive apt-get install -y libXrender1 libasound2 \
-    libdbus-glib-1-2 libgtk2.0-0 libpango1.0-0 libxt6 wget bzip2 sudo
-
-# Install Firefox
-run mkdir /application
-run cd /application; wget -O - \
-    http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/25.0/linux-x86_64/en-US/firefox-25.0.tar.bz2 | tar jx
-
-# create sysadmin account
-run useradd -m -d /data -p saIVpsc0EVTwA sysadmin
-run sed -Ei 's/sudo:x:27:/sudo:x:27:sysadmin/' /etc/group
-run sed -Ei 's/(\%sudo\s+ALL=\(ALL\:ALL\) )ALL/\1 NOPASSWD:ALL/' /etc/sudoers
-
-# Autorun firefox. -no-remote is necessary to create a new container, as firefox
-# appears to communicate with itself through X11.
-cmd ["/bin/sh", "-c", "/usr/bin/sudo -u sysadmin -H -E /application/firefox/firefox -no-remote"]

contrib/docker-device-tool/device_tool.go

@@ -1,170 +0,0 @@
-package main
-
-import (
-	"flag"
-	"fmt"
-	"github.com/dotcloud/docker/graphdriver/devmapper"
-	"os"
-	"path"
-	"sort"
-	"strconv"
-	"strings"
-)
-
-func usage() {
-	fmt.Fprintf(os.Stderr, "Usage: %s <flags>  [status] | [list] | [device id]  | [resize new-pool-size] | [snap new-id base-id] | [remove id] | [mount id mountpoint]\n", os.Args[0])
-	flag.PrintDefaults()
-	os.Exit(1)
-}
-
-func byteSizeFromString(arg string) (int64, error) {
-	digits := ""
-	rest := ""
-	last := strings.LastIndexAny(arg, "0123456789")
-	if last >= 0 {
-		digits = arg[:last+1]
-		rest = arg[last+1:]
-	}
-
-	val, err := strconv.ParseInt(digits, 10, 64)
-	if err != nil {
-		return val, err
-	}
-
-	rest = strings.ToLower(strings.TrimSpace(rest))
-
-	var multiplier int64 = 1
-	switch rest {
-	case "":
-		multiplier = 1
-	case "k", "kb":
-		multiplier = 1024
-	case "m", "mb":
-		multiplier = 1024 * 1024
-	case "g", "gb":
-		multiplier = 1024 * 1024 * 1024
-	case "t", "tb":
-		multiplier = 1024 * 1024 * 1024 * 1024
-	default:
-		return 0, fmt.Errorf("Unknown size unit: %s", rest)
-	}
-
-	return val * multiplier, nil
-}
-
-func main() {
-	root := flag.String("r", "/var/lib/docker", "Docker root dir")
-	flDebug := flag.Bool("D", false, "Debug mode")
-
-	flag.Parse()
-
-	if *flDebug {
-		os.Setenv("DEBUG", "1")
-	}
-
-	if flag.NArg() < 1 {
-		usage()
-	}
-
-	args := flag.Args()
-
-	home := path.Join(*root, "devicemapper")
-	devices, err := devmapper.NewDeviceSet(home, false)
-	if err != nil {
-		fmt.Println("Can't initialize device mapper: ", err)
-		os.Exit(1)
-	}
-
-	switch args[0] {
-	case "status":
-		status := devices.Status()
-		fmt.Printf("Pool name: %s\n", status.PoolName)
-		fmt.Printf("Data Loopback file: %s\n", status.DataLoopback)
-		fmt.Printf("Metadata Loopback file: %s\n", status.MetadataLoopback)
-		fmt.Printf("Sector size: %d\n", status.SectorSize)
-		fmt.Printf("Data use: %d of %d (%.1f %%)\n", status.Data.Used, status.Data.Total, 100.0*float64(status.Data.Used)/float64(status.Data.Total))
-		fmt.Printf("Metadata use: %d of %d (%.1f %%)\n", status.Metadata.Used, status.Metadata.Total, 100.0*float64(status.Metadata.Used)/float64(status.Metadata.Total))
-		break
-	case "list":
-		ids := devices.List()
-		sort.Strings(ids)
-		for _, id := range ids {
-			fmt.Println(id)
-		}
-		break
-	case "device":
-		if flag.NArg() < 2 {
-			usage()
-		}
-		status, err := devices.GetDeviceStatus(args[1])
-		if err != nil {
-			fmt.Println("Can't get device info: ", err)
-			os.Exit(1)
-		}
-		fmt.Printf("Id: %d\n", status.DeviceId)
-		fmt.Printf("Size: %d\n", status.Size)
-		fmt.Printf("Transaction Id: %d\n", status.TransactionId)
-		fmt.Printf("Size in Sectors: %d\n", status.SizeInSectors)
-		fmt.Printf("Mapped Sectors: %d\n", status.MappedSectors)
-		fmt.Printf("Highest Mapped Sector: %d\n", status.HighestMappedSector)
-		break
-	case "resize":
-		if flag.NArg() < 2 {
-			usage()
-		}
-
-		size, err := byteSizeFromString(args[1])
-		if err != nil {
-			fmt.Println("Invalid size: ", err)
-			os.Exit(1)
-		}
-
-		err = devices.ResizePool(size)
-		if err != nil {
-			fmt.Println("Error resizeing pool: ", err)
-			os.Exit(1)
-		}
-
-		break
-	case "snap":
-		if flag.NArg() < 3 {
-			usage()
-		}
-
-		err := devices.AddDevice(args[1], args[2])
-		if err != nil {
-			fmt.Println("Can't create snap device: ", err)
-			os.Exit(1)
-		}
-		break
-	case "remove":
-		if flag.NArg() < 2 {
-			usage()
-		}
-
-		err := devices.RemoveDevice(args[1])
-		if err != nil {
-			fmt.Println("Can't remove device: ", err)
-			os.Exit(1)
-		}
-		break
-	case "mount":
-		if flag.NArg() < 3 {
-			usage()
-		}
-
-		err := devices.MountDevice(args[1], args[2], false)
-		if err != nil {
-			fmt.Println("Can't create snap device: ", err)
-			os.Exit(1)
-		}
-		break
-	default:
-		fmt.Printf("Unknown command %s\n", args[0])
-		usage()
-
-		os.Exit(1)
-	}
-
-	return
-}

contrib/host-integration/Dockerfile.dev

@@ -1,27 +0,0 @@
-#
-# This Dockerfile will create an image that allows to generate upstart and
-# systemd scripts (more to come)
-#
-# docker-version 0.6.2
-#
-
-FROM		ubuntu:12.10
-MAINTAINER	Guillaume J. Charmes <guillaume@dotcloud.com>
-
-RUN		apt-get update && apt-get install -y wget git mercurial
-
-# Install Go
-RUN		wget --no-check-certificate https://go.googlecode.com/files/go1.1.2.linux-amd64.tar.gz -O go-1.1.2.tar.gz
-RUN		tar -xzvf go-1.1.2.tar.gz && mv /go /goroot
-RUN		mkdir /go
-
-ENV		GOROOT	  /goroot
-ENV		GOPATH	  /go
-ENV		PATH	  $GOROOT/bin:$PATH
-
-RUN		go get github.com/dotcloud/docker && cd /go/src/github.com/dotcloud/docker && git checkout v0.6.3
-ADD		manager.go	/manager/
-RUN		cd /manager && go build -o /usr/bin/manager
-
-ENTRYPOINT	["/usr/bin/manager"]
-

contrib/host-integration/Dockerfile.min

@@ -1,4 +0,0 @@
-FROM		busybox
-MAINTAINER	Guillaume J. Charmes <guillaume@dotcloud.com>
-ADD		manager	  /usr/bin/
-ENTRYPOINT	["/usr/bin/manager"]

contrib/host-integration/manager.go

@@ -1,130 +0,0 @@
-package main
-
-import (
-	"bytes"
-	"encoding/json"
-	"flag"
-	"fmt"
-	"github.com/dotcloud/docker"
-	"os"
-	"strings"
-	"text/template"
-)
-
-var templates = map[string]string{
-
-	"upstart": `description "{{.description}}"
-author "{{.author}}"
-start on filesystem and started lxc-net and started docker
-stop on runlevel [!2345]
-respawn
-exec /home/vagrant/goroot/bin/docker start -a {{.container_id}}
-`,
-
-	"systemd": `[Unit]
-	Description={{.description}}
-	Author={{.author}}
-	After=docker.service
-
-[Service]
-	Restart=always
-	ExecStart=/usr/bin/docker start -a {{.container_id}}
-	ExecStop=/usr/bin/docker stop -t 2 {{.container_id}}
-
-[Install]
-	WantedBy=local.target
-`,
-}
-
-func main() {
-	// Parse command line for custom options
-	kind := flag.String("t", "upstart", "Type of manager requested")
-	author := flag.String("a", "<none>", "Author of the image")
-	description := flag.String("d", "<none>", "Description of the image")
-	flag.Usage = func() {
-		fmt.Fprintf(os.Stderr, "\nUsage: manager <container id>\n\n")
-		flag.PrintDefaults()
-	}
-	flag.Parse()
-
-	// We require at least the container ID
-	if flag.NArg() != 1 {
-		println(flag.NArg())
-		flag.Usage()
-		return
-	}
-
-	// Check that the requested process manager is supported
-	if _, exists := templates[*kind]; !exists {
-		panic("Unkown script template")
-	}
-
-	// Load the requested template
-	tpl, err := template.New("processManager").Parse(templates[*kind])
-	if err != nil {
-		panic(err)
-	}
-
-	// Create stdout/stderr buffers
-	bufOut := bytes.NewBuffer(nil)
-	bufErr := bytes.NewBuffer(nil)
-
-	// Instanciate the Docker CLI
-	cli := docker.NewDockerCli(nil, bufOut, bufErr, "unix", "/var/run/docker.sock")
-	// Retrieve the container info
-	if err := cli.CmdInspect(flag.Arg(0)); err != nil {
-		// As of docker v0.6.3, CmdInspect always returns nil
-		panic(err)
-	}
-
-	// If there is nothing in the error buffer, then the Docker daemon is there and the container has been found
-	if bufErr.Len() == 0 {
-		// Unmarshall the resulting container data
-		c := []*docker.Container{{}}
-		if err := json.Unmarshal(bufOut.Bytes(), &c); err != nil {
-			panic(err)
-		}
-		// Reset the buffers
-		bufOut.Reset()
-		bufErr.Reset()
-		// Retrieve the info of the linked image
-		if err := cli.CmdInspect(c[0].Image); err != nil {
-			panic(err)
-		}
-		// If there is nothing in the error buffer, then the image has been found.
-		if bufErr.Len() == 0 {
-			// Unmarshall the resulting image data
-			img := []*docker.Image{{}}
-			if err := json.Unmarshal(bufOut.Bytes(), &img); err != nil {
-				panic(err)
-			}
-			// If no author has been set, use the one from the image
-			if *author == "<none>" && img[0].Author != "" {
-				*author = strings.Replace(img[0].Author, "\"", "", -1)
-			}
-			// If no description has been set, use the comment from the image
-			if *description == "<none>" && img[0].Comment != "" {
-				*description = strings.Replace(img[0].Comment, "\"", "", -1)
-			}
-		}
-	}
-
-	/// Old version: Wrtie the resulting script to file
-	// f, err := os.OpenFile(kind, os.O_CREATE|os.O_WRONLY, 0755)
-	// if err != nil {
-	// 	panic(err)
-	// }
-	// defer f.Close()
-
-	// Create a map with needed data
-	data := map[string]string{
-		"author":       *author,
-		"description":  *description,
-		"container_id": flag.Arg(0),
-	}
-
-	// Process the template and output it on Stdout
-	if err := tpl.Execute(os.Stdout, data); err != nil {
-		panic(err)
-	}
-}

contrib/host-integration/manager.sh

@@ -1,53 +0,0 @@
-#!/bin/sh
-set -e
-
-usage() {
-	echo >&2 "usage: $0 [-a author] [-d description] container [manager]"
-	echo >&2 "   ie: $0 -a 'John Smith' 4ec9612a37cd systemd"
-	echo >&2 "   ie: $0 -d 'Super Cool System' 4ec9612a37cd # defaults to upstart"
-	exit 1
-}
-
-auth='<none>'
-desc='<none>'
-have_auth=
-have_desc=
-while getopts a:d: opt; do
-	case "$opt" in
-		a)
-			auth="$OPTARG"
-			have_auth=1
-			;;
-		d)
-			desc="$OPTARG"
-			have_desc=1
-			;;
-	esac
-done
-shift $(($OPTIND - 1))
-
-[ $# -ge 1 -a $# -le 2 ] || usage
-
-cid="$1"
-script="${2:-upstart}"
-if [ ! -e "manager/$script" ]; then
-	echo >&2 "Error: manager type '$script' is unknown (PRs always welcome!)."
-	echo >&2 'The currently supported types are:'
-	echo >&2 "  $(cd manager && echo *)"
-	exit 1
-fi
-
-# TODO https://github.com/dotcloud/docker/issues/734 (docker inspect formatting)
-#if command -v docker > /dev/null 2>&1; then
-#	image="$(docker inspect -f '{{.Image}}' "$cid")"
-#	if [ "$image" ]; then
-#		if [ -z "$have_auth" ]; then
-#			auth="$(docker inspect -f '{{.Author}}' "$image")"
-#		fi
-#		if [ -z "$have_desc" ]; then
-#			desc="$(docker inspect -f '{{.Comment}}' "$image")"
-#		fi
-#	fi
-#fi
-
-exec "manager/$script" "$cid" "$auth" "$desc"

contrib/host-integration/manager/systemd

@@ -1,20 +0,0 @@
-#!/bin/sh
-set -e
-
-cid="$1"
-auth="$2"
-desc="$3"
-
-cat <<-EOF
-	[Unit]
-	Description=$desc
-	Author=$auth
-	After=docker.service
-	
-	[Service]
-	ExecStart=/usr/bin/docker start -a $cid
-	ExecStop=/usr/bin/docker stop -t 2 $cid
-	
-	[Install]
-	WantedBy=local.target
-EOF

contrib/host-integration/manager/upstart

@@ -1,15 +0,0 @@
-#!/bin/sh
-set -e
-
-cid="$1"
-auth="$2"
-desc="$3"
-
-cat <<-EOF
-	description "$(echo "$desc" | sed 's/"/\\"/g')"
-	author "$(echo "$auth" | sed 's/"/\\"/g')"
-	start on filesystem and started lxc-net and started docker
-	stop on runlevel [!2345]
-	respawn
-	exec /usr/bin/docker start -a "$cid"
-EOF

contrib/init/openrc/docker.confd

@@ -1,13 +0,0 @@
-# /etc/conf.d/docker: config file for /etc/init.d/docker
-
-# where the docker daemon output gets piped
-#DOCKER_LOGFILE="/var/log/docker.log"
-
-# where docker's pid get stored
-#DOCKER_PIDFILE="/run/docker.pid"
-
-# where the docker daemon itself is run from
-#DOCKER_BINARY="/usr/bin/docker"
-
-# any other random options you want to pass to docker
-DOCKER_OPTS=""

contrib/init/openrc/docker.initd

@@ -1,31 +0,0 @@
-#!/sbin/runscript
-# Copyright 1999-2013 Gentoo Foundation
-# Distributed under the terms of the GNU General Public License v2
-# $Header: $
-
-DOCKER_LOGFILE=${DOCKER_LOGFILE:-/var/log/${SVCNAME}.log}
-DOCKER_PIDFILE=${DOCKER_PIDFILE:-/run/${SVCNAME}.pid}
-DOCKER_BINARY=${DOCKER_BINARY:-/usr/bin/docker}
-DOCKER_OPTS=${DOCKER_OPTS:-}
-
-start() {
-	checkpath -f -m 0644 -o root:docker "$DOCKER_LOGFILE"
-
-	ebegin "Starting docker daemon"
-	start-stop-daemon --start --background \
-		--exec "$DOCKER_BINARY" \
-		--pidfile "$DOCKER_PIDFILE" \
-		--stdout "$DOCKER_LOGFILE" \
-		--stderr "$DOCKER_LOGFILE" \
-		-- -d -p "$DOCKER_PIDFILE" \
-		$DOCKER_OPTS
-	eend $?
-}
-
-stop() {
-	ebegin "Stopping docker daemon"
-	start-stop-daemon --stop \
-		--exec "$DOCKER_BINARY" \
-		--pidfile "$DOCKER_PIDFILE"
-	eend $?
-}

contrib/init/systemd/docker.service

@@ -1,11 +0,0 @@
-[Unit]
-Description=Docker Application Container Engine 
-Documentation=http://docs.docker.io
-After=network.target
-
-[Service]
-ExecStartPre=/bin/mount --make-rprivate /
-ExecStart=/usr/bin/docker -d
-
-[Install]
-WantedBy=multi-user.target

contrib/init/sysvinit/docker

@@ -1,96 +0,0 @@
-#!/bin/sh
-
-### BEGIN INIT INFO
-# Provides:           docker
-# Required-Start:     $syslog $remote_fs
-# Required-Stop:      $syslog $remote_fs
-# Default-Start:      2 3 4 5
-# Default-Stop:       0 1 6
-# Short-Description:  Create lightweight, portable, self-sufficient containers.
-# Description:
-#  Docker is an open-source project to easily create lightweight, portable,
-#  self-sufficient containers from any application. The same container that a
-#  developer builds and tests on a laptop can run at scale, in production, on
-#  VMs, bare metal, OpenStack clusters, public clouds and more.
-### END INIT INFO
-
-BASE=$(basename $0)
-
-DOCKER=/usr/bin/$BASE
-DOCKER_PIDFILE=/var/run/$BASE.pid
-DOCKER_OPTS=
-
-PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
-
-# Get lsb functions
-. /lib/lsb/init-functions
-
-if [ -f /etc/default/$BASE ]; then
-	. /etc/default/$BASE
-fi
-
-# see also init_is_upstart in /lib/lsb/init-functions (which isn't available in Ubuntu 12.04, or we'd use it)
-if [ -x /sbin/initctl ] && /sbin/initctl version 2>/dev/null | /bin/grep -q upstart; then
-	log_failure_msg "Docker is managed via upstart, try using service $BASE $1"
-	exit 1
-fi
-
-# Check docker is present
-if [ ! -x $DOCKER ]; then
-	log_failure_msg "$DOCKER not present or not executable"
-	exit 1
-fi
-
-fail_unless_root() {
-	if [ "$(id -u)" != '0' ]; then
-		log_failure_msg "Docker must be run as root"
-		exit 1
-	fi
-}
-
-case "$1" in
-	start)
-		fail_unless_root
-		log_begin_msg "Starting Docker: $BASE"
-		mount | grep cgroup >/dev/null || mount -t cgroup none /sys/fs/cgroup 2>/dev/null
-		start-stop-daemon --start --background \
-			--exec "$DOCKER" \
-			--pidfile "$DOCKER_PIDFILE" \
-			-- -d -p "$DOCKER_PIDFILE" \
-			$DOCKER_OPTS
-		log_end_msg $?
-		;;
-
-	stop)
-		fail_unless_root
-		log_begin_msg "Stopping Docker: $BASE"
-		start-stop-daemon --stop \
-			--pidfile "$DOCKER_PIDFILE"
-		log_end_msg $?
-		;;
-
-	restart)
-		fail_unless_root
-		docker_pid=`cat "$DOCKER_PIDFILE" 2>/dev/null`
-		[ -n "$docker_pid" ] \
-			&& ps -p $docker_pid > /dev/null 2>&1 \
-			&& $0 stop
-		$0 start
-		;;
-
-	force-reload)
-		fail_unless_root
-		$0 restart
-		;;
-
-	status)
-		status_of_proc -p "$DOCKER_PIDFILE" "$DOCKER" docker
-		;;
-
-	*)
-		echo "Usage: $0 {start|stop|restart|status}"
-		exit 1
-		;;
-esac
-
-exit 0

contrib/init/upstart/docker.conf

@@ -1,15 +0,0 @@
-description "Docker daemon"
-
-start on filesystem and started lxc-net
-stop on runlevel [!2345]
-
-respawn
-
-script
-	DOCKER=/usr/bin/$UPSTART_JOB
-	DOCKER_OPTS=
-	if [ -f /etc/default/$UPSTART_JOB ]; then
-		. /etc/default/$UPSTART_JOB
-	fi
-	"$DOCKER" -d $DOCKER_OPTS
-end script

contrib/mkimage-arch.sh

@@ -62,6 +62,6 @@ mknod -m 666 ${DEV}/full c 1 7
 mknod -m 600 ${DEV}/initctl p
 mknod -m 666 ${DEV}/ptmx c 5 2
 
-tar --numeric-owner -C $ROOTFS -c . | docker import - archlinux
+tar -C $ROOTFS -c . | docker import - archlinux
 docker run -i -t archlinux echo Success.
 rm -rf $ROOTFS

contrib/mkimage-busybox.sh

@@ -35,5 +35,5 @@ do
     cp -a /dev/$X dev
 done
 
-tar --numeric-owner -cf- . | docker import - busybox
+tar -cf- . | docker import - busybox
 docker run -i -u root busybox /bin/echo Success.

contrib/mkimage-centos.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+# Create a CentOS base image for Docker
+# From unclejack https://github.com/dotcloud/docker/issues/290
+set -e
+
+MIRROR_URL="http://centos.netnitco.net/6.4/os/x86_64/"
+MIRROR_URL_UPDATES="http://centos.netnitco.net/6.4/updates/x86_64/"
+
+yum install -y febootstrap xz
+
+febootstrap -i bash -i coreutils -i tar -i bzip2 -i gzip -i vim-minimal -i wget -i patch -i diffutils -i iproute -i yum centos centos64  $MIRROR_URL -u $MIRROR_URL_UPDATES
+touch centos64/etc/resolv.conf
+touch centos64/sbin/init
+
+tar --numeric-owner -Jcpf centos-64.tar.xz -C centos64 .

contrib/mkimage-debian.sh

@@ -0,0 +1,83 @@
+#!/bin/bash
+set -e
+
+# these should match the names found at http://www.debian.org/releases/
+stableSuite='wheezy'
+testingSuite='jessie'
+unstableSuite='sid'
+
+variant='minbase'
+include='iproute,iputils-ping'
+
+repo="$1"
+suite="$2"
+mirror="${3:-}" # stick to the default debootstrap mirror if one is not provided
+
+if [ ! "$repo" ] || [ ! "$suite" ]; then
+	echo >&2 "usage: $0 repo suite [mirror]"
+	echo >&2
+	echo >&2 "   ie: $0 tianon/debian squeeze"
+	echo >&2 "       $0 tianon/debian squeeze http://ftp.uk.debian.org/debian/"
+	echo >&2
+	echo >&2 "   ie: $0 tianon/ubuntu precise"
+	echo >&2 "       $0 tianon/ubuntu precise http://mirrors.melbourne.co.uk/ubuntu/"
+	echo >&2
+	exit 1
+fi
+
+target="/tmp/docker-rootfs-debian-$suite-$$-$RANDOM"
+
+cd "$(dirname "$(readlink -f "$BASH_SOURCE")")"
+returnTo="$(pwd -P)"
+
+set -x
+
+# bootstrap
+mkdir -p "$target"
+sudo debootstrap --verbose --variant="$variant" --include="$include" "$suite" "$target" "$mirror"
+
+cd "$target"
+
+# prevent init scripts from running during install/update
+#  policy-rc.d (for most scripts)
+echo $'#!/bin/sh\nexit 101' | sudo tee usr/sbin/policy-rc.d > /dev/null
+sudo chmod +x usr/sbin/policy-rc.d
+#  initctl (for some pesky upstart scripts)
+sudo chroot . dpkg-divert --local --rename --add /sbin/initctl
+sudo ln -sf /bin/true sbin/initctl
+# see https://github.com/dotcloud/docker/issues/446#issuecomment-16953173
+
+# shrink the image, since apt makes us fat (wheezy: ~157.5MB vs ~120MB)
+sudo chroot . apt-get clean
+
+# while we're at it, apt is unnecessarily slow inside containers
+#  this forces dpkg not to call sync() after package extraction and speeds up install
+#    the benefit is huge on spinning disks, and the penalty is nonexistent on SSD or decent server virtualization
+echo 'force-unsafe-io' | sudo tee etc/dpkg/dpkg.cfg.d/02apt-speedup > /dev/null
+#  we want to effectively run "apt-get clean" after every install to keep images small
+echo 'DPkg::Post-Invoke {"/bin/rm -f /var/cache/apt/archives/*.deb || true";};' | sudo tee etc/apt/apt.conf.d/no-cache > /dev/null
+
+# helpful undo lines for each the above tweaks (for lack of a better home to keep track of them):
+#  rm /usr/sbin/policy-rc.d
+#  rm /sbin/initctl; dpkg-divert --rename --remove /sbin/initctl
+#  rm /etc/dpkg/dpkg.cfg.d/02apt-speedup
+#  rm /etc/apt/apt.conf.d/no-cache
+
+# create the image (and tag $repo:$suite)
+sudo tar -c . | docker import - $repo $suite
+
+# test the image
+docker run -i -t $repo:$suite echo success
+
+if [ "$suite" = "$stableSuite" -o "$suite" = 'stable' ]; then
+	# tag latest
+	docker tag $repo:$suite $repo latest
+	
+	# tag the specific debian release version
+	ver=$(docker run $repo:$suite cat /etc/debian_version)
+	docker tag $repo:$suite $repo $ver
+fi
+
+# cleanup
+cd "$returnTo"
+sudo rm -rf "$target"

contrib/mkimage-debootstrap.sh

@@ -1,241 +0,0 @@
-#!/bin/bash
-set -e
-
-variant='minbase'
-include='iproute,iputils-ping'
-arch='amd64' # intentionally undocumented for now
-skipDetection=
-strictDebootstrap=
-justTar=
-
-usage() {
-	echo >&2
-	
-	echo >&2 "usage: $0 [options] repo suite [mirror]"
-	
-	echo >&2
-	echo >&2 'options: (not recommended)'
-	echo >&2 "  -p set an http_proxy for debootstrap"
-	echo >&2 "  -v $variant # change default debootstrap variant"
-	echo >&2 "  -i $include # change default package includes"
-	echo >&2 "  -d # strict debootstrap (do not apply any docker-specific tweaks)"
-	echo >&2 "  -s # skip version detection and tagging (ie, precise also tagged as 12.04)"
-	echo >&2 "     # note that this will also skip adding universe and/or security/updates to sources.list"
-	echo >&2 "  -t # just create a tarball, especially for dockerbrew (uses repo as tarball name)"
-	
-	echo >&2
-	echo >&2 "   ie: $0 username/debian squeeze"
-	echo >&2 "       $0 username/debian squeeze http://ftp.uk.debian.org/debian/"
-	
-	echo >&2
-	echo >&2 "   ie: $0 username/ubuntu precise"
-	echo >&2 "       $0 username/ubuntu precise http://mirrors.melbourne.co.uk/ubuntu/"
-	
-	echo >&2
-	echo >&2 "   ie: $0 -t precise.tar.bz2 precise"
-	echo >&2 "       $0 -t wheezy.tgz wheezy"
-	echo >&2 "       $0 -t wheezy-uk.tar.xz wheezy http://ftp.uk.debian.org/debian/"
-	
-	echo >&2
-}
-
-# these should match the names found at http://www.debian.org/releases/
-debianStable=wheezy
-debianUnstable=sid
-# this should match the name found at http://releases.ubuntu.com/
-ubuntuLatestLTS=precise
-
-while getopts v:i:a:p:dst name; do
-	case "$name" in
-		p)
-			http_proxy="$OPTARG"
-			;;
-		v)
-			variant="$OPTARG"
-			;;
-		i)
-			include="$OPTARG"
-			;;
-		a)
-			arch="$OPTARG"
-			;;
-		d)
-			strictDebootstrap=1
-			;;
-		s)
-			skipDetection=1
-			;;
-		t)
-			justTar=1
-			;;
-		?)
-			usage
-			exit 0
-			;;
-	esac
-done
-shift $(($OPTIND - 1))
-
-repo="$1"
-suite="$2"
-mirror="${3:-}" # stick to the default debootstrap mirror if one is not provided
-
-if [ ! "$repo" ] || [ ! "$suite" ]; then
-	usage
-	exit 1
-fi
-
-# some rudimentary detection for whether we need to "sudo" our docker calls
-docker=''
-if docker version > /dev/null 2>&1; then
-	docker='docker'
-elif sudo docker version > /dev/null 2>&1; then
-	docker='sudo docker'
-elif command -v docker > /dev/null 2>&1; then
-	docker='docker'
-else
-	echo >&2 "warning: either docker isn't installed, or your current user cannot run it;"
-	echo >&2 "         this script is not likely to work as expected"
-	sleep 3
-	docker='docker' # give us a command-not-found later
-fi
-
-# make sure we have an absolute path to our final tarball so we can still reference it properly after we change directory
-if [ "$justTar" ]; then
-	if [ ! -d "$(dirname "$repo")" ]; then
-		echo >&2 "error: $(dirname "$repo") does not exist"
-		exit 1
-	fi
-	repo="$(cd "$(dirname "$repo")" && pwd -P)/$(basename "$repo")"
-fi
-
-# will be filled in later, if [ -z "$skipDetection" ]
-lsbDist=''
-
-target="/tmp/docker-rootfs-debootstrap-$suite-$$-$RANDOM"
-
-cd "$(dirname "$(readlink -f "$BASH_SOURCE")")"
-returnTo="$(pwd -P)"
-
-set -x
-
-# bootstrap
-mkdir -p "$target"
-sudo http_proxy=$http_proxy debootstrap --verbose --variant="$variant" --include="$include" --arch="$arch" "$suite" "$target" "$mirror"
-
-cd "$target"
-
-if [ -z "$strictDebootstrap" ]; then
-	# prevent init scripts from running during install/update
-	#  policy-rc.d (for most scripts)
-	echo $'#!/bin/sh\nexit 101' | sudo tee usr/sbin/policy-rc.d > /dev/null
-	sudo chmod +x usr/sbin/policy-rc.d
-	#  initctl (for some pesky upstart scripts)
-	sudo chroot . dpkg-divert --local --rename --add /sbin/initctl
-	sudo ln -sf /bin/true sbin/initctl
-	# see https://github.com/dotcloud/docker/issues/446#issuecomment-16953173
-	
-	# shrink the image, since apt makes us fat (wheezy: ~157.5MB vs ~120MB)
-	sudo chroot . apt-get clean
-	
-	# while we're at it, apt is unnecessarily slow inside containers
-	#  this forces dpkg not to call sync() after package extraction and speeds up install
-	#    the benefit is huge on spinning disks, and the penalty is nonexistent on SSD or decent server virtualization
-	echo 'force-unsafe-io' | sudo tee etc/dpkg/dpkg.cfg.d/02apt-speedup > /dev/null
-	#  we want to effectively run "apt-get clean" after every install to keep images small (see output of "apt-get clean -s" for context)
-	{
-		aptGetClean='rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
-		echo 'DPkg::Post-Invoke { "'$aptGetClean'"; };'
-		echo 'APT::Update::Post-Invoke { "'$aptGetClean'"; };'
-		echo 'Dir::Cache::pkgcache ""; Dir::Cache::srcpkgcache "";'
-	} | sudo tee etc/apt/apt.conf.d/no-cache > /dev/null
-	#  and remove the translations, too
-	echo 'Acquire::Languages "none";' | sudo tee etc/apt/apt.conf.d/no-languages > /dev/null
-	
-	# helpful undo lines for each the above tweaks (for lack of a better home to keep track of them):
-	#  rm /usr/sbin/policy-rc.d
-	#  rm /sbin/initctl; dpkg-divert --rename --remove /sbin/initctl
-	#  rm /etc/dpkg/dpkg.cfg.d/02apt-speedup
-	#  rm /etc/apt/apt.conf.d/no-cache
-	#  rm /etc/apt/apt.conf.d/no-languages
-	
-	if [ -z "$skipDetection" ]; then
-		# see also rudimentary platform detection in hack/install.sh
-		lsbDist=''
-		if [ -r etc/lsb-release ]; then
-			lsbDist="$(. etc/lsb-release && echo "$DISTRIB_ID")"
-		fi
-		if [ -z "$lsbDist" ] && [ -r etc/debian_version ]; then
-			lsbDist='Debian'
-		fi
-		
-		case "$lsbDist" in
-			Debian)
-				# add the updates and security repositories
-				if [ "$suite" != "$debianUnstable" -a "$suite" != 'unstable' ]; then
-					# ${suite}-updates only applies to non-unstable
-					sudo sed -i "p; s/ $suite main$/ ${suite}-updates main/" etc/apt/sources.list
-					
-					# same for security updates
-					echo "deb http://security.debian.org/ $suite/updates main" | sudo tee -a etc/apt/sources.list > /dev/null
-				fi
-				;;
-			Ubuntu)
-				# add the universe, updates, and security repositories
-				sudo sed -i "
-					s/ $suite main$/ $suite main universe/; p;
-					s/ $suite main/ ${suite}-updates main/; p;
-					s/ $suite-updates main/ ${suite}-security main/
-				" etc/apt/sources.list
-				;;
-		esac
-	fi
-fi
-
-if [ "$justTar" ]; then
-	# create the tarball file so it has the right permissions (ie, not root)
-	touch "$repo"
-	
-	# fill the tarball
-	sudo tar --numeric-owner -caf "$repo" .
-else
-	# create the image (and tag $repo:$suite)
-	sudo tar --numeric-owner -c . | $docker import - $repo:$suite
-	
-	# test the image
-	$docker run -i -t $repo:$suite echo success
-	
-	if [ -z "$skipDetection" ]; then
-		case "$lsbDist" in
-			Debian)
-				if [ "$suite" = "$debianStable" -o "$suite" = 'stable' ] && [ -r etc/debian_version ]; then
-					# tag latest
-					$docker tag $repo:$suite $repo:latest
-					
-					if [ -r etc/debian_version ]; then
-						# tag the specific debian release version (which is only reasonable to tag on debian stable)
-						ver=$(cat etc/debian_version)
-						$docker tag $repo:$suite $repo:$ver
-					fi
-				fi
-				;;
-			Ubuntu)
-				if [ "$suite" = "$ubuntuLatestLTS" ]; then
-					# tag latest
-					$docker tag $repo:$suite $repo:latest
-				fi
-				if [ -r etc/lsb-release ]; then
-					lsbRelease="$(. etc/lsb-release && echo "$DISTRIB_RELEASE")"
-					if [ "$lsbRelease" ]; then
-						# tag specific Ubuntu version number, if available (12.04, etc.)
-						$docker tag $repo:$suite $repo:$lsbRelease
-					fi
-				fi
-				;;
-		esac
-	fi
-fi
-
-# cleanup
-cd "$returnTo"
-sudo rm -rf "$target"

contrib/mkimage-rinse.sh

@@ -1,112 +0,0 @@
-#!/bin/bash
-set -e
-
-repo="$1"
-distro="$2"
-mirror="$3"
-
-if [ ! "$repo" ] || [ ! "$distro" ]; then
-	self="$(basename $0)"
-	echo >&2 "usage: $self repo distro [mirror]"
-	echo >&2
-	echo >&2 "   ie: $self username/centos centos-5"
-	echo >&2 "       $self username/centos centos-6"
-	echo >&2
-	echo >&2 "   ie: $self username/slc slc-5"
-	echo >&2 "       $self username/slc slc-6"
-	echo >&2
-	echo >&2 "   ie: $self username/centos centos-5 http://vault.centos.org/5.8/os/x86_64/CentOS/"
-	echo >&2 "       $self username/centos centos-6 http://vault.centos.org/6.3/os/x86_64/Packages/"
-	echo >&2
-	echo >&2 'See /etc/rinse for supported values of "distro" and for examples of'
-	echo >&2 '  expected values of "mirror".'
-	echo >&2
-	echo >&2 'This script is tested to work with the original upstream version of rinse,'
-	echo >&2 '  found at http://www.steve.org.uk/Software/rinse/ and also in Debian at'
-	echo >&2 '  http://packages.debian.org/wheezy/rinse -- as always, YMMV.'
-	echo >&2
-	exit 1
-fi
-
-target="/tmp/docker-rootfs-rinse-$distro-$$-$RANDOM"
-
-cd "$(dirname "$(readlink -f "$BASH_SOURCE")")"
-returnTo="$(pwd -P)"
-
-rinseArgs=( --arch amd64 --distribution "$distro" --directory "$target" )
-if [ "$mirror" ]; then
-	rinseArgs+=( --mirror "$mirror" )
-fi
-
-set -x
-
-mkdir -p "$target"
-
-sudo rinse "${rinseArgs[@]}"
-
-cd "$target"
-
-# rinse fails a little at setting up /dev, so we'll just wipe it out and create our own
-sudo rm -rf dev
-sudo mkdir -m 755 dev
-(
-	cd dev
-	sudo ln -sf /proc/self/fd ./
-	sudo mkdir -m 755 pts
-	sudo mkdir -m 1777 shm
-	sudo mknod -m 600 console c 5 1
-	sudo mknod -m 600 initctl p
-	sudo mknod -m 666 full c 1 7
-	sudo mknod -m 666 null c 1 3
-	sudo mknod -m 666 ptmx c 5 2
-	sudo mknod -m 666 random c 1 8
-	sudo mknod -m 666 tty c 5 0
-	sudo mknod -m 666 tty0 c 4 0
-	sudo mknod -m 666 urandom c 1 9
-	sudo mknod -m 666 zero c 1 5
-)
-
-# effectively: febootstrap-minimize --keep-zoneinfo --keep-rpmdb --keep-services "$target"
-#  locales
-sudo rm -rf usr/{{lib,share}/locale,{lib,lib64}/gconv,bin/localedef,sbin/build-locale-archive}
-#  docs
-sudo rm -rf usr/share/{man,doc,info,gnome/help}
-#  cracklib
-sudo rm -rf usr/share/cracklib
-#  i18n
-sudo rm -rf usr/share/i18n
-#  yum cache
-sudo rm -rf var/cache/yum
-sudo mkdir -p --mode=0755 var/cache/yum
-#  sln
-sudo rm -rf sbin/sln
-#  ldconfig
-#sudo rm -rf sbin/ldconfig
-sudo rm -rf etc/ld.so.cache var/cache/ldconfig
-sudo mkdir -p --mode=0755 var/cache/ldconfig
-
-# allow networking init scripts inside the container to work without extra steps
-echo 'NETWORKING=yes' | sudo tee etc/sysconfig/network > /dev/null
-
-# to restore locales later:
-#  yum reinstall glibc-common
-
-version=
-if [ -r etc/redhat-release ]; then
-	version="$(sed -E 's/^[^0-9.]*([0-9.]+).*$/\1/' etc/redhat-release)"
-elif [ -r etc/SuSE-release ]; then
-	version="$(awk '/^VERSION/ { print $3 }' etc/SuSE-release)"
-fi
-
-if [ -z "$version" ]; then
-	echo >&2 "warning: cannot autodetect OS version, using $distro as tag"
-	sleep 20
-	version="$distro"
-fi
-
-sudo tar --numeric-owner -c . | docker import - $repo:$version
-
-docker run -i -t $repo:$version echo success
-
-cd "$returnTo"
-sudo rm -rf "$target"

contrib/mkimage-unittest.sh

@@ -44,6 +44,6 @@ do
 done
 
 chmod 0755 $ROOTFS # See #486
-tar --numeric-owner -cf- . | docker import - docker-ut
+tar -cf- . | docker import - docker-ut
 docker run -i -u root docker-ut /bin/echo Success.
 rm -rf $ROOTFS

contrib/mkseccomp.pl

@@ -1,77 +0,0 @@
-#!/usr/bin/perl
-#
-# A simple helper script to help people build seccomp profiles for
-# Docker/LXC.  The goal is mostly to reduce the attack surface to the
-# kernel, by restricting access to rarely used, recently added or not used
-# syscalls.
-#
-# This script processes one or more files which contain the list of system
-# calls to be allowed.  See mkseccomp.sample for more information how you
-# can configure the list of syscalls.  When run, this script produces output
-# which, when stored in a file, can be passed to docker as follows:
-#
-# docker run -lxc-conf="lxc.seccomp=$file" <rest of arguments>
-#
-# The included sample file shows how to cut about a quarter of all syscalls,
-# which affecting most applications.
-#
-# For specific situations it is possible to reduce the list further. By
-# reducing the list to just those syscalls required by a certain application
-# you can make it difficult for unknown/unexpected code to run.
-#
-# Run this script as follows:
-#
-# ./mkseccomp.pl < mkseccomp.sample >syscalls.list
-# or
-# ./mkseccomp.pl mkseccomp.sample >syscalls.list
-#
-# Multiple files can be specified, in which case the lists of syscalls are
-# combined.
-#
-# By Martijn van Oosterhout <kleptog@svana.org> Nov 2013
-
-# How it works:
-#
-# This program basically spawns two processes to form a chain like:
-#
-# <process data section to prefix __NR_> | cpp | <add header and filter unknown syscalls>
-
-use strict;
-use warnings;
-
-if( -t ) {
-    print STDERR "Helper script to make seccomp filters for Docker/LXC.\n";
-    print STDERR "Usage: mkseccomp.pl [files...]\n";
-    exit 1;
-}
-
-my $pid = open(my $in, "-|") // die "Couldn't fork1 ($!)\n";
-
-if($pid == 0) {  # Child
-    $pid = open(my $out, "|-") // die "Couldn't fork2 ($!)\n";
-
-    if($pid == 0) { # Child, which execs cpp
-        exec "cpp" or die "Couldn't exec cpp ($!)\n";
-        exit 1;
-    }
-
-    # Process the DATA section and output to cpp
-    print $out "#include <sys/syscall.h>\n";
-    while(<>) {
-        if(/^\w/) {
-            print $out "__NR_$_";
-        }
-    }
-    close $out;
-    exit 0;
-
-}
-
-# Print header and then process output from cpp.
-print "1\n";
-print "whitelist\n";
-
-while(<$in>) {
-    print if( /^[0-9]/ );
-}
-

contrib/mkseccomp.sample

@@ -1,444 +0,0 @@
-/* This sample file is an example for mkseccomp.pl to produce a seccomp file
- * which restricts syscalls that are only useful for an admin but allows the
- * vast majority of normal userspace programs to run normally.
- *
- * The format of this file is one line per syscall.  This is then processed
- * and passed to 'cpp' to convert the names to numbers using whatever is
- * correct for your platform.  As such C-style comments are permitted.  Note
- * this also means that C preprocessor macros are also allowed.  So it is
- * possible to create groups surrounded by #ifdef/#endif and control their
- * inclusion via #define (not #include).
- *
- * Syscalls that don't exist on your architecture are silently filtered out.
- * Syscalls marked with (*) are required for a container to spawn a bash
- * shell successfully (not necessarily full featured).  Listing the same
- * syscall multiple times is no problem.
- *
- * If you want to make a list specifically for one application the easiest
- * way is to run the application under strace, like so:
- *
- * $ strace -f -q -c -o strace.out application args...
- *
- * Once you have a reasonable sample of the execution of the program, exit
- * it.  The file strace.out will have a summary of the syscalls used.  Copy
- * that list into this file, comment out everything else except the starred
- * syscalls (which you need for the container to start) and you're done.
- *
- * To get the list of syscalls from the strace output this works well for
- * me
- *
- * $ cut -c52 < strace.out
- *
- * This sample list was compiled as a combination of all the syscalls
- * available on i386 and amd64 on Ubuntu Precise, as such it may not contain
- * everything and not everything may be relevent for your system.  This
- * shouldn't be a problem.
- */
-
-// Filesystem/File descriptor related
-access                 // (*)
-chdir                  // (*)
-chmod
-chown
-chown32
-close                  // (*)
-creat
-dup                    // (*)
-dup2                   // (*)
-dup3
-epoll_create
-epoll_create1
-epoll_ctl
-epoll_ctl_old
-epoll_pwait
-epoll_wait
-epoll_wait_old
-eventfd
-eventfd2
-faccessat              // (*)
-fadvise64
-fadvise64_64
-fallocate
-fanotify_init
-fanotify_mark
-ioctl                  // (*)
-fchdir
-fchmod
-fchmodat
-fchown
-fchown32
-fchownat
-fcntl                  // (*)
-fcntl64
-fdatasync
-fgetxattr
-flistxattr
-flock
-fremovexattr
-fsetxattr
-fstat                  // (*)
-fstat64
-fstatat64
-fstatfs
-fstatfs64
-fsync
-ftruncate
-ftruncate64
-getcwd                 // (*)
-getdents               // (*)
-getdents64
-getxattr
-inotify_add_watch
-inotify_init
-inotify_init1
-inotify_rm_watch
-io_cancel
-io_destroy
-io_getevents
-io_setup
-io_submit
-lchown
-lchown32
-lgetxattr
-link
-linkat
-listxattr
-llistxattr
-llseek
-_llseek
-lremovexattr
-lseek                  // (*)
-lsetxattr
-lstat
-lstat64
-mkdir
-mkdirat
-mknod
-mknodat
-newfstatat
-_newselect
-oldfstat
-oldlstat
-oldolduname
-oldstat
-olduname
-oldwait4
-open                   // (*)
-openat                 // (*)
-pipe                   // (*)
-pipe2
-poll
-ppoll
-pread64
-preadv
-futimesat
-pselect6
-pwrite64
-pwritev
-read                   // (*)
-readahead
-readdir
-readlink
-readlinkat
-readv
-removexattr
-rename
-renameat
-rmdir
-select
-sendfile
-sendfile64
-setxattr
-splice
-stat                   // (*)
-stat64
-statfs                 // (*)
-statfs64
-symlink
-symlinkat
-sync
-sync_file_range
-sync_file_range2
-syncfs
-tee
-truncate
-truncate64
-umask
-unlink
-unlinkat
-ustat
-utime
-utimensat
-utimes
-write                  // (*)
-writev
-
-// Network related
-accept
-accept4
-bind                   // (*)
-connect                // (*)
-getpeername
-getsockname            // (*)
-getsockopt
-listen
-recv
-recvfrom               // (*)
-recvmmsg
-recvmsg
-send
-sendmmsg
-sendmsg
-sendto                 // (*)
-setsockopt
-shutdown
-socket                 // (*)
-socketcall
-socketpair
-
-// Signal related
-pause
-rt_sigaction           // (*)
-rt_sigpending
-rt_sigprocmask         // (*)
-rt_sigqueueinfo
-rt_sigreturn           // (*)
-rt_sigsuspend
-rt_sigtimedwait
-rt_tgsigqueueinfo
-sigaction
-sigaltstack            // (*)
-signal
-signalfd
-signalfd4
-sigpending
-sigprocmask
-sigreturn
-sigsuspend
-
-// Other needed POSIX
-alarm
-brk                    // (*)
-clock_adjtime
-clock_getres
-clock_gettime
-clock_nanosleep
-//clock_settime
-gettimeofday
-nanosleep
-nice
-sysinfo
-syslog
-time
-timer_create
-timer_delete
-timerfd_create
-timerfd_gettime
-timerfd_settime
-timer_getoverrun
-timer_gettime
-timer_settime
-times
-uname                  // (*)
-
-// Memory control
-madvise
-mbind
-mincore
-mlock
-mlockall
-mmap                   // (*)
-mmap2
-mprotect               // (*)
-mremap
-msync
-munlock
-munlockall
-munmap                 // (*)
-remap_file_pages
-set_mempolicy
-vmsplice
-
-// Process control
-capget
-//capset
-clone                  // (*)
-execve                 // (*)
-exit                   // (*)
-exit_group             // (*)
-fork
-getcpu
-getpgid
-getpgrp                // (*)
-getpid                 // (*)
-getppid                // (*)
-getpriority
-getresgid
-getresgid32
-getresuid
-getresuid32
-getrlimit              // (*)
-getrusage
-getsid
-getuid                 // (*)
-getuid32
-getegid                // (*)
-getegid32
-geteuid                // (*)
-geteuid32
-getgid                 // (*)
-getgid32
-getgroups
-getgroups32
-getitimer
-get_mempolicy
-kill
-//personality
-prctl
-prlimit64
-sched_getaffinity
-sched_getparam
-sched_get_priority_max
-sched_get_priority_min
-sched_getscheduler
-sched_rr_get_interval
-//sched_setaffinity
-//sched_setparam
-//sched_setscheduler
-sched_yield
-setfsgid
-setfsgid32
-setfsuid
-setfsuid32
-setgid
-setgid32
-setgroups
-setgroups32
-setitimer
-setpgid                // (*)
-setpriority
-setregid
-setregid32
-setresgid
-setresgid32
-setresuid
-setresuid32
-setreuid
-setreuid32
-setrlimit
-setsid
-setuid
-setuid32
-ugetrlimit
-vfork
-wait4                  // (*)
-waitid
-waitpid
-
-// IPC
-ipc
-mq_getsetattr
-mq_notify
-mq_open
-mq_timedreceive
-mq_timedsend
-mq_unlink
-msgctl
-msgget
-msgrcv
-msgsnd
-semctl
-semget
-semop
-semtimedop
-shmat
-shmctl
-shmdt
-shmget
-
-// Linux specific, mostly needed for thread-related stuff
-arch_prctl             // (*)
-get_robust_list
-get_thread_area
-gettid
-futex                  // (*)
-restart_syscall        // (*)
-set_robust_list        // (*)
-set_thread_area
-set_tid_address        // (*)
-tgkill
-tkill
-
-// Admin syscalls, these are blocked
-//acct
-//adjtimex
-//bdflush
-//chroot
-//create_module
-//delete_module
-//get_kernel_syms      // Obsolete
-//idle                 // Obsolete
-//init_module
-//ioperm
-//iopl
-//ioprio_get
-//ioprio_set
-//kexec_load
-//lookup_dcookie       // oprofile only?
-//migrate_pages        // NUMA
-//modify_ldt
-//mount
-//move_pages           // NUMA
-//name_to_handle_at    // NFS server
-//nfsservctl           // NFS server
-//open_by_handle_at    // NFS server
-//perf_event_open
-//pivot_root
-//process_vm_readv     // For debugger
-//process_vm_writev    // For debugger
-//ptrace               // For debugger
-//query_module
-//quotactl
-//reboot
-//setdomainname
-//sethostname
-//setns
-//settimeofday
-//sgetmask             // Obsolete
-//ssetmask             // Obsolete
-//stime
-//swapoff
-//swapon
-//_sysctl
-//sysfs
-//sys_setaltroot
-//umount
-//umount2
-//unshare
-//uselib
-//vhangup
-//vm86
-//vm86old
-
-// Kernel key management
-//add_key
-//keyctl
-//request_key
-
-// Unimplemented
-//afs_syscall
-//break
-//ftime
-//getpmsg
-//gtty
-//lock
-//madvise1
-//mpx
-//prof
-//profil
-//putpmsg
-//security
-//stty
-//tuxcall
-//ulimit
-//vserver

contrib/udev/80-docker.rules

@@ -1,3 +0,0 @@
-# hide docker's loopback devices from udisks, and thus from user desktops
-SUBSYSTEM=="block", ENV{DM_NAME}=="docker-*", ENV{UDISKS_PRESENTATION_HIDE}="1", ENV{UDISKS_IGNORE}="1"
-SUBSYSTEM=="block", DEVPATH=="/devices/virtual/block/loop*", ATTR{loop/backing_file}=="/var/lib/docker/*", ENV{UDISKS_PRESENTATION_HIDE}="1", ENV{UDISKS_IGNORE}="1"

contrib/vagrant-docker/README.md

@@ -1,50 +1,3 @@
-# Vagrant integration
+# Vagrant-docker
 
-Currently there are at least 4 different projects that we are aware of that deals
-with integration with [Vagrant](http://vagrantup.com/) at different levels. One
-approach is to use Docker as a [provisioner](http://docs.vagrantup.com/v2/provisioning/index.html)
-which means you can create containers and pull base images on VMs using Docker's
-CLI and the other is to use Docker as a [provider](http://docs.vagrantup.com/v2/providers/index.html),
-meaning you can use Vagrant to control Docker containers.
-
-
-### Provisioners
-
-* [Vocker](https://github.com/fgrehm/vocker)
-* [Ventriloquist](https://github.com/fgrehm/ventriloquist)
-
-### Providers
-
-* [docker-provider](https://github.com/fgrehm/docker-provider)
-* [vagrant-shell](https://github.com/destructuring/vagrant-shell)
-
-## Setting up Vagrant-docker with the Remote API
-
-The initial Docker upstart script will not work because it runs on `127.0.0.1`, which is not accessible to the host machine. Instead, we need to change the script to connect to `0.0.0.0`. To do this, modify `/etc/init/docker.conf` to look like this:
-
-```
-description     "Docker daemon"
-
-start on filesystem and started lxc-net
-stop on runlevel [!2345]
-
-respawn
-
-script
-    /usr/bin/docker -d -H=tcp://0.0.0.0:4243/
-end script
-```
-
-Once that's done, you need to set up a SSH tunnel between your host machine and the vagrant machine that's running Docker. This can be done by running the following command in a host terminal:
-
-```
-ssh -L 4243:localhost:4243 -p 2222 vagrant@localhost
-```
-
-(The first 4243 is what your host can connect to, the second 4243 is what port Docker is running on in the vagrant machine, and the 2222 is the port Vagrant is providing for SSH. If VirtualBox is the VM you're using, you can see what value "2222" should be by going to: Network > Adapter 1 > Advanced > Port Forwarding in the VirtualBox GUI.)
-
-Note that because the port has been changed, to run docker commands from within the command line you must run them like this:
-
-```
-sudo docker -H 0.0.0.0:4243 < commands for docker >
-```
+This is a placeholder for the official vagrant-docker, a plugin for Vagrant (http://vagrantup.com) which exposes Docker as a provider.

contrib/vim-syntax/syntax/dockerfile.vim

@@ -20,5 +20,3 @@ highlight link dockerfileString String
 
 syntax match dockerfileComment "\v^\s*#.*$"
 highlight link dockerfileComment Comment
-
-set commentstring=#\ %s

contrib/zfs/MAINTAINERS

@@ -1 +0,0 @@
-Gurjeet Singh <gurjeet@singh.im> (gurjeet.singh.im)

contrib/zfs/README.md

@@ -1,23 +0,0 @@
-# ZFS Storage Driver
-
-This is a placeholder to declare the presence and status of ZFS storage driver
-for containers.
-
-The current development is done in Gurjeet Singh's fork of Docker, under the
-branch named [zfs_driver].
-
-[zfs_driver]: https://github.com/gurjeet/docker/tree/zfs_driver
-
-
-# Status
-
-Alpha: The code is now capable of creating, running and destroying containers
-and images.
-
-The code is under development. Contributions in the form of suggestions,
-code-reviews, and patches are welcome.
-
-Please send the communication to gurjeet@singh.im and CC at least one Docker
-mailing list.
-
-

devmapper/deviceset.go

@@ -1,18 +1,18 @@
-// +build linux
-
 package devmapper
 
 import (
 	"encoding/json"
-	"errors"
 	"fmt"
 	"github.com/dotcloud/docker/utils"
 	"io"
 	"io/ioutil"
+	"os"
+	"os/exec"
 	"path"
 	"path/filepath"
 	"strconv"
 	"sync"
+	"syscall"
 	"time"
 )
 
@@ -38,6 +38,7 @@ type MetaData struct {
 type DeviceSet struct {
 	MetaData
 	sync.Mutex
+	initialized      bool
 	root             string
 	devicePrefix     string
 	TransactionId    uint64
@@ -57,16 +58,6 @@ type Status struct {
 	MetadataLoopback string
 	Data             DiskUsage
 	Metadata         DiskUsage
-	SectorSize       uint64
-}
-
-type DevStatus struct {
-	DeviceId            int
-	Size                uint64
-	TransactionId       uint64
-	SizeInSectors       uint64
-	MappedSectors       uint64
-	HighestMappedSector uint64
 }
 
 func getDevName(name string) string {
@@ -105,7 +96,7 @@ func (devices *DeviceSet) hasImage(name string) bool {
 	dirname := devices.loopbackDir()
 	filename := path.Join(dirname, name)
 
-	_, err := osStat(filename)
+	_, err := os.Stat(filename)
 	return err == nil
 }
 
@@ -117,20 +108,19 @@ func (devices *DeviceSet) ensureImage(name string, size int64) (string, error) {
 	dirname := devices.loopbackDir()
 	filename := path.Join(dirname, name)
 
-	if err := osMkdirAll(dirname, 0700); err != nil && !osIsExist(err) {
+	if err := os.MkdirAll(dirname, 0700); err != nil && !os.IsExist(err) {
 		return "", err
 	}
 
-	if _, err := osStat(filename); err != nil {
-		if !osIsNotExist(err) {
+	if _, err := os.Stat(filename); err != nil {
+		if !os.IsNotExist(err) {
 			return "", err
 		}
 		utils.Debugf("Creating loopback file %s for device-manage use", filename)
-		file, err := osOpenFile(filename, osORdWr|osOCreate, 0600)
+		file, err := os.OpenFile(filename, os.O_RDWR|os.O_CREATE, 0600)
 		if err != nil {
 			return "", err
 		}
-		defer file.Close()
 
 		if err = file.Truncate(size); err != nil {
 			return "", err
@@ -174,8 +164,8 @@ func (devices *DeviceSet) saveMetadata() error {
 	if err := tmpFile.Close(); err != nil {
 		return fmt.Errorf("Error closing metadata file %s: %s", tmpFile.Name(), err)
 	}
-	if err := osRename(tmpFile.Name(), devices.jsonFile()); err != nil {
-		return fmt.Errorf("Error committing metadata file %s: %s", tmpFile.Name(), err)
+	if err := os.Rename(tmpFile.Name(), devices.jsonFile()); err != nil {
+		return fmt.Errorf("Error committing metadata file", err)
 	}
 
 	if devices.NewTransactionId != devices.TransactionId {
@@ -225,9 +215,9 @@ func (devices *DeviceSet) activateDeviceIfNeeded(hash string) error {
 func (devices *DeviceSet) createFilesystem(info *DevInfo) error {
 	devname := info.DevName()
 
-	err := execRun("mkfs.ext4", "-E", "discard,lazy_itable_init=0,lazy_journal_init=0", devname)
+	err := exec.Command("mkfs.ext4", "-E", "discard,lazy_itable_init=0,lazy_journal_init=0", devname).Run()
 	if err != nil {
-		err = execRun("mkfs.ext4", "-E", "discard,lazy_itable_init=0", devname)
+		err = exec.Command("mkfs.ext4", "-E", "discard,lazy_itable_init=0", devname).Run()
 	}
 	if err != nil {
 		utils.Debugf("\n--->Err: %s\n", err)
@@ -252,7 +242,7 @@ func (devices *DeviceSet) loadMetaData() error {
 	devices.NewTransactionId = devices.TransactionId
 
 	jsonData, err := ioutil.ReadFile(devices.jsonFile())
-	if err != nil && !osIsNotExist(err) {
+	if err != nil && !os.IsNotExist(err) {
 		utils.Debugf("\n--->Err: %s\n", err)
 		return err
 	}
@@ -337,13 +327,14 @@ func (devices *DeviceSet) setupBaseImage() error {
 }
 
 func setCloseOnExec(name string) {
-	if fileInfos, _ := ioutil.ReadDir("/proc/self/fd"); fileInfos != nil {
+	fileInfos, _ := ioutil.ReadDir("/proc/self/fd")
+	if fileInfos != nil {
 		for _, i := range fileInfos {
-			link, _ := osReadlink(filepath.Join("/proc/self/fd", i.Name()))
+			link, _ := os.Readlink(filepath.Join("/proc/self/fd", i.Name()))
 			if link == name {
 				fd, err := strconv.Atoi(i.Name())
 				if err == nil {
-					sysCloseOnExec(fd)
+					syscall.CloseOnExec(fd)
 				}
 			}
 		}
@@ -366,90 +357,13 @@ func minor(device uint64) uint64 {
 	return (device & 0xff) | ((device >> 12) & 0xfff00)
 }
 
-func (devices *DeviceSet) ResizePool(size int64) error {
-	dirname := devices.loopbackDir()
-	datafilename := path.Join(dirname, "data")
-	metadatafilename := path.Join(dirname, "metadata")
-
-	datafile, err := osOpenFile(datafilename, osORdWr, 0)
-	if datafile == nil {
-		return err
-	}
-	defer datafile.Close()
-
-	fi, err := datafile.Stat()
-	if fi == nil {
-		return err
-	}
-
-	if fi.Size() > size {
-		return fmt.Errorf("Can't shrink file")
-	}
-
-	dataloopback := FindLoopDeviceFor(datafile)
-	if dataloopback == nil {
-		return fmt.Errorf("Unable to find loopback mount for: %s", datafilename)
-	}
-	defer dataloopback.Close()
-
-	metadatafile, err := osOpenFile(metadatafilename, osORdWr, 0)
-	if metadatafile == nil {
-		return err
-	}
-	defer metadatafile.Close()
-
-	metadataloopback := FindLoopDeviceFor(metadatafile)
-	if metadataloopback == nil {
-		return fmt.Errorf("Unable to find loopback mount for: %s", metadatafilename)
-	}
-	defer metadataloopback.Close()
-
-	// Grow loopback file
-	if err := datafile.Truncate(size); err != nil {
-		return fmt.Errorf("Unable to grow loopback file: %s", err)
-	}
-
-	// Reload size for loopback device
-	if err := LoopbackSetCapacity(dataloopback); err != nil {
-		return fmt.Errorf("Unable to update loopback capacity: %s", err)
-	}
-
-	// Suspend the pool
-	if err := suspendDevice(devices.getPoolName()); err != nil {
-		return fmt.Errorf("Unable to suspend pool: %s", err)
-	}
-
-	// Reload with the new block sizes
-	if err := reloadPool(devices.getPoolName(), dataloopback, metadataloopback); err != nil {
-		return fmt.Errorf("Unable to reload pool: %s", err)
-	}
-
-	// Resume the pool
-	if err := resumeDevice(devices.getPoolName()); err != nil {
-		return fmt.Errorf("Unable to resume pool: %s", err)
-	}
-
-	return nil
-}
-
-func (devices *DeviceSet) initDevmapper(doInit bool) error {
+func (devices *DeviceSet) initDevmapper() error {
 	logInit(devices)
 
 	// Make sure the sparse images exist in <root>/devicemapper/data and
 	// <root>/devicemapper/metadata
 
-	hasData := devices.hasImage("data")
-	hasMetadata := devices.hasImage("metadata")
-
-	if !doInit && !hasData {
-		return errors.New("Loopback data file not found")
-	}
-
-	if !doInit && !hasMetadata {
-		return errors.New("Loopback metadata file not found")
-	}
-
-	createdLoopback := !hasData || !hasMetadata
+	createdLoopback := !devices.hasImage("data") || !devices.hasImage("metadata")
 	data, err := devices.ensureImage("data", DefaultDataLoopbackSize)
 	if err != nil {
 		utils.Debugf("Error device ensureImage (data): %s\n", err)
@@ -463,11 +377,11 @@ func (devices *DeviceSet) initDevmapper(doInit bool) error {
 
 	// Set the device prefix from the device id and inode of the docker root dir
 
-	st, err := osStat(devices.root)
+	st, err := os.Stat(devices.root)
 	if err != nil {
 		return fmt.Errorf("Error looking up dir %s: %s", devices.root, err)
 	}
-	sysSt := toSysStatT(st.Sys())
+	sysSt := st.Sys().(*syscall.Stat_t)
 	// "reg-" stands for "regular file".
 	// In the future we might use "dev-" for "device file", etc.
 	// docker-maj,min[-inode] stands for:
@@ -494,14 +408,14 @@ func (devices *DeviceSet) initDevmapper(doInit bool) error {
 	if info.Exists == 0 {
 		utils.Debugf("Pool doesn't exist. Creating it.")
 
-		dataFile, err := attachLoopDevice(data)
+		dataFile, err := AttachLoopDevice(data)
 		if err != nil {
 			utils.Debugf("\n--->Err: %s\n", err)
 			return err
 		}
 		defer dataFile.Close()
 
-		metadataFile, err := attachLoopDevice(metadata)
+		metadataFile, err := AttachLoopDevice(metadata)
 		if err != nil {
 			utils.Debugf("\n--->Err: %s\n", err)
 			return err
@@ -524,11 +438,9 @@ func (devices *DeviceSet) initDevmapper(doInit bool) error {
 	}
 
 	// Setup the base image
-	if doInit {
-		if err := devices.setupBaseImage(); err != nil {
-			utils.Debugf("Error device setupBaseImage: %s\n", err)
-			return err
-		}
+	if err := devices.setupBaseImage(); err != nil {
+		utils.Debugf("Error device setupBaseImage: %s\n", err)
+		return err
 	}
 
 	return nil
@@ -538,6 +450,11 @@ func (devices *DeviceSet) AddDevice(hash, baseHash string) error {
 	devices.Lock()
 	defer devices.Unlock()
 
+	if err := devices.ensureInit(); err != nil {
+		utils.Debugf("Error init: %s\n", err)
+		return err
+	}
+
 	if devices.Devices[hash] != nil {
 		return fmt.Errorf("hash %s already exists", hash)
 	}
@@ -605,6 +522,11 @@ func (devices *DeviceSet) RemoveDevice(hash string) error {
 	devices.Lock()
 	defer devices.Unlock()
 
+	if err := devices.ensureInit(); err != nil {
+		utils.Debugf("\n--->Err: %s\n", err)
+		return err
+	}
+
 	return devices.removeDevice(hash)
 }
 
@@ -640,7 +562,7 @@ func (devices *DeviceSet) deactivateDevice(hash string) error {
 // or b) the 1 second timeout expires.
 func (devices *DeviceSet) waitRemove(hash string) error {
 	utils.Debugf("[deviceset %s] waitRemove(%s)", devices.devicePrefix, hash)
-	defer utils.Debugf("[deviceset %s] waitRemove(%) END", devices.devicePrefix, hash)
+	defer utils.Debugf("[deviceset %s] waitRemove END", devices.devicePrefix, hash)
 	devname, err := devices.byHash(hash)
 	if err != nil {
 		return err
@@ -653,13 +575,10 @@ func (devices *DeviceSet) waitRemove(hash string) error {
 			// The error might actually be something else, but we can't differentiate.
 			return nil
 		}
-		if i%100 == 0 {
-			utils.Debugf("Waiting for removal of %s: exists=%d", devname, devinfo.Exists)
-		}
+		utils.Debugf("Waiting for removal of %s: exists=%d", devname, devinfo.Exists)
 		if devinfo.Exists == 0 {
 			break
 		}
-
 		time.Sleep(1 * time.Millisecond)
 	}
 	if i == 1000 {
@@ -682,9 +601,7 @@ func (devices *DeviceSet) waitClose(hash string) error {
 		if err != nil {
 			return err
 		}
-		if i%100 == 0 {
-			utils.Debugf("Waiting for unmount of %s: opencount=%d", devname, devinfo.OpenCount)
-		}
+		utils.Debugf("Waiting for unmount of %s: opencount=%d", devname, devinfo.OpenCount)
 		if devinfo.OpenCount == 0 {
 			break
 		}
@@ -712,16 +629,19 @@ func (devices *DeviceSet) byHash(hash string) (devname string, err error) {
 }
 
 func (devices *DeviceSet) Shutdown() error {
+	utils.Debugf("[deviceset %s] shutdown()", devices.devicePrefix)
+	defer utils.Debugf("[deviceset %s] shutdown END", devices.devicePrefix)
 	devices.Lock()
+	utils.Debugf("[devmapper] Shutting down DeviceSet: %s", devices.root)
 	defer devices.Unlock()
 
-	utils.Debugf("[deviceset %s] shutdown()", devices.devicePrefix)
-	utils.Debugf("[devmapper] Shutting down DeviceSet: %s", devices.root)
-	defer utils.Debugf("[deviceset %s] shutdown END", devices.devicePrefix)
+	if !devices.initialized {
+		return nil
+	}
 
 	for path, count := range devices.activeMounts {
 		for i := count; i > 0; i-- {
-			if err := sysUnmount(path, 0); err != nil {
+			if err := syscall.Unmount(path, 0); err != nil {
 				utils.Debugf("Shutdown unmounting %s, error: %s\n", path, err)
 			}
 		}
@@ -751,21 +671,25 @@ func (devices *DeviceSet) MountDevice(hash, path string, readOnly bool) error {
 	devices.Lock()
 	defer devices.Unlock()
 
+	if err := devices.ensureInit(); err != nil {
+		return fmt.Errorf("Error initializing devmapper: %s", err)
+	}
+
 	if err := devices.activateDeviceIfNeeded(hash); err != nil {
 		return fmt.Errorf("Error activating devmapper device for '%s': %s", hash, err)
 	}
 
 	info := devices.Devices[hash]
 
-	var flags uintptr = sysMsMgcVal
+	var flags uintptr = syscall.MS_MGC_VAL
 
 	if readOnly {
-		flags = flags | sysMsRdOnly
+		flags = flags | syscall.MS_RDONLY
 	}
 
-	err := sysMount(info.DevName(), path, "ext4", flags, "discard")
-	if err != nil && err == sysEInval {
-		err = sysMount(info.DevName(), path, "ext4", flags, "")
+	err := syscall.Mount(info.DevName(), path, "ext4", flags, "discard")
+	if err != nil && err == syscall.EINVAL {
+		err = syscall.Mount(info.DevName(), path, "ext4", flags, "")
 	}
 	if err != nil {
 		return fmt.Errorf("Error mounting '%s' on '%s': %s", info.DevName(), path, err)
@@ -774,7 +698,7 @@ func (devices *DeviceSet) MountDevice(hash, path string, readOnly bool) error {
 	count := devices.activeMounts[path]
 	devices.activeMounts[path] = count + 1
 
-	return devices.setInitialized(hash)
+	return nil
 }
 
 func (devices *DeviceSet) UnmountDevice(hash, path string, deactivate bool) error {
@@ -784,7 +708,7 @@ func (devices *DeviceSet) UnmountDevice(hash, path string, deactivate bool) erro
 	defer devices.Unlock()
 
 	utils.Debugf("[devmapper] Unmount(%s)", path)
-	if err := sysUnmount(path, 0); err != nil {
+	if err := syscall.Unmount(path, 0); err != nil {
 		utils.Debugf("\n--->Err: %s\n", err)
 		return err
 	}
@@ -812,6 +736,9 @@ func (devices *DeviceSet) HasDevice(hash string) bool {
 	devices.Lock()
 	defer devices.Unlock()
 
+	if err := devices.ensureInit(); err != nil {
+		return false
+	}
 	return devices.Devices[hash] != nil
 }
 
@@ -819,6 +746,10 @@ func (devices *DeviceSet) HasInitializedDevice(hash string) bool {
 	devices.Lock()
 	defer devices.Unlock()
 
+	if err := devices.ensureInit(); err != nil {
+		return false
+	}
+
 	info := devices.Devices[hash]
 	return info != nil && info.Initialized
 }
@@ -827,6 +758,10 @@ func (devices *DeviceSet) HasActivatedDevice(hash string) bool {
 	devices.Lock()
 	defer devices.Unlock()
 
+	if err := devices.ensureInit(); err != nil {
+		return false
+	}
+
 	info := devices.Devices[hash]
 	if info == nil {
 		return false
@@ -835,7 +770,15 @@ func (devices *DeviceSet) HasActivatedDevice(hash string) bool {
 	return devinfo != nil && devinfo.Exists != 0
 }
 
-func (devices *DeviceSet) setInitialized(hash string) error {
+func (devices *DeviceSet) SetInitialized(hash string) error {
+	devices.Lock()
+	defer devices.Unlock()
+
+	if err := devices.ensureInit(); err != nil {
+		utils.Debugf("\n--->Err: %s\n", err)
+		return err
+	}
+
 	info := devices.Devices[hash]
 	if info == nil {
 		return fmt.Errorf("Unknown device %s", hash)
@@ -851,109 +794,57 @@ func (devices *DeviceSet) setInitialized(hash string) error {
 	return nil
 }
 
-func (devices *DeviceSet) List() []string {
-	devices.Lock()
-	defer devices.Unlock()
-
-	ids := make([]string, len(devices.Devices))
-	i := 0
-	for k := range devices.Devices {
-		ids[i] = k
-		i++
-	}
-	return ids
-}
-
-func (devices *DeviceSet) deviceStatus(devName string) (sizeInSectors, mappedSectors, highestMappedSector uint64, err error) {
-	var params string
-	_, sizeInSectors, _, params, err = getStatus(devName)
-	if err != nil {
-		return
-	}
-	if _, err = fmt.Sscanf(params, "%d %d", &mappedSectors, &highestMappedSector); err == nil {
-		return
-	}
-	return
-}
-
-func (devices *DeviceSet) GetDeviceStatus(hash string) (*DevStatus, error) {
-	devices.Lock()
-	defer devices.Unlock()
-
-	info := devices.Devices[hash]
-	if info == nil {
-		return nil, fmt.Errorf("No device %s", hash)
-	}
-
-	status := &DevStatus{
-		DeviceId:      info.DeviceId,
-		Size:          info.Size,
-		TransactionId: info.TransactionId,
-	}
-
-	if err := devices.activateDeviceIfNeeded(hash); err != nil {
-		return nil, fmt.Errorf("Error activating devmapper device for '%s': %s", hash, err)
-	}
-
-	if sizeInSectors, mappedSectors, highestMappedSector, err := devices.deviceStatus(info.DevName()); err != nil {
-		return nil, err
-	} else {
-		status.SizeInSectors = sizeInSectors
-		status.MappedSectors = mappedSectors
-		status.HighestMappedSector = highestMappedSector
-	}
-
-	return status, nil
-}
-
-func (devices *DeviceSet) poolStatus() (totalSizeInSectors, transactionId, dataUsed, dataTotal, metadataUsed, metadataTotal uint64, err error) {
-	var params string
-	if _, totalSizeInSectors, _, params, err = getStatus(devices.getPoolName()); err == nil {
-		_, err = fmt.Sscanf(params, "%d %d/%d %d/%d", &transactionId, &metadataUsed, &metadataTotal, &dataUsed, &dataTotal)
-	}
-	return
-}
-
 func (devices *DeviceSet) Status() *Status {
 	devices.Lock()
 	defer devices.Unlock()
 
 	status := &Status{}
 
+	if err := devices.ensureInit(); err != nil {
+		return status
+	}
+
 	status.PoolName = devices.getPoolName()
 	status.DataLoopback = path.Join(devices.loopbackDir(), "data")
 	status.MetadataLoopback = path.Join(devices.loopbackDir(), "metadata")
 
-	totalSizeInSectors, _, dataUsed, dataTotal, metadataUsed, metadataTotal, err := devices.poolStatus()
+	_, totalSizeInSectors, _, params, err := getStatus(devices.getPoolName())
 	if err == nil {
-		// Convert from blocks to bytes
-		blockSizeInSectors := totalSizeInSectors / dataTotal
+		var transactionId, dataUsed, dataTotal, metadataUsed, metadataTotal uint64
+		if _, err := fmt.Sscanf(params, "%d %d/%d %d/%d", &transactionId, &metadataUsed, &metadataTotal, &dataUsed, &dataTotal); err == nil {
+			// Convert from blocks to bytes
+			blockSizeInSectors := totalSizeInSectors / dataTotal
 
-		status.Data.Used = dataUsed * blockSizeInSectors * 512
-		status.Data.Total = dataTotal * blockSizeInSectors * 512
+			status.Data.Used = dataUsed * blockSizeInSectors * 512
+			status.Data.Total = dataTotal * blockSizeInSectors * 512
 
-		// metadata blocks are always 4k
-		status.Metadata.Used = metadataUsed * 4096
-		status.Metadata.Total = metadataTotal * 4096
-
-		status.SectorSize = blockSizeInSectors * 512
+			// metadata blocks are always 4k
+			status.Metadata.Used = metadataUsed * 4096
+			status.Metadata.Total = metadataTotal * 4096
+		}
 	}
 
 	return status
 }
 
-func NewDeviceSet(root string, doInit bool) (*DeviceSet, error) {
+func (devices *DeviceSet) ensureInit() error {
+	if !devices.initialized {
+		devices.initialized = true
+		if err := devices.initDevmapper(); err != nil {
+			utils.Debugf("\n--->Err: %s\n", err)
+			return err
+		}
+	}
+	return nil
+}
+
+func NewDeviceSet(root string) *DeviceSet {
 	SetDevDir("/dev")
 
-	devices := &DeviceSet{
+	return &DeviceSet{
+		initialized:  false,
 		root:         root,
 		MetaData:     MetaData{Devices: make(map[string]*DevInfo)},
 		activeMounts: make(map[string]int),
 	}
-
-	if err := devices.initDevmapper(doInit); err != nil {
-		return nil, err
-	}
-
-	return devices, nil
 }

devmapper/devmapper.go

@@ -0,0 +1,709 @@
+package devmapper
+
+/*
+#cgo LDFLAGS: -L. -ldevmapper
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <libdevmapper.h>
+#include <linux/loop.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <linux/fs.h>
+#include <errno.h>
+
+#ifndef LOOP_CTL_GET_FREE
+#define LOOP_CTL_GET_FREE       0x4C82
+#endif
+
+// FIXME: this could easily be rewritten in go
+char*			attach_loop_device(const char *filename, int *loop_fd_out)
+{
+  struct loop_info64	loopinfo = {0};
+  struct stat		st;
+  char			buf[64];
+  int			i, loop_fd, fd, start_index;
+  char*			loopname;
+
+
+  *loop_fd_out = -1;
+
+  start_index = 0;
+  fd = open("/dev/loop-control", O_RDONLY);
+  if (fd >= 0) {
+    start_index = ioctl(fd, LOOP_CTL_GET_FREE);
+    close(fd);
+
+    if (start_index < 0)
+      start_index = 0;
+  }
+
+  fd = open(filename, O_RDWR);
+  if (fd < 0) {
+    perror("open");
+    return NULL;
+  }
+
+  loop_fd = -1;
+  for (i = start_index ; loop_fd < 0 ; i++ ) {
+    if (sprintf(buf, "/dev/loop%d", i) < 0) {
+	close(fd);
+	return NULL;
+    }
+
+    if (stat(buf, &st)) {
+      if (!S_ISBLK(st.st_mode)) {
+	 fprintf(stderr, "[error] Loopback device %s is not a block device.\n", buf);
+      } else if (errno == ENOENT) {
+	fprintf(stderr, "[error] There are no more loopback device available.\n");
+      } else {
+	fprintf(stderr, "[error] Unkown error trying to stat the loopback device %s (errno: %d).\n", buf, errno);
+      }
+      close(fd);
+      return NULL;
+    }
+
+    loop_fd = open(buf, O_RDWR);
+    if (loop_fd < 0 && errno == ENOENT) {
+      fprintf(stderr, "[error] The loopback device %s does not exists.\n", buf);
+      close(fd);
+      return NULL;
+    } else if (loop_fd < 0) {
+	fprintf(stderr, "[error] Unkown error openning the loopback device %s. (errno: %d)\n", buf, errno);
+	continue;
+    }
+
+    if (ioctl(loop_fd, LOOP_SET_FD, (void *)(size_t)fd) < 0) {
+      int errsv = errno;
+      close(loop_fd);
+      loop_fd = -1;
+      if (errsv != EBUSY) {
+        close(fd);
+        fprintf(stderr, "cannot set up loopback device %s: %s", buf, strerror(errsv));
+        return NULL;
+      }
+      continue;
+    }
+
+    close(fd);
+
+    strncpy((char*)loopinfo.lo_file_name, buf, LO_NAME_SIZE);
+    loopinfo.lo_offset = 0;
+    loopinfo.lo_flags = LO_FLAGS_AUTOCLEAR;
+
+    if (ioctl(loop_fd, LOOP_SET_STATUS64, &loopinfo) < 0) {
+      perror("ioctl LOOP_SET_STATUS64");
+      if (ioctl(loop_fd, LOOP_CLR_FD, 0) < 0) {
+        perror("ioctl LOOP_CLR_FD");
+      }
+      close(loop_fd);
+      fprintf (stderr, "cannot set up loopback device info");
+      return (NULL);
+    }
+
+    loopname = strdup(buf);
+    if (loopname == NULL) {
+      close(loop_fd);
+      return (NULL);
+    }
+
+    *loop_fd_out = loop_fd;
+    return (loopname);
+  }
+
+  return (NULL);
+}
+
+static int64_t	get_block_size(int fd)
+{
+  uint64_t	size;
+
+  if (ioctl(fd, BLKGETSIZE64, &size) == -1)
+    return -1;
+  return ((int64_t)size);
+}
+
+extern void DevmapperLogCallback(int level, char *file, int line, int dm_errno_or_class, char *str);
+
+static void
+log_cb(int level, const char *file, int line,
+       int dm_errno_or_class, const char *f, ...)
+{
+  char buffer[256];
+  va_list ap;
+
+  va_start(ap, f);
+  vsnprintf(buffer, 256, f, ap);
+  va_end(ap);
+
+  DevmapperLogCallback(level, (char *)file, line, dm_errno_or_class, buffer);
+}
+
+static void
+log_with_errno_init ()
+{
+  dm_log_with_errno_init(log_cb);
+}
+
+*/
+import "C"
+
+import (
+	"errors"
+	"fmt"
+	"github.com/dotcloud/docker/utils"
+	"os"
+	"runtime"
+	"syscall"
+	"unsafe"
+)
+
+type DevmapperLogger interface {
+	log(level int, file string, line int, dmError int, message string)
+}
+
+const (
+	DeviceCreate TaskType = iota
+	DeviceReload
+	DeviceRemove
+	DeviceRemoveAll
+	DeviceSuspend
+	DeviceResume
+	DeviceInfo
+	DeviceDeps
+	DeviceRename
+	DeviceVersion
+	DeviceStatus
+	DeviceTable
+	DeviceWaitevent
+	DeviceList
+	DeviceClear
+	DeviceMknodes
+	DeviceListVersions
+	DeviceTargetMsg
+	DeviceSetGeometry
+)
+
+const (
+	AddNodeOnResume AddNodeType = iota
+	AddNodeOnCreate
+)
+
+var (
+	ErrTaskRun              = errors.New("dm_task_run failed")
+	ErrTaskSetName          = errors.New("dm_task_set_name failed")
+	ErrTaskSetMessage       = errors.New("dm_task_set_message failed")
+	ErrTaskSetAddNode       = errors.New("dm_task_set_add_node failed")
+	ErrTaskSetRO            = errors.New("dm_task_set_ro failed")
+	ErrTaskAddTarget        = errors.New("dm_task_add_target failed")
+	ErrGetDriverVersion     = errors.New("dm_task_get_driver_version failed")
+	ErrAttachLoopbackDevice = errors.New("loopback mounting failed")
+	ErrGetBlockSize         = errors.New("Can't get block size")
+	ErrUdevWait             = errors.New("wait on udev cookie failed")
+	ErrSetDevDir            = errors.New("dm_set_dev_dir failed")
+	ErrGetLibraryVersion    = errors.New("dm_get_library_version failed")
+	ErrCreateRemoveTask     = errors.New("Can't create task of type DeviceRemove")
+	ErrRunRemoveDevice      = errors.New("running removeDevice failed")
+)
+
+type (
+	Task struct {
+		unmanaged *C.struct_dm_task
+	}
+	Info struct {
+		Exists        int
+		Suspended     int
+		LiveTable     int
+		InactiveTable int
+		OpenCount     int32
+		EventNr       uint32
+		Major         uint32
+		Minor         uint32
+		ReadOnly      int
+		TargetCount   int32
+	}
+	TaskType    int
+	AddNodeType int
+)
+
+func (t *Task) destroy() {
+	if t != nil {
+		C.dm_task_destroy(t.unmanaged)
+		runtime.SetFinalizer(t, nil)
+	}
+}
+
+func TaskCreate(tasktype TaskType) *Task {
+	c_task := C.dm_task_create(C.int(tasktype))
+	if c_task == nil {
+		return nil
+	}
+	task := &Task{unmanaged: c_task}
+	runtime.SetFinalizer(task, (*Task).destroy)
+	return task
+}
+
+func (t *Task) Run() error {
+	if res := C.dm_task_run(t.unmanaged); res != 1 {
+		return ErrTaskRun
+	}
+	return nil
+}
+
+func (t *Task) SetName(name string) error {
+	c_name := C.CString(name)
+	defer free(c_name)
+
+	if res := C.dm_task_set_name(t.unmanaged, c_name); res != 1 {
+		return ErrTaskSetName
+	}
+	return nil
+}
+
+func (t *Task) SetMessage(message string) error {
+	c_message := C.CString(message)
+	defer free(c_message)
+
+	if res := C.dm_task_set_message(t.unmanaged, c_message); res != 1 {
+		return ErrTaskSetMessage
+	}
+	return nil
+}
+
+func (t *Task) SetSector(sector uint64) error {
+	if res := C.dm_task_set_sector(t.unmanaged, C.uint64_t(sector)); res != 1 {
+		return ErrTaskSetAddNode
+	}
+	return nil
+}
+
+func (t *Task) SetCookie(cookie *uint32, flags uint16) error {
+	c_cookie := C.uint32_t(*cookie)
+	if res := C.dm_task_set_cookie(t.unmanaged, &c_cookie, C.uint16_t(flags)); res != 1 {
+		return ErrTaskSetAddNode
+	}
+	*cookie = uint32(c_cookie)
+	return nil
+}
+
+func (t *Task) SetAddNode(add_node AddNodeType) error {
+	if res := C.dm_task_set_add_node(t.unmanaged, C.dm_add_node_t(add_node)); res != 1 {
+		return ErrTaskSetAddNode
+	}
+	return nil
+}
+
+func (t *Task) SetRo() error {
+	if res := C.dm_task_set_ro(t.unmanaged); res != 1 {
+		return ErrTaskSetRO
+	}
+	return nil
+}
+
+func (t *Task) AddTarget(start uint64, size uint64, ttype string, params string) error {
+	c_ttype := C.CString(ttype)
+	defer free(c_ttype)
+
+	c_params := C.CString(params)
+	defer free(c_params)
+
+	if res := C.dm_task_add_target(t.unmanaged, C.uint64_t(start), C.uint64_t(size), c_ttype, c_params); res != 1 {
+		return ErrTaskAddTarget
+	}
+	return nil
+}
+
+func (t *Task) GetDriverVersion() (string, error) {
+	buffer := C.CString(string(make([]byte, 128)))
+	defer free(buffer)
+
+	if res := C.dm_task_get_driver_version(t.unmanaged, buffer, 128); res != 1 {
+		return "", ErrGetDriverVersion
+	}
+	return C.GoString(buffer), nil
+}
+
+func (t *Task) GetInfo() (*Info, error) {
+	c_info := C.struct_dm_info{}
+	if res := C.dm_task_get_info(t.unmanaged, &c_info); res != 1 {
+		return nil, ErrGetDriverVersion
+	}
+	return &Info{
+		Exists:        int(c_info.exists),
+		Suspended:     int(c_info.suspended),
+		LiveTable:     int(c_info.live_table),
+		InactiveTable: int(c_info.inactive_table),
+		OpenCount:     int32(c_info.open_count),
+		EventNr:       uint32(c_info.event_nr),
+		Major:         uint32(c_info.major),
+		Minor:         uint32(c_info.minor),
+		ReadOnly:      int(c_info.read_only),
+		TargetCount:   int32(c_info.target_count),
+	}, nil
+}
+
+func (t *Task) GetNextTarget(next uintptr) (uintptr, uint64, uint64, string, string) {
+	var (
+		c_start, c_length       C.uint64_t
+		c_target_type, c_params *C.char
+	)
+
+	nextp := C.dm_get_next_target(t.unmanaged, unsafe.Pointer(next), &c_start, &c_length, &c_target_type, &c_params)
+	return uintptr(nextp), uint64(c_start), uint64(c_length), C.GoString(c_target_type), C.GoString(c_params)
+}
+
+func AttachLoopDevice(filename string) (*os.File, error) {
+	c_filename := C.CString(filename)
+	defer free(c_filename)
+
+	var fd C.int
+	res := C.attach_loop_device(c_filename, &fd)
+	if res == nil {
+		if os.Getenv("DEBUG") != "" {
+			C.perror(C.CString(fmt.Sprintf("[debug] Error attach_loop_device(%s, %d)", filename, int(fd))))
+		}
+		return nil, ErrAttachLoopbackDevice
+	}
+	defer free(res)
+
+	return os.NewFile(uintptr(fd), C.GoString(res)), nil
+}
+
+func getBlockSize(fd uintptr) int {
+	var size uint64
+
+	if _, _, err := syscall.Syscall(syscall.SYS_IOCTL, fd, C.BLKGETSIZE64, uintptr(unsafe.Pointer(&size))); err != 0 {
+		utils.Debugf("Error ioctl: %s", err)
+		return -1
+	}
+	return int(size)
+}
+
+func GetBlockDeviceSize(file *os.File) (uint64, error) {
+	if size := C.get_block_size(C.int(file.Fd())); size == -1 {
+		return 0, ErrGetBlockSize
+	} else {
+		return uint64(size), nil
+	}
+}
+
+func UdevWait(cookie uint32) error {
+	if res := C.dm_udev_wait(C.uint32_t(cookie)); res != 1 {
+		utils.Debugf("Failed to wait on udev cookie %d", cookie)
+		return ErrUdevWait
+	}
+	return nil
+}
+
+func LogInitVerbose(level int) {
+	C.dm_log_init_verbose(C.int(level))
+}
+
+var dmLogger DevmapperLogger = nil
+
+func logInit(logger DevmapperLogger) {
+	dmLogger = logger
+	C.log_with_errno_init()
+}
+
+func SetDevDir(dir string) error {
+	c_dir := C.CString(dir)
+	defer free(c_dir)
+
+	if res := C.dm_set_dev_dir(c_dir); res != 1 {
+		utils.Debugf("Error dm_set_dev_dir")
+		return ErrSetDevDir
+	}
+	return nil
+}
+
+func GetLibraryVersion() (string, error) {
+	buffer := C.CString(string(make([]byte, 128)))
+	defer free(buffer)
+
+	if res := C.dm_get_library_version(buffer, 128); res != 1 {
+		return "", ErrGetLibraryVersion
+	}
+	return C.GoString(buffer), nil
+}
+
+// Useful helper for cleanup
+func RemoveDevice(name string) error {
+	task := TaskCreate(DeviceRemove)
+	if task == nil {
+		return ErrCreateRemoveTask
+	}
+	if err := task.SetName(name); err != nil {
+		utils.Debugf("Can't set task name %s", name)
+		return err
+	}
+	if err := task.Run(); err != nil {
+		return ErrRunRemoveDevice
+	}
+	return nil
+}
+
+func free(p *C.char) {
+	C.free(unsafe.Pointer(p))
+}
+
+// This is the programmatic example of "dmsetup create"
+func createPool(poolName string, dataFile *os.File, metadataFile *os.File) error {
+	task, err := createTask(DeviceCreate, poolName)
+	if task == nil {
+		return err
+	}
+
+	size, err := GetBlockDeviceSize(dataFile)
+	if err != nil {
+		return fmt.Errorf("Can't get data size")
+	}
+
+	params := metadataFile.Name() + " " + dataFile.Name() + " 128 32768"
+	if err := task.AddTarget(0, size/512, "thin-pool", params); err != nil {
+		return fmt.Errorf("Can't add target")
+	}
+
+	var cookie uint32 = 0
+	if err := task.SetCookie(&cookie, 0); err != nil {
+		return fmt.Errorf("Can't set cookie")
+	}
+
+	if err := task.Run(); err != nil {
+		return fmt.Errorf("Error running DeviceCreate")
+	}
+
+	UdevWait(cookie)
+
+	return nil
+}
+
+func createTask(t TaskType, name string) (*Task, error) {
+	task := TaskCreate(t)
+	if task == nil {
+		return nil, fmt.Errorf("Can't create task of type %d", int(t))
+	}
+	if err := task.SetName(name); err != nil {
+		return nil, fmt.Errorf("Can't set task name %s", name)
+	}
+	return task, nil
+}
+
+func getInfo(name string) (*Info, error) {
+	task, err := createTask(DeviceInfo, name)
+	if task == nil {
+		return nil, err
+	}
+	if err := task.Run(); err != nil {
+		return nil, err
+	}
+	return task.GetInfo()
+}
+
+func getStatus(name string) (uint64, uint64, string, string, error) {
+	task, err := createTask(DeviceStatus, name)
+	if task == nil {
+		utils.Debugf("getStatus: Error createTask: %s", err)
+		return 0, 0, "", "", err
+	}
+	if err := task.Run(); err != nil {
+		utils.Debugf("getStatus: Error Run: %s", err)
+		return 0, 0, "", "", err
+	}
+
+	devinfo, err := task.GetInfo()
+	if err != nil {
+		utils.Debugf("getStatus: Error GetInfo: %s", err)
+		return 0, 0, "", "", err
+	}
+	if devinfo.Exists == 0 {
+		utils.Debugf("getStatus: Non existing device %s", name)
+		return 0, 0, "", "", fmt.Errorf("Non existing device %s", name)
+	}
+
+	_, start, length, target_type, params := task.GetNextTarget(0)
+	return start, length, target_type, params, nil
+}
+
+func setTransactionId(poolName string, oldId uint64, newId uint64) error {
+	task, err := createTask(DeviceTargetMsg, poolName)
+	if task == nil {
+		return err
+	}
+
+	if err := task.SetSector(0); err != nil {
+		return fmt.Errorf("Can't set sector")
+	}
+
+	if err := task.SetMessage(fmt.Sprintf("set_transaction_id %d %d", oldId, newId)); err != nil {
+		return fmt.Errorf("Can't set message")
+	}
+
+	if err := task.Run(); err != nil {
+		return fmt.Errorf("Error running setTransactionId")
+	}
+	return nil
+}
+
+func suspendDevice(name string) error {
+	task, err := createTask(DeviceSuspend, name)
+	if task == nil {
+		return err
+	}
+	if err := task.Run(); err != nil {
+		return fmt.Errorf("Error running DeviceSuspend")
+	}
+	return nil
+}
+
+func resumeDevice(name string) error {
+	task, err := createTask(DeviceResume, name)
+	if task == nil {
+		return err
+	}
+
+	var cookie uint32 = 0
+	if err := task.SetCookie(&cookie, 0); err != nil {
+		return fmt.Errorf("Can't set cookie")
+	}
+
+	if err := task.Run(); err != nil {
+		return fmt.Errorf("Error running DeviceSuspend")
+	}
+
+	UdevWait(cookie)
+
+	return nil
+}
+
+func createDevice(poolName string, deviceId int) error {
+	utils.Debugf("[devmapper] createDevice(poolName=%v, deviceId=%v)", poolName, deviceId)
+	task, err := createTask(DeviceTargetMsg, poolName)
+	if task == nil {
+		return err
+	}
+
+	if err := task.SetSector(0); err != nil {
+		return fmt.Errorf("Can't set sector")
+	}
+
+	if err := task.SetMessage(fmt.Sprintf("create_thin %d", deviceId)); err != nil {
+		return fmt.Errorf("Can't set message")
+	}
+
+	if err := task.Run(); err != nil {
+		return fmt.Errorf("Error running createDevice")
+	}
+	return nil
+}
+
+func deleteDevice(poolName string, deviceId int) error {
+	task, err := createTask(DeviceTargetMsg, poolName)
+	if task == nil {
+		return err
+	}
+
+	if err := task.SetSector(0); err != nil {
+		return fmt.Errorf("Can't set sector")
+	}
+
+	if err := task.SetMessage(fmt.Sprintf("delete %d", deviceId)); err != nil {
+		return fmt.Errorf("Can't set message")
+	}
+
+	if err := task.Run(); err != nil {
+		return fmt.Errorf("Error running deleteDevice")
+	}
+	return nil
+}
+
+func removeDevice(name string) error {
+	utils.Debugf("[devmapper] removeDevice START")
+	defer utils.Debugf("[devmapper] removeDevice END")
+	task, err := createTask(DeviceRemove, name)
+	if task == nil {
+		return err
+	}
+	if err = task.Run(); err != nil {
+		return fmt.Errorf("Error running removeDevice")
+	}
+	return nil
+}
+
+func activateDevice(poolName string, name string, deviceId int, size uint64) error {
+	task, err := createTask(DeviceCreate, name)
+	if task == nil {
+		return err
+	}
+
+	params := fmt.Sprintf("%s %d", poolName, deviceId)
+	if err := task.AddTarget(0, size/512, "thin", params); err != nil {
+		return fmt.Errorf("Can't add target")
+	}
+	if err := task.SetAddNode(AddNodeOnCreate); err != nil {
+		return fmt.Errorf("Can't add node")
+	}
+
+	var cookie uint32 = 0
+	if err := task.SetCookie(&cookie, 0); err != nil {
+		return fmt.Errorf("Can't set cookie")
+	}
+
+	if err := task.Run(); err != nil {
+		return fmt.Errorf("Error running DeviceCreate")
+	}
+
+	UdevWait(cookie)
+
+	return nil
+}
+
+func (devices *DeviceSet) createSnapDevice(poolName string, deviceId int, baseName string, baseDeviceId int) error {
+	devinfo, _ := getInfo(baseName)
+	doSuspend := devinfo != nil && devinfo.Exists != 0
+
+	if doSuspend {
+		if err := suspendDevice(baseName); err != nil {
+			return err
+		}
+	}
+
+	task, err := createTask(DeviceTargetMsg, poolName)
+	if task == nil {
+		if doSuspend {
+			resumeDevice(baseName)
+		}
+		return err
+	}
+
+	if err := task.SetSector(0); err != nil {
+		if doSuspend {
+			resumeDevice(baseName)
+		}
+		return fmt.Errorf("Can't set sector")
+	}
+
+	if err := task.SetMessage(fmt.Sprintf("create_snap %d %d", deviceId, baseDeviceId)); err != nil {
+		if doSuspend {
+			resumeDevice(baseName)
+		}
+		return fmt.Errorf("Can't set message")
+	}
+
+	if err := task.Run(); err != nil {
+		if doSuspend {
+			resumeDevice(baseName)
+		}
+		return fmt.Errorf("Error running DeviceCreate")
+	}
+
+	if doSuspend {
+		if err := resumeDevice(baseName); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}

devmapper/devmapper_log.go

@@ -1,5 +1,3 @@
-// +build linux
-
 package devmapper
 
 import "C"

devmapper/docker-device-tool/device_tool.go

@@ -0,0 +1,62 @@
+package main
+
+import (
+	"fmt"
+	"github.com/dotcloud/docker/devmapper"
+	"os"
+)
+
+func usage() {
+	fmt.Printf("Usage: %s [snap new-id base-id] | [remove id] | [mount id mountpoint]\n", os.Args[0])
+	os.Exit(1)
+}
+
+func main() {
+	devices := devmapper.NewDeviceSet("/var/lib/docker")
+
+	if len(os.Args) < 2 {
+		usage()
+	}
+
+	cmd := os.Args[1]
+	if cmd == "snap" {
+		if len(os.Args) < 4 {
+			usage()
+		}
+
+		err := devices.AddDevice(os.Args[2], os.Args[3])
+		if err != nil {
+			fmt.Println("Can't create snap device: ", err)
+			os.Exit(1)
+		}
+	} else if cmd == "remove" {
+		if len(os.Args) < 3 {
+			usage()
+		}
+
+		err := devices.RemoveDevice(os.Args[2])
+		if err != nil {
+			fmt.Println("Can't remove device: ", err)
+			os.Exit(1)
+		}
+	} else if cmd == "mount" {
+		if len(os.Args) < 4 {
+			usage()
+		}
+
+		err := devices.MountDevice(os.Args[2], os.Args[3])
+		if err != nil {
+			fmt.Println("Can't create snap device: ", err)
+			os.Exit(1)
+		}
+	} else {
+		fmt.Printf("Unknown command %s\n", cmd)
+		if len(os.Args) < 4 {
+			usage()
+		}
+
+		os.Exit(1)
+	}
+
+	return
+}

docker/docker.go

@@ -4,12 +4,16 @@ import (
 	"flag"
 	"fmt"
 	"github.com/dotcloud/docker"
-	"github.com/dotcloud/docker/engine"
 	"github.com/dotcloud/docker/sysinit"
 	"github.com/dotcloud/docker/utils"
+	"io/ioutil"
 	"log"
+	"net"
 	"os"
+	"os/signal"
+	"strconv"
 	"strings"
+	"syscall"
 )
 
 var (
@@ -23,26 +27,21 @@ func main() {
 		sysinit.SysInit()
 		return
 	}
-
-	var (
-		flVersion            = flag.Bool("v", false, "Print version information and quit")
-		flDaemon             = flag.Bool("d", false, "Enable daemon mode")
-		flDebug              = flag.Bool("D", false, "Enable debug mode")
-		flAutoRestart        = flag.Bool("r", true, "Restart previously running containers")
-		bridgeName           = flag.String("b", "", "Attach containers to a pre-existing network bridge; use 'none' to disable container networking")
-		bridgeIp             = flag.String("bip", "", "Use this CIDR notation address for the network bridge's IP, not compatible with -b")
-		pidfile              = flag.String("p", "/var/run/docker.pid", "Path to use for daemon PID file")
-		flRoot               = flag.String("g", "/var/lib/docker", "Path to use as the root of the docker runtime")
-		flEnableCors         = flag.Bool("api-enable-cors", false, "Enable CORS headers in the remote API")
-		flDns                = docker.NewListOpts(docker.ValidateIp4Address)
-		flEnableIptables     = flag.Bool("iptables", true, "Disable docker's addition of iptables rules")
-		flDefaultIp          = flag.String("ip", "0.0.0.0", "Default IP address to use when binding container ports")
-		flInterContainerComm = flag.Bool("icc", true, "Enable inter-container communication")
-		flGraphDriver        = flag.String("s", "", "Force the docker runtime to use a specific storage driver")
-		flHosts              = docker.NewListOpts(docker.ValidateHost)
-	)
-	flag.Var(&flDns, "dns", "Force docker to use specific DNS servers")
-	flag.Var(&flHosts, "H", "Multiple tcp://host:port or unix://path/to/socket to bind in daemon mode, single connection otherwise")
+	// FIXME: Switch d and D ? (to be more sshd like)
+	flVersion := flag.Bool("v", false, "Print version information and quit")
+	flDaemon := flag.Bool("d", false, "Daemon mode")
+	flDebug := flag.Bool("D", false, "Debug mode")
+	flAutoRestart := flag.Bool("r", true, "Restart previously running containers")
+	bridgeName := flag.String("b", "", "Attach containers to a pre-existing network bridge. Use 'none' to disable container networking")
+	pidfile := flag.String("p", "/var/run/docker.pid", "File containing process PID")
+	flGraphPath := flag.String("g", "/var/lib/docker", "Path to graph storage base dir.")
+	flEnableCors := flag.Bool("api-enable-cors", false, "Enable CORS requests in the remote api.")
+	flDns := flag.String("dns", "", "Set custom dns servers")
+	flHosts := utils.ListOpts{fmt.Sprintf("unix://%s", docker.DEFAULTUNIXSOCKET)}
+	flag.Var(&flHosts, "H", "tcp://host:port to bind/connect to or unix://path/to/socket to use")
+	flEnableIptables := flag.Bool("iptables", true, "Disable iptables within docker")
+	flDefaultIp := flag.String("ip", "0.0.0.0", "Default ip address to use when binding a containers ports")
+	flInterContainerComm := flag.Bool("enable-container-comm", false, "Enable inter-container communication")
 
 	flag.Parse()
 
@@ -50,15 +49,17 @@ func main() {
 		showVersion()
 		return
 	}
-	if flHosts.Len() == 0 {
-		// If we do not have a host, default to unix socket
-		flHosts.Set(fmt.Sprintf("unix://%s", docker.DEFAULTUNIXSOCKET))
+	if len(flHosts) > 1 {
+		flHosts = flHosts[1:] //trick to display a nice default value in the usage
+	}
+	for i, flHost := range flHosts {
+		flHosts[i] = utils.ParseHost(docker.DEFAULTHTTPHOST, docker.DEFAULTHTTPPORT, flHost)
 	}
 
-	if *bridgeName != "" && *bridgeIp != "" {
-		log.Fatal("You specified -b & -bip, mutually exclusive options. Please specify only one.")
+	bridge := docker.DefaultNetworkBridge
+	if *bridgeName != "" {
+		bridge = *bridgeName
 	}
-
 	if *flDebug {
 		os.Setenv("DEBUG", "1")
 	}
@@ -69,45 +70,41 @@ func main() {
 			flag.Usage()
 			return
 		}
-		eng, err := engine.New(*flRoot)
-		if err != nil {
-			log.Fatal(err)
+		var dns []string
+		if *flDns != "" {
+			dns = []string{*flDns}
 		}
-		// Load plugin: httpapi
-		job := eng.Job("initapi")
-		job.Setenv("Pidfile", *pidfile)
-		job.Setenv("Root", *flRoot)
-		job.SetenvBool("AutoRestart", *flAutoRestart)
-		job.SetenvBool("EnableCors", *flEnableCors)
-		job.SetenvList("Dns", flDns.GetAll())
-		job.SetenvBool("EnableIptables", *flEnableIptables)
-		job.Setenv("BridgeIface", *bridgeName)
-		job.Setenv("BridgeIp", *bridgeIp)
-		job.Setenv("DefaultIp", *flDefaultIp)
-		job.SetenvBool("InterContainerCommunication", *flInterContainerComm)
-		job.Setenv("GraphDriver", *flGraphDriver)
-		if err := job.Run(); err != nil {
-			log.Fatal(err)
+
+		ip := net.ParseIP(*flDefaultIp)
+
+		config := &docker.DaemonConfig{
+			Pidfile:                     *pidfile,
+			GraphPath:                   *flGraphPath,
+			AutoRestart:                 *flAutoRestart,
+			EnableCors:                  *flEnableCors,
+			Dns:                         dns,
+			EnableIptables:              *flEnableIptables,
+			BridgeIface:                 bridge,
+			ProtoAddresses:              flHosts,
+			DefaultIp:                   ip,
+			InterContainerCommunication: *flInterContainerComm,
 		}
-		// Serve api
-		job = eng.Job("serveapi", flHosts.GetAll()...)
-		job.SetenvBool("Logging", true)
-		if err := job.Run(); err != nil {
+		if err := daemon(config); err != nil {
 			log.Fatal(err)
+			os.Exit(-1)
 		}
 	} else {
-		if flHosts.Len() > 1 {
+		if len(flHosts) > 1 {
 			log.Fatal("Please specify only one -H")
+			return
 		}
-		protoAddrParts := strings.SplitN(flHosts.GetAll()[0], "://", 2)
+		protoAddrParts := strings.SplitN(flHosts[0], "://", 2)
 		if err := docker.ParseCommands(protoAddrParts[0], protoAddrParts[1], flag.Args()...); err != nil {
 			if sterr, ok := err.(*utils.StatusError); ok {
-				if sterr.Status != "" {
-					log.Println(sterr.Status)
-				}
-				os.Exit(sterr.StatusCode)
+				os.Exit(sterr.Status)
 			}
 			log.Fatal(err)
+			os.Exit(-1)
 		}
 	}
 }
@@ -115,3 +112,74 @@ func main() {
 func showVersion() {
 	fmt.Printf("Docker version %s, build %s\n", VERSION, GITCOMMIT)
 }
+
+func createPidFile(pidfile string) error {
+	if pidString, err := ioutil.ReadFile(pidfile); err == nil {
+		pid, err := strconv.Atoi(string(pidString))
+		if err == nil {
+			if _, err := os.Stat(fmt.Sprintf("/proc/%d/", pid)); err == nil {
+				return fmt.Errorf("pid file found, ensure docker is not running or delete %s", pidfile)
+			}
+		}
+	}
+
+	file, err := os.Create(pidfile)
+	if err != nil {
+		return err
+	}
+
+	defer file.Close()
+
+	_, err = fmt.Fprintf(file, "%d", os.Getpid())
+	return err
+}
+
+func removePidFile(pidfile string) {
+	if err := os.Remove(pidfile); err != nil {
+		log.Printf("Error removing %s: %s", pidfile, err)
+	}
+}
+
+func daemon(config *docker.DaemonConfig) error {
+	if err := createPidFile(config.Pidfile); err != nil {
+		log.Fatal(err)
+	}
+	defer removePidFile(config.Pidfile)
+
+	c := make(chan os.Signal, 1)
+	signal.Notify(c, os.Interrupt, os.Kill, syscall.SIGTERM)
+	go func() {
+		sig := <-c
+		log.Printf("Received signal '%v', exiting\n", sig)
+		removePidFile(config.Pidfile)
+		os.Exit(0)
+	}()
+	server, err := docker.NewServer(config)
+	if err != nil {
+		return err
+	}
+	chErrors := make(chan error, len(config.ProtoAddresses))
+	for _, protoAddr := range config.ProtoAddresses {
+		protoAddrParts := strings.SplitN(protoAddr, "://", 2)
+		if protoAddrParts[0] == "unix" {
+			syscall.Unlink(protoAddrParts[1])
+		} else if protoAddrParts[0] == "tcp" {
+			if !strings.HasPrefix(protoAddrParts[1], "127.0.0.1") {
+				log.Println("/!\\ DON'T BIND ON ANOTHER IP ADDRESS THAN 127.0.0.1 IF YOU DON'T KNOW WHAT YOU'RE DOING /!\\")
+			}
+		} else {
+			log.Fatal("Invalid protocol format.")
+			os.Exit(-1)
+		}
+		go func() {
+			chErrors <- docker.ListenAndServe(protoAddrParts[0], protoAddrParts[1], server, true)
+		}()
+	}
+	for i := 0; i < len(config.ProtoAddresses); i += 1 {
+		err := <-chErrors
+		if err != nil {
+			return err
+		}
+	}
+	return nil
+}

docs/Dockerfile

@@ -1,16 +1,11 @@
 from ubuntu:12.04
 maintainer Nick Stinemates
-#
-#    docker build -t docker:docs . && docker run -p 8000:8000 docker:docs
-#
 
 run apt-get update
 run apt-get install -y python-setuptools make
 run easy_install pip
-#from docs/requirements.txt, but here to increase cacheability
-run pip install Sphinx==1.1.3
-run pip install sphinxcontrib-httpdomain==1.1.9
 add . /docs
+run pip install -r /docs/requirements.txt
 run cd /docs; make docs
 
 expose 8000

docs/MAINTAINERS

@@ -1,4 +1,2 @@
 Andy Rothfusz <andy@dotcloud.com> (@metalivedev)
 Ken Cochrane <ken@dotcloud.com> (@kencochrane)
-James Turnbull <james@lovedthanlost.net> (@jamtur01)
-Sven Dowideit <SvenDowideit@fosiki.com> (@SvenDowideit)

docs/Makefile

@@ -31,7 +31,7 @@ help:
 #	@echo "  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
 #	@echo "  latexpdf   to make LaTeX files and run them through pdflatex"
 #	@echo "  text       to make text files"
-	@echo "  man        to make a manual page"
+#	@echo "  man        to make manual pages"
 #	@echo "  texinfo    to make Texinfo files"
 #	@echo "  info       to make Texinfo files and run them through makeinfo"
 #	@echo "  gettext    to make PO message catalogs"

docs/README.md

@@ -39,27 +39,25 @@ Getting Started
 To edit and test the docs, you'll need to install the Sphinx tool and
 its dependencies. There are two main ways to install this tool:
 
-###Native Installation
+Native Installation
+...................
 
-Install dependencies from `requirements.txt` file in your `docker/docs`
-directory:
+* Install sphinx: `pip install sphinx`
+    * Mac OS X: `[sudo] pip-2.7 install sphinx`
+* Install sphinx httpdomain contrib package: `pip install sphinxcontrib-httpdomain`
+    * Mac OS X: `[sudo] pip-2.7 install sphinxcontrib-httpdomain`
+* If pip is not available you can probably install it using your favorite package manager as **python-pip**
 
-* Linux: `pip install -r docs/requirements.txt`
-
-* Mac OS X: `[sudo] pip-2.7 -r docs/requirements.txt`
-
-###Alternative Installation: Docker Container
+Alternative Installation: Docker Container
+..........................................
 
 If you're running ``docker`` on your development machine then you may
-find it easier and cleaner to use the docs Dockerfile. This installs Sphinx
+find it easier and cleaner to use the Dockerfile. This installs Sphinx
 in a container, adds the local ``docs/`` directory and builds the HTML
 docs inside the container, even starting a simple HTTP server on port
-8000 so that you can connect and see your changes.
-
-In the ``docker`` source directory, run:
-    ```make docs```
-
-This is the equivalent to ``make clean server`` since each container starts clean.
+8000 so that you can connect and see your changes. Just run ``docker
+build .`` and run the resulting image. This is the equivalent to
+``make clean server`` since each container starts clean.
 
 Usage
 -----
@@ -128,8 +126,7 @@ Guides on using sphinx
 
 * Code examples
 
-  * Start typed commands with ``$ `` (dollar space) so that they 
-    are easily differentiated from program output.
+  * Start without $, so it's easy to copy and paste.
   * Use "sudo" with docker to ensure that your command is runnable
     even if they haven't [used the *docker*
     group](http://docs.docker.io/en/latest/use/basics/#why-sudo).
@@ -138,7 +135,7 @@ Manpages
 --------
 
 * To make the manpages, run ``make man``. Please note there is a bug
-  in Sphinx 1.1.3 which makes this fail.  Upgrade to the latest version
+  in spinx 1.1.3 which makes this fail.  Upgrade to the latest version
   of Sphinx.
 * Then preview the manpage by running ``man _build/man/docker.1``,
   where ``_build/man/docker.1`` is the path to the generated manfile

docs/requirements.txt

@@ -1,2 +1,2 @@
 Sphinx==1.1.3
-sphinxcontrib-httpdomain==1.1.9
+sphinxcontrib-httpdomain==1.1.8

docs/sources/api/docker_remote_api.rst

@@ -26,147 +26,14 @@ Docker Remote API
 2. Versions
 ===========
 
-The current version of the API is 1.7
+The current version of the API is 1.6
 
 Calling /images/<name>/insert is the same as calling
-/v1.7/images/<name>/insert
+/v1.6/images/<name>/insert
 
 You can still call an old version of the api using
 /v1.0/images/<name>/insert
 
-
-v1.8
-****
-
-Full Documentation
-------------------
-
-:doc:`docker_remote_api_v1.8`
-
-What's new
-----------
-
-.. http:post:: /build
-
-   **New!** This endpoint now returns build status as json stream. In case
-   of a build error, it returns the exit status of the failed command.
-
-.. http:get:: /containers/(id)/json
-
-    **New!** This endpoint now returns the host config for the container.
-
-.. http:post:: /images/create
-.. http:post:: /images/(name)/insert
-.. http:post:: /images/(name)/push
-
-  **New!** progressDetail object was added in the JSON. It's now possible
-  to get the current value and the total of the progress without having to
-  parse the string.
-
-v1.7
-****
-
-Full Documentation
-------------------
-
-:doc:`docker_remote_api_v1.7`
-
-What's new
-----------
-
-.. http:get:: /images/json
-
-   The format of the json returned from this uri changed.  Instead of an entry
-   for each repo/tag on an image, each image is only represented once, with a
-   nested attribute indicating the repo/tags that apply to that image.
-
-   Instead of:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Content-Type: application/json
-
-      [
-        {
-          "VirtualSize": 131506275,
-          "Size": 131506275,
-          "Created": 1365714795,
-          "Id": "8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c",
-          "Tag": "12.04",
-          "Repository": "ubuntu"
-        },
-        {
-          "VirtualSize": 131506275,
-          "Size": 131506275,
-          "Created": 1365714795,
-          "Id": "8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c",
-          "Tag": "latest",
-          "Repository": "ubuntu"
-        },
-        {
-          "VirtualSize": 131506275,
-          "Size": 131506275,
-          "Created": 1365714795,
-          "Id": "8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c",
-          "Tag": "precise",
-          "Repository": "ubuntu"
-        },
-        {
-          "VirtualSize": 180116135,
-          "Size": 24653,
-          "Created": 1364102658,
-          "Id": "b750fe79269d2ec9a3c593ef05b4332b1d1a02a62b4accb2c21d589ff2f5f2dc",
-          "Tag": "12.10",
-          "Repository": "ubuntu"
-        },
-        {
-          "VirtualSize": 180116135,
-          "Size": 24653,
-          "Created": 1364102658,
-          "Id": "b750fe79269d2ec9a3c593ef05b4332b1d1a02a62b4accb2c21d589ff2f5f2dc",
-          "Tag": "quantal",
-          "Repository": "ubuntu"
-        }
-      ]
-
-   The returned json looks like this:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Content-Type: application/json
-      
-      [
-        {
-           "RepoTag": [
-             "ubuntu:12.04",
-             "ubuntu:precise",
-             "ubuntu:latest"
-           ],
-           "Id": "8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c",
-           "Created": 1365714795,
-           "Size": 131506275,
-           "VirtualSize": 131506275
-        },
-        {
-           "RepoTag": [
-             "ubuntu:12.10",
-             "ubuntu:quantal"
-           ],
-           "ParentId": "27cf784147099545",
-           "Id": "b750fe79269d2ec9a3c593ef05b4332b1d1a02a62b4accb2c21d589ff2f5f2dc",
-           "Created": 1364102658,
-           "Size": 24653,
-           "VirtualSize": 180116135
-        }
-      ]
-
-.. http:get:: /images/viz
-
-   This URI no longer exists.  The ``images -viz`` output is now generated in
-   the client, using the ``/images/json`` data.
-
 v1.6
 ****
 

docs/sources/api/docker_remote_api_v1.0.rst

@@ -970,33 +970,28 @@ Create a new image from a container's changes
 
 	**Example request**:
 
-    .. sourcecode:: http
+        .. sourcecode:: http
 
-       POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
-       Content-Type: application/json
-   
-       {
-           "Cmd": ["cat", "/world"],
-           "PortSpecs":["22"]
-       }
+           POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
 
-    **Example response**:
+        **Example response**:
 
-    .. sourcecode:: http
+        .. sourcecode:: http
 
-       HTTP/1.1 201 OK
-       Content-Type: application/vnd.docker.raw-stream
+           HTTP/1.1 201 OK
+	   Content-Type: application/vnd.docker.raw-stream
 
-       {"Id":"596069db4bf5"}
+           {"Id":"596069db4bf5"}
 
-    :query container: source container
-    :query repo: repository
-    :query tag: tag
-    :query m: commit message
-    :query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
-    :statuscode 201: no error
-    :statuscode 404: no such container
-    :statuscode 500: server error
+	:query container: source container
+	:query repo: repository
+	:query tag: tag
+	:query m: commit message
+	:query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
+	:query run: config automatically applied when the image is run. (ex: {"Cmd": ["cat", "/world"], "PortSpecs":["22"]})
+        :statuscode 201: no error
+	:statuscode 404: no such container
+        :statuscode 500: server error
 
 
 3. Going further

docs/sources/api/docker_remote_api_v1.1.rst

@@ -977,37 +977,32 @@ Create a new image from a container's changes
 
 .. http:post:: /commit
 
-    Create a new image from a container's changes
+	Create a new image from a container's changes
 
-    **Example request**:
-    
-    .. sourcecode:: http
-    
-       POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
-       Content-Type: application/json
-   
-       {
-           "Cmd": ["cat", "/world"],
-           "PortSpecs":["22"]
-       }
-    
-    **Example response**:
-    
-    .. sourcecode:: http
-    
-        HTTP/1.1 201 OK
-        Content-Type: application/vnd.docker.raw-stream
+	**Example request**:
 
-        {"Id":"596069db4bf5"}
+        .. sourcecode:: http
 
-    :query container: source container
-    :query repo: repository
-    :query tag: tag
-    :query m: commit message
-    :query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
-    :statuscode 201: no error
-    :statuscode 404: no such container
-    :statuscode 500: server error
+           POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
+
+        **Example response**:
+
+        .. sourcecode:: http
+
+           HTTP/1.1 201 OK
+	   Content-Type: application/vnd.docker.raw-stream
+
+           {"Id":"596069db4bf5"}
+
+	:query container: source container
+	:query repo: repository
+	:query tag: tag
+	:query m: commit message
+	:query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
+	:query run: config automatically applied when the image is run. (ex: {"Cmd": ["cat", "/world"], "PortSpecs":["22"]})
+        :statuscode 201: no error
+	:statuscode 404: no such container
+        :statuscode 500: server error
 
 
 3. Going further

docs/sources/api/docker_remote_api_v1.2.rst

@@ -985,37 +985,32 @@ Create a new image from a container's changes
 
 .. http:post:: /commit
 
-    Create a new image from a container's changes
+	Create a new image from a container's changes
 
-    **Example request**:
+	**Example request**:
 
-    .. sourcecode:: http
+        .. sourcecode:: http
 
-       POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
-       Content-Type: application/json
-   
-       {
-           "Cmd": ["cat", "/world"],
-           "PortSpecs":["22"]
-       }
+           POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
 
-    **Example response**:
+        **Example response**:
 
-    .. sourcecode:: http
+        .. sourcecode:: http
 
-       HTTP/1.1 201 OK
+           HTTP/1.1 201 OK
 	   Content-Type: application/vnd.docker.raw-stream
 
-       {"Id":"596069db4bf5"}
+           {"Id":"596069db4bf5"}
 
-    :query container: source container
-    :query repo: repository
-    :query tag: tag
-    :query m: commit message
-    :query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
-    :statuscode 201: no error
-    :statuscode 404: no such container
-    :statuscode 500: server error
+	:query container: source container
+	:query repo: repository
+	:query tag: tag
+	:query m: commit message
+	:query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
+	:query run: config automatically applied when the image is run. (ex: {"Cmd": ["cat", "/world"], "PortSpecs":["22"]})
+        :statuscode 201: no error
+	:statuscode 404: no such container
+        :statuscode 500: server error
 
 
 3. Going further

docs/sources/api/docker_remote_api_v1.3.rst

@@ -1034,37 +1034,32 @@ Create a new image from a container's changes
 
 .. http:post:: /commit
 
-    Create a new image from a container's changes
+	Create a new image from a container's changes
 
-    **Example request**:
+	**Example request**:
 
-    .. sourcecode:: http
+        .. sourcecode:: http
 
-       POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
-       Content-Type: application/json
-   
-       {
-           "Cmd": ["cat", "/world"],
-           "PortSpecs":["22"]
-       }
+           POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
 
-    **Example response**:
+        **Example response**:
 
-    .. sourcecode:: http
+        .. sourcecode:: http
 
-       HTTP/1.1 201 OK
+           HTTP/1.1 201 OK
 	   Content-Type: application/vnd.docker.raw-stream
 
-       {"Id":"596069db4bf5"}
+           {"Id":"596069db4bf5"}
 
-    :query container: source container
-    :query repo: repository
-    :query tag: tag
-    :query m: commit message
-    :query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
-    :statuscode 201: no error
-    :statuscode 404: no such container
-    :statuscode 500: server error
+	:query container: source container
+	:query repo: repository
+	:query tag: tag
+	:query m: commit message
+	:query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
+	:query run: config automatically applied when the image is run. (ex: {"Cmd": ["cat", "/world"], "PortSpecs":["22"]})
+        :statuscode 201: no error
+	:statuscode 404: no such container
+        :statuscode 500: server error
 
 
 Monitor Docker's events

docs/sources/api/docker_remote_api_v1.4.rst

@@ -943,36 +943,36 @@ Build an image from Dockerfile via stdin
 
 .. http:post:: /build
 
-    Build an image from Dockerfile via stdin
+   Build an image from Dockerfile via stdin
 
-    **Example request**:
+   **Example request**:
 
-    .. sourcecode:: http
+   .. sourcecode:: http
 
-        POST /build HTTP/1.1
+      POST /build HTTP/1.1
 
-        {{ STREAM }}
+      {{ STREAM }}
 
-    **Example response**:
+   **Example response**:
 
-    .. sourcecode:: http
+   .. sourcecode:: http
 
-        HTTP/1.1 200 OK
+      HTTP/1.1 200 OK
 
-        {{ STREAM }}
+      {{ STREAM }}
 
 
-    The stream must be a tar archive compressed with one of the following algorithms:
-    identity (no compression), gzip, bzip2, xz. The archive must include a file called
-    `Dockerfile` at its root. It may include any number of other files, which will be
-    accessible in the build context (See the ADD build command).
+       The stream must be a tar archive compressed with one of the following algorithms:
+       identity (no compression), gzip, bzip2, xz. The archive must include a file called
+       `Dockerfile` at its root. It may include any number of other files, which will be
+       accessible in the build context (See the ADD build command).
 
-    The Content-type header should be set to "application/tar".
+       The Content-type header should be set to "application/tar".
 
-    :query t: repository name (and optionally a tag) to be applied to the resulting image in case of success
-    :query q: suppress verbose build output
+	:query t: repository name (and optionally a tag) to be applied to the resulting image in case of success
+	:query q: suppress verbose build output
     :query nocache: do not use the cache when building the image
-    :statuscode 200: no error
+	:statuscode 200: no error
     :statuscode 500: server error
 
 
@@ -1069,8 +1069,8 @@ Show the docker version information
 		"GoVersion":"go1.0.3"
 	   }
 
-    :statuscode 200: no error
-    :statuscode 500: server error
+        :statuscode 200: no error
+	:statuscode 500: server error
 
 
 Create a new image from a container's changes
@@ -1084,28 +1084,23 @@ Create a new image from a container's changes
 
     .. sourcecode:: http
 
-       POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
-       Content-Type: application/json
-   
-       {
-           "Cmd": ["cat", "/world"],
-           "PortSpecs":["22"]
-       }
+        POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
 
     **Example response**:
 
     .. sourcecode:: http
 
-       HTTP/1.1 201 OK
-	   Content-Type: application/vnd.docker.raw-stream
+        HTTP/1.1 201 OK
+	    Content-Type: application/vnd.docker.raw-stream
 
-       {"Id":"596069db4bf5"}
+        {"Id":"596069db4bf5"}
 
     :query container: source container
     :query repo: repository
     :query tag: tag
     :query m: commit message
     :query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
+    :query run: config automatically applied when the image is run. (ex: {"Cmd": ["cat", "/world"], "PortSpecs":["22"]})
     :statuscode 201: no error
     :statuscode 404: no such container
     :statuscode 500: server error

docs/sources/api/docker_remote_api_v1.5.rst

@@ -1050,37 +1050,32 @@ Create a new image from a container's changes
 
 .. http:post:: /commit
 
-    Create a new image from a container's changes
+  Create a new image from a container's changes
 
-    **Example request**:
+  **Example request**:
 
-    .. sourcecode:: http
+  .. sourcecode:: http
 
-       POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
-       Content-Type: application/json
-   
-       {
-           "Cmd": ["cat", "/world"],
-           "PortSpecs":["22"]
-       }
+    POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
 
-    **Example response**:
+  **Example response**:
 
-    .. sourcecode:: http
+  .. sourcecode:: http
 
-       HTTP/1.1 201 OK
-	   Content-Type: application/vnd.docker.raw-stream
+    HTTP/1.1 201 OK
+    Content-Type: application/vnd.docker.raw-stream
 
-       {"Id":"596069db4bf5"}
+    {"Id":"596069db4bf5"}
 
-    :query container: source container
-    :query repo: repository
-    :query tag: tag
-    :query m: commit message
-    :query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
-    :statuscode 201: no error
-    :statuscode 404: no such container
-    :statuscode 500: server error
+  :query container: source container
+  :query repo: repository
+  :query tag: tag
+  :query m: commit message
+  :query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
+  :query run: config automatically applied when the image is run. (ex: {"Cmd": ["cat", "/world"], "PortSpecs":["22"]})
+  :statuscode 201: no error
+  :statuscode 404: no such container
+  :statuscode 500: server error
 
 Monitor Docker's events
 ***********************

docs/sources/api/docker_remote_api_v1.6.rst

@@ -13,12 +13,9 @@ Docker Remote API v1.6
 1. Brief introduction
 =====================
 
-- The Remote API has replaced rcli
-- The daemon listens on ``unix:///var/run/docker.sock``, but you can
-  :ref:`bind_docker`.
-- The API tends to be REST, but for some complex commands, like
-  ``attach`` or ``pull``, the HTTP connection is hijacked to transport
-  ``stdout, stdin`` and ``stderr``
+- The Remote API is replacing rcli
+- Default port in the docker daemon is 4243
+- The API tends to be REST, but for some complex commands, like attach or pull, the HTTP connection is hijacked to transport stdout stdin and stderr
 
 2. Endpoints
 ============
@@ -121,7 +118,8 @@ Create a container
 		"AttachStdin":false,
 		"AttachStdout":true,
 		"AttachStderr":true,
-		"ExposedPorts":{},
+		"PortSpecs":null,
+		"Privileged": false,
 		"Tty":false,
 		"OpenStdin":false,
 		"StdinOnce":false,
@@ -134,6 +132,7 @@ Create a container
 		"Volumes":{},
 		"VolumesFrom":"",
 		"WorkingDir":""
+
 	   }
 	   
 	**Example response**:
@@ -149,63 +148,11 @@ Create a container
 	   }
 	
 	:jsonparam config: the container's configuration
- 	:query name: container name to use
 	:statuscode 201: no error
 	:statuscode 404: no such container
 	:statuscode 406: impossible to attach (container not running)
 	:statuscode 500: server error
 
-	**More Complex Example request, in 2 steps.**
-	**First, use create to expose a Private Port, which can be bound back to a Public Port at startup**:
-
-	.. sourcecode:: http
-
-	   POST /containers/create HTTP/1.1
-	   Content-Type: application/json
-
-	   {
-		"Cmd":[
-			"/usr/sbin/sshd","-D"
-		],
-		"Image":"image-with-sshd",
-		"ExposedPorts":{"22/tcp":{}}
-		}
-
-	**Example response**:
-
-	.. sourcecode:: http
-
-	   HTTP/1.1 201 OK
-	   Content-Type: application/json
-
-	   {
-		"Id":"e90e34656806"
-		"Warnings":[]
-	   }
-
-	**Second, start (using the ID returned above) the image we just created, mapping the ssh port 22 to something on the host**:
-
-	.. sourcecode:: http
-
-	   POST /containers/e90e34656806/start HTTP/1.1
-	   Content-Type: application/json
-
-	   {
-		"PortBindings": { "22/tcp": [{ "HostPort": "11022" }]} 
-		}
-
-	**Example response**:
-
-	.. sourcecode:: http
-
-		HTTP/1.1 204 No Content
-		Content-Type: text/plain; charset=utf-8
-		Content-Length: 0
-
-	**Now you can ssh into your new container on port 11022.**
-
-
-
 
 Inspect a container
 *******************
@@ -240,7 +187,7 @@ Inspect a container
 				"AttachStdin": false,
 				"AttachStdout": true,
 				"AttachStderr": true,
-				"ExposedPorts": {},
+				"PortSpecs": null,
 				"Tty": false,
 				"OpenStdin": false,
 				"StdinOnce": false,
@@ -411,12 +358,7 @@ Start a container
 
            {
                 "Binds":["/tmp:/tmp"],
-                "LxcConf":{"lxc.utsname":"docker"},
-                "ContainerIDFile": "",
-                "Privileged": false,
-                "PortBindings": {"22/tcp": [{HostIp:"", HostPort:""}]},
-                "Links": [],
-                "PublishAllPorts": false
+                "LxcConf":{"lxc.utsname":"docker"}
            }
 
         **Example response**:
@@ -500,8 +442,7 @@ Kill a container
 	.. sourcecode:: http
 
 	   HTTP/1.1 204 OK
-
-	:query signal: Signal to send to the container (integer). When not set, SIGKILL is assumed and the call will waits for the container to exit.
+	   	
 	:statuscode 204: no error
 	:statuscode 404: no such container
 	:statuscode 500: server error
@@ -849,7 +790,7 @@ Inspect an image
 				"AttachStdin":false,
 				"AttachStdout":false,
 				"AttachStderr":false,
-				"ExposedPorts":{},
+				"PortSpecs":null,
 				"Tty":true,
 				"OpenStdin":true,
 				"StdinOnce":false,
@@ -1049,36 +990,36 @@ Build an image from Dockerfile via stdin
 
 .. http:post:: /build
 
-    Build an image from Dockerfile via stdin
+   Build an image from Dockerfile via stdin
 
-    **Example request**:
+   **Example request**:
 
-    .. sourcecode:: http
+   .. sourcecode:: http
 
-        POST /build HTTP/1.1
+      POST /build HTTP/1.1
 
-        {{ STREAM }}
+      {{ STREAM }}
 
-    **Example response**:
+   **Example response**:
 
-    .. sourcecode:: http
+   .. sourcecode:: http
 
-        HTTP/1.1 200 OK
+      HTTP/1.1 200 OK
 
-        {{ STREAM }}
+      {{ STREAM }}
 
 
-    The stream must be a tar archive compressed with one of the following algorithms:
-    identity (no compression), gzip, bzip2, xz. The archive must include a file called
-    `Dockerfile` at its root. It may include any number of other files, which will be
-    accessible in the build context (See the ADD build command).
-    
-    The Content-type header should be set to "application/tar".
+       The stream must be a tar archive compressed with one of the following algorithms:
+       identity (no compression), gzip, bzip2, xz. The archive must include a file called
+       `Dockerfile` at its root. It may include any number of other files, which will be
+       accessible in the build context (See the ADD build command).
 
-    :query t: repository name (and optionally a tag) to be applied to the resulting image in case of success
-    :query q: suppress verbose build output
+       The Content-type header should be set to "application/tar".
+
+	:query t: repository name (and optionally a tag) to be applied to the resulting image in case of success
+	:query q: suppress verbose build output
     :query nocache: do not use the cache when building the image
-    :statuscode 200: no error
+	:statuscode 200: no error
     :statuscode 500: server error
 
 
@@ -1190,13 +1131,7 @@ Create a new image from a container's changes
 
     .. sourcecode:: http
 
-       POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
-       Content-Type: application/json
-       
-       {
-           "Cmd": ["cat", "/world"],
-           "ExposedPorts":{"22/tcp":{}}
-       }
+        POST /commit?container=44c004db4b17&m=message&repo=myrepo HTTP/1.1
 
     **Example response**:
 
@@ -1212,6 +1147,7 @@ Create a new image from a container's changes
     :query tag: tag
     :query m: commit message
     :query author: author (eg. "John Hannibal Smith <hannibal@a-team.com>")
+    :query run: config automatically applied when the image is run. (ex: {"Cmd": ["cat", "/world"], "PortSpecs":["22"]})
     :statuscode 201: no error
     :statuscode 404: no such container
     :statuscode 500: server error

docs/sources/api/registry_index_spec.rst

@@ -37,7 +37,7 @@ We expect that there will be only one instance of the index, run and managed by
 - It delegates authentication and authorization to the Index Auth service using tokens
 - It supports different storage backends (S3, cloud files, local FS)
 - It doesn’t have a local database
-- `Source Code <https://github.com/dotcloud/docker-registry>`_
+- It will be open-sourced at some point
 
 We expect that there will be multiple registries out there. To help to grasp the context, here are some examples of registries:
 
@@ -46,6 +46,10 @@ We expect that there will be multiple registries out there. To help to grasp the
 - **vendor registry**: such a registry is provided by a software vendor, who wants to distribute docker images. It would be operated and managed by the vendor. Only users authorized by the vendor would be able to get write access. Some images would be public (accessible for anyone), others private (accessible only for authorized users). Authentication and authorization would be delegated to the Index. The goal of vendor registries is to let someone do “docker pull basho/riak1.3” and automatically push from the vendor registry (instead of a sponsor registry); i.e. get all the convenience of a sponsor registry, while retaining control on the asset distribution.
 - **private registry**: such a registry is located behind a firewall, or protected by an additional security layer (HTTP authorization, SSL client-side certificates, IP address authorization...). The registry is operated by a private entity, outside of dotCloud’s control. It can optionally delegate additional authorization to the Index, but it is not mandatory.
 
+.. note::
+
+    Mirror registries and private registries which do not use the Index don’t even need to run the registry code. They can be implemented by any kind of transport implementing HTTP GET and PUT. Read-only registries can be powered by a simple static HTTP server.
+
 .. note::
 
     The latter implies that while HTTP is the protocol of choice for a registry, multiple schemes are possible (and in some cases, trivial):

docs/sources/api/remote_api_client_libraries.rst

@@ -1,4 +1,4 @@
-:title: Remote API Client Libraries
+:title: Registry API
 :description: Various client libraries available to use with the Docker remote API
 :keywords: API, Docker, index, registry, REST, documentation, clients, Python, Ruby, Javascript, Erlang, Go
 
@@ -12,28 +12,26 @@ compatibility. Please file issues with the library owners.  If you
 find more library implementations, please list them in Docker doc bugs
 and we will add the libraries here.
 
-+----------------------+----------------+--------------------------------------------+----------+
-| Language/Framework   | Name           | Repository                                 | Status   |
-+======================+================+============================================+==========+
-| Python               | docker-py      | https://github.com/dotcloud/docker-py      | Active   |
-+----------------------+----------------+--------------------------------------------+----------+
-| Ruby                 | docker-client  | https://github.com/geku/docker-client      | Outdated |
-+----------------------+----------------+--------------------------------------------+----------+
-| Ruby                 | docker-api     | https://github.com/swipely/docker-api      | Active   |
-+----------------------+----------------+--------------------------------------------+----------+
-| Javascript (NodeJS)  | docker.io      | https://github.com/appersonlabs/docker.io  | Active   |
-|                      |                | Install via NPM: `npm install docker.io`   |          |
-+----------------------+----------------+--------------------------------------------+----------+
-| Javascript           | docker-js      | https://github.com/dgoujard/docker-js      | Active   |
-+----------------------+----------------+--------------------------------------------+----------+
-| Javascript (Angular) | dockerui       | https://github.com/crosbymichael/dockerui  | Active   |
-| **WebUI**            |                |                                            |          |
-+----------------------+----------------+--------------------------------------------+----------+
-| Java                 | docker-java    | https://github.com/kpelykh/docker-java     | Active   |
-+----------------------+----------------+--------------------------------------------+----------+
-| Erlang               | erldocker      | https://github.com/proger/erldocker        | Active   |
-+----------------------+----------------+--------------------------------------------+----------+
-| Go                   | go-dockerclient| https://github.com/fsouza/go-dockerclient  | Active   |
-+----------------------+----------------+--------------------------------------------+----------+
-| PHP                  | Alvine         | http://pear.alvine.io/ (alpha)             | Active   |
-+----------------------+----------------+--------------------------------------------+----------+
++----------------------+----------------+--------------------------------------------+
+| Language/Framework   | Name           | Repository                                 |
++======================+================+============================================+
+| Python               | docker-py      | https://github.com/dotcloud/docker-py      |
++----------------------+----------------+--------------------------------------------+
+| Ruby                 | docker-client  | https://github.com/geku/docker-client      |
++----------------------+----------------+--------------------------------------------+
+| Ruby                 | docker-api     | https://github.com/swipely/docker-api      |
++----------------------+----------------+--------------------------------------------+
+| Javascript (NodeJS)  | docker.io      | https://github.com/appersonlabs/docker.io  |
+|                      |                | Install via NPM: `npm install docker.io`   |
++----------------------+----------------+--------------------------------------------+
+| Javascript           | docker-js      | https://github.com/dgoujard/docker-js      |
++----------------------+----------------+--------------------------------------------+
+| Javascript (Angular) | dockerui       | https://github.com/crosbymichael/dockerui  |
+| **WebUI**            |                |                                            |
++----------------------+----------------+--------------------------------------------+
+| Java                 | docker-java    | https://github.com/kpelykh/docker-java     |
++----------------------+----------------+--------------------------------------------+
+| Erlang               | erldocker      | https://github.com/proger/erldocker        |
++----------------------+----------------+--------------------------------------------+
+| Go                   | go-dockerclient| https://github.com/fsouza/go-dockerclient  |
++----------------------+----------------+--------------------------------------------+

docs/sources/conf.py

@@ -40,11 +40,7 @@ html_additional_pages = {
 
 # Add any Sphinx extension module names here, as strings. They can be extensions
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sphinxcontrib.httpdomain', 'sphinx.ext.extlinks']
-
-# Configure extlinks
-extlinks = { 'issue': ('https://github.com/dotcloud/docker/issues/%s',
-	               'Issue ') }
+extensions = ['sphinxcontrib.httpdomain']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -235,7 +231,7 @@ latex_documents = [
 # One entry per manual page. List of tuples
 # (source start file, name, description, authors, manual section).
 man_pages = [
-    ('commandline/cli', 'docker', u'Docker Documentation',
+    ('toctree', 'docker', u'Docker Documentation',
      [u'Team Docker'], 1)
 ]
 

docs/sources/contributing/contributing.rst

@@ -5,21 +5,5 @@
 Contributing to Docker
 ======================
 
-Want to hack on Docker? Awesome! 
+Want to hack on Docker? Awesome! The repository includes `all the instructions you need to get started <https://github.com/dotcloud/docker/blob/master/CONTRIBUTING.md>`_.
 
-The repository includes `all the instructions you need to get
-started <https://github.com/dotcloud/docker/blob/master/CONTRIBUTING.md>`_.
-
-The `developer environment Dockerfile
-<https://github.com/dotcloud/docker/blob/master/Dockerfile>`_
-specifies the tools and versions used to test and build Docker.
-
-If you're making changes to the documentation, see the
-`README.md <https://github.com/dotcloud/docker/blob/master/docs/README.md>`_.
-
-The `documentation environment Dockerfile
-<https://github.com/dotcloud/docker/blob/master/docs/Dockerfile>`_
-specifies the tools and versions used to build the Documentation.
-
-Further interesting details can be found in the `Packaging hints
-<https://github.com/dotcloud/docker/blob/master/hack/PACKAGERS.md>`_.

docs/sources/contributing/devenvironment.rst

@@ -42,10 +42,11 @@ This following command will build a development environment using the Dockerfile
 
 .. code-block:: bash
 
-    sudo make build
+    sudo docker build -t docker .
 
-If the build is successful, congratulations! You have produced a clean build of 
-docker, neatly encapsulated in a standard build environment. 
+
+
+If the build is successful, congratulations! You have produced a clean build of docker, neatly encapsulated in a standard build environment. 
 
 
 Step 4: Build the Docker Binary
@@ -55,41 +56,28 @@ To create the Docker binary, run this command:
 
 .. code-block:: bash
 
-	sudo make binary
+	sudo docker run -lxc-conf=lxc.aa_profile=unconfined -privileged -v `pwd`:/go/src/github.com/dotcloud/docker docker hack/make.sh binary
 
 This will create the Docker binary in ``./bundles/<version>-dev/binary/``
 
-Using your built Docker binary
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The binary is available outside the container in the directory 
-``./bundles/<version>-dev/binary/``. You can swap your host docker executable 
-with this binary for live testing - for example, on ubuntu: 
-
-.. code-block:: bash
-
-	sudo service docker stop ; sudo cp $(which docker) $(which docker)_ ; sudo cp ./bundles/<version>-dev/binary/docker-<version>-dev $(which docker);sudo service docker start
-	
-.. note:: Its safer to run the tests below before swapping your hosts docker binary.
-
 
 Step 5: Run the Tests
 ---------------------
 
+To run the Docker test cases you first need to disable `AppArmor <https://wiki.ubuntu.com/AppArmor>`_ using the following commands
+
+.. code-block:: bash
+
+	sudo /etc/init.d/apparmor stop
+	sudo /etc/init.d/apparmor teardown
+
 To execute the test cases, run this command:
 
 .. code-block:: bash
 
-	sudo make test
+	sudo docker run -lxc-conf=lxc.aa_profile=unconfined -privileged -v `pwd`:/go/src/github.com/dotcloud/docker docker hack/make.sh test
 
 
-Note: if you're running the tests in vagrant, you need to specify a dns entry in 
-the command (either edit the Makefile, or run the step manually): 
-
-.. code-block:: bash
-
-	sudo docker run -dns 8.8.8.8 -privileged -v `pwd`:/go/src/github.com/dotcloud/docker docker hack/make.sh test
-
 If the test are successful then the tail of the output should look something like this
 
 .. code-block:: bash
@@ -130,24 +118,15 @@ You can run an interactive session in the newly built container:
 
 .. code-block:: bash
 
-	sudo make shell
+	sudo docker run -privileged -i -t docker bash
 
-	# type 'exit' or Ctrl-D to exit
+	# type 'exit' to exit
 
 
-Extra Step: Build and view the Documentation
--------------------------------------------
 
-If you want to read the documentation from a local website, or are making changes
-to it, you can build the documentation and then serve it by:
-
-.. code-block:: bash
-
-	sudo make docs
-    # when its done, you can point your browser to http://yourdockerhost:8000
-	# type Ctrl-C to exit
+.. note:: The binary is available outside the container in the directory  ``./bundles/<version>-dev/binary/``. You can swap your host docker executable with this binary for live testing - for example, on ubuntu: ``sudo service docker stop ; sudo cp $(which docker) $(which docker)_ ; sudo cp ./bundles/<version>-dev/binary/docker-<version>-dev $(which docker);sudo service docker start``.
 
 
 **Need More Help?**
 
-If you need more help then hop on to the `#docker-dev IRC channel <irc://chat.freenode.net#docker-dev>`_ or post a message on the `Docker developer mailing list <https://groups.google.com/d/forum/docker-dev>`_.
+If you need more help then hop on to the `#docker-dev IRC channel <irc://chat.freenode.net#docker-dev>`_ or post a message on the `Docker developer mailinglist <https://groups.google.com/d/forum/docker-dev>`_.

docs/sources/examples/cfengine_process_management.rst

@@ -1,137 +0,0 @@
-:title: Process Management with CFEngine
-:description: Managing containerized processes with CFEngine
-:keywords: cfengine, process, management, usage, docker, documentation
-
-Process Management with CFEngine
-================================
-
-Create Docker containers with managed processes.
-
-Docker monitors one process in each running container and the container lives or dies with that process.
-By introducing CFEngine inside Docker containers, we can alleviate a few of the issues that may arise:
-
-* It is possible to easily start multiple processes within a container, all of which will be managed automatically, with the normal ``docker run`` command.
-* If a managed process dies or crashes, CFEngine will start it again within 1 minute.
-* The container itself will live as long as the CFEngine scheduling daemon (cf-execd) lives. With CFEngine, we are able to decouple the life of the container from the uptime of the service it provides.
-
-
-How it works
-------------
-
-CFEngine, together with the cfe-docker integration policies, are installed as part of the Dockerfile. This builds CFEngine into our Docker image.
-
-The Dockerfile's ``ENTRYPOINT`` takes an arbitrary amount of commands (with any desired arguments) as parameters.
-When we run the Docker container these parameters get written to CFEngine policies and CFEngine takes over to ensure that the desired processes are running in the container.
-
-CFEngine scans the process table for the ``basename`` of the commands given to the ``ENTRYPOINT`` and runs the command to start the process if the ``basename`` is not found.
-For example, if we start the container with ``docker run "/path/to/my/application parameters"``, CFEngine will look for a process named ``application`` and run the command.
-If an entry for ``application`` is not found in the process table at any point in time, CFEngine will execute ``/path/to/my/application parameters`` to start the application once again.
-The check on the process table happens every minute.
-
-Note that it is therefore important that the command to start your application leaves a process with the basename of the command.
-This can be made more flexible by making some minor adjustments to the CFEngine policies, if desired.
-
-
-Usage
------
-
-This example assumes you have Docker installed and working.
-We will install and manage ``apache2`` and ``sshd`` in a single container.
-
-There are three steps:
-
-1. Install CFEngine into the container.
-2. Copy the CFEngine Docker process management policy into the containerized CFEngine installation.
-3. Start your application processes as part of the ``docker run`` command.
-
-
-Building the container image
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The first two steps can be done as part of a Dockerfile, as follows.
-
-.. code-block:: bash
-
-    FROM ubuntu
-    MAINTAINER Eystein Måløy Stenberg <eytein.stenberg@gmail.com>
-
-    RUN apt-get -y install wget lsb-release unzip
-
-    # install latest CFEngine
-    RUN wget -qO- http://cfengine.com/pub/gpg.key | apt-key add -
-    RUN echo "deb http://cfengine.com/pub/apt $(lsb_release -cs) main" > /etc/apt/sources.list.d/cfengine-community.list
-    RUN apt-get update
-    RUN apt-get install cfengine-community
-
-    # install cfe-docker process management policy
-    RUN wget --no-check-certificate https://github.com/estenberg/cfe-docker/archive/master.zip -P /tmp/ && unzip /tmp/master.zip -d /tmp/
-    RUN cp /tmp/cfe-docker-master/cfengine/bin/* /var/cfengine/bin/
-    RUN cp /tmp/cfe-docker-master/cfengine/inputs/* /var/cfengine/inputs/
-    RUN rm -rf /tmp/cfe-docker-master /tmp/master.zip
-
-    # apache2 and openssh are just for testing purposes, install your own apps here
-    RUN apt-get -y install openssh-server apache2
-    RUN mkdir -p /var/run/sshd
-    RUN echo "root:password" | chpasswd  # need a password for ssh
-
-    ENTRYPOINT ["/var/cfengine/bin/docker_processes_run.sh"]
-
-
-By saving this file as ``Dockerfile`` to a working directory, you can then build your container with the docker build command,
-e.g. ``docker build -t managed_image``.
-
-Testing the container
-~~~~~~~~~~~~~~~~~~~~~
-
-Start the container with ``apache2`` and ``sshd`` running and managed, forwarding a port to our SSH instance:
-
-.. code-block:: bash
-
-    docker run -p 127.0.0.1:222:22 -d managed_image "/usr/sbin/sshd" "/etc/init.d/apache2 start"
-
-We now clearly see one of the benefits of the cfe-docker integration: it allows to start several processes
-as part of a normal ``docker run`` command.
-
-We can now log in to our new container and see that both ``apache2`` and ``sshd`` are running. We have set the root password to
-"password" in the Dockerfile above and can use that to log in with ssh:
-
-.. code-block:: bash
-
-    ssh -p222 root@127.0.0.1
-
-    ps -ef
-    UID        PID  PPID  C STIME TTY          TIME CMD
-    root         1     0  0 07:48 ?        00:00:00 /bin/bash /var/cfengine/bin/docker_processes_run.sh /usr/sbin/sshd /etc/init.d/apache2 start
-    root        18     1  0 07:48 ?        00:00:00 /var/cfengine/bin/cf-execd -F
-    root        20     1  0 07:48 ?        00:00:00 /usr/sbin/sshd
-    root        32     1  0 07:48 ?        00:00:00 /usr/sbin/apache2 -k start
-    www-data    34    32  0 07:48 ?        00:00:00 /usr/sbin/apache2 -k start
-    www-data    35    32  0 07:48 ?        00:00:00 /usr/sbin/apache2 -k start
-    www-data    36    32  0 07:48 ?        00:00:00 /usr/sbin/apache2 -k start
-    root        93    20  0 07:48 ?        00:00:00 sshd: root@pts/0 
-    root       105    93  0 07:48 pts/0    00:00:00 -bash
-    root       112   105  0 07:49 pts/0    00:00:00 ps -ef
-
-
-If we stop apache2, it will be started again within a minute by CFEngine.
-
-.. code-block:: bash
-
-    service apache2 status
-     Apache2 is running (pid 32).
-    service apache2 stop
-             * Stopping web server apache2 ... waiting    [ OK ]
-    service apache2 status
-     Apache2 is NOT running.
-    # ... wait up to 1 minute...
-    service apache2 status
-     Apache2 is running (pid 173).
-
-
-Adapting to your applications
------------------------------
-
-To make sure your applications get managed in the same manner, there are just two things you need to adjust from the above example:
-
-* In the Dockerfile used above, install your applications instead of ``apache2`` and ``sshd``.
-* When you start the container with ``docker run``, specify the command line arguments to your applications rather than ``apache2`` and ``sshd``.

docs/sources/examples/couchdb_data_volumes.rst

@@ -10,7 +10,7 @@ CouchDB Service
 .. include:: example_header.inc
 
 Here's an example of using data volumes to share the same data between
-two CouchDB containers.  This could be used for hot upgrades, testing
+2 CouchDB containers.  This could be used for hot upgrades, testing
 different versions of CouchDB on the same data, etc.
 
 Create first database
@@ -20,28 +20,28 @@ Note that we're marking ``/var/lib/couchdb`` as a data volume.
 
 .. code-block:: bash
 
-    COUCH1=$(sudo docker run -d -p 5984 -v /var/lib/couchdb shykes/couchdb:2013-05-03)
+    COUCH1=$(sudo docker run -d -v /var/lib/couchdb shykes/couchdb:2013-05-03)
 
 Add data to the first database
 ------------------------------
 
-We're assuming your Docker host is reachable at ``localhost``. If not,
-replace ``localhost`` with the public IP of your Docker host.
+We're assuming your docker host is reachable at `localhost`. If not,
+replace `localhost` with the public IP of your docker host.
 
 .. code-block:: bash
 
     HOST=localhost
-    URL="http://$HOST:$(sudo docker port $COUCH1 5984 | grep -Po '\d+$')/_utils/"
+    URL="http://$HOST:$(sudo docker port $COUCH1 5984)/_utils/"
     echo "Navigate to $URL in your browser, and use the couch interface to add data"
 
 Create second database
 ----------------------
 
-This time, we're requesting shared access to ``$COUCH1``'s volumes.
+This time, we're requesting shared access to $COUCH1's volumes.
 
 .. code-block:: bash
 
-    COUCH2=$(sudo docker run -d -p 5984 -volumes-from $COUCH1 shykes/couchdb:2013-05-03)
+    COUCH2=$(sudo docker run -d -volumes-from $COUCH1 shykes/couchdb:2013-05-03)
 
 Browse data on the second database
 ----------------------------------
@@ -49,8 +49,8 @@ Browse data on the second database
 .. code-block:: bash
 
     HOST=localhost
-    URL="http://$HOST:$(sudo docker port $COUCH2 5984 | grep -Po '\d+$')/_utils/"
+    URL="http://$HOST:$(sudo docker port $COUCH2 5984)/_utils/"
     echo "Navigate to $URL in your browser. You should see the same data as in the first database"'!'
 
-Congratulations, you are now running two Couchdb containers, completely
+Congratulations, you are running 2 Couchdb containers, completely
 isolated from each other *except* for their data.

docs/sources/examples/example_header.inc

@@ -1,7 +1,4 @@
 
 .. note::
     
-    * This example assumes you have Docker running in daemon mode. For
-      more information please see :ref:`running_examples`.
-    * **If you don't like sudo** then see :ref:`dockergroup`
-
+    This example assumes you have Docker running in daemon mode. For more information please see :ref:`running_examples`