Hi guys, it just might be me, but clicking the AMI from http://cloud-images.ubuntu.com/locator/ec2/ is broken to me. So I just did it via the normal Create Instance Wizard.
I just though some people might have the same issue.
Sorry if my markdown for links is screwed up, I went by the examples, since I'm used to the `[]()` traditional one.
Cheers.
This change makes docker compute container size correctly.
The old code isn't taking hard links into account. Containers could
seem like they're up to 1-1.5x larger than they really were.
This change makes docker compute layer size correctly.
The old code isn't taking hard links into account. Layers could
seem like they're up to 1-1.5x larger than they really were.
This is the proposed fix for #2506. It provides a more complete message
with regards to name collisions including informing of the opposing
containers ID.
I have included a test to ensure that the correct short id is displayed
to make the message easier to understand.
Finish container cleanup before setting the state to stopped.
Otherwise, for an application that exits quickly, a call to Restart
can allow Start to be called again before cleanup is done, resulting in
overritten data in the Container struct.
In the dind environment running on a Fedora host, the lxc utils get
confused by the /sys/fs/cgroup/cpuacct,cpu cgroup mount and lxc-start
fails trying to access the wrong cgroup directory.
Each arg to docker run should be placed in a separate string.
Otherwise, when starting the command via exec.Cmd, the command is
interpreted as "echo test", which can't be found.
I found that the original example wasn't starting the docker container when docker is started. So, I modified it using information I found from https://groups.google.com/forum/#!msg/docker-user/OWz8EOpiXuo/L_uxGFQuYlkJ
Also, upstart wasn't starting docker if it wasn't running ahead of time. So, I added a change to the docker upstart script to solve this.
When getting the private port used by a container using
`docker port` returns 0.0.0.0:49353, stript just the portnumber of the address with awk to used it in the rest of the example code.
- Remove the overrides config.vm.box and config.vm.box_url and use
the same values for all providers.
- Use the same private key path for all providers. It is still possible
to set a different private key path through the environment variable
SSH_PRIVKEY_PATH if desired (your AWS key may be different from your
Virtualbox key).
- Allow the environment variable AWS_INSTANCE_TYPE to specify the
instance type of instead of hard coding the AWS instance type as
't1.micro'. 't1.micro' is still the default if unspecified.
- Use the same environment variables for keys as the Amazon provided EC2
API tools. This allows people who already have the EC2 tools set up
correctly to use 'vagrant up' with less environment configuration
than before.
- Rewrite the provisioning code. The goal is to be idempotent and to
correctly install docker for all providers instead of just virtualbox.
It will conditionally install the virtualbox guest additions if
virtualbox is the provider.
- Update the AWS install documentation to reflect the changes.
Iptables interprets "-d 0.0.0.0" as "-d 0.0.0.0/32", not /0. This
results in the DNAT rule never matching any traffic if not bound
to a specific host IP.
Fixes#2598
lxc-start requires / to be mounted private, otherwise the changes
it does inside the container (both mounts and unmounts) will propagate
out to the host.
We work around this by starting up lxc-start in its own namespace where
we set / to rshared.
Unfortunately go can't really execute any code between clone and exec,
so we can't do this in a nice way. Instead we have a horrible hack that
use the unshare command, the shell and the mount command...
Retrieve /etc/resolv.conf data (if available)
Add checkNameserverOverlaps and call it to
make sure there are no conflicts
Add utils.GetNameserversAsCIDR and tests
Read /etc/resolv.conf and pull out nameservers,
formatting them as a CIDR block ("1.2.3.4/32")
This changes the server side code to make sure that:
1) the source of a bind mount isn't /
The bind mount "/:/foo" isn't allowed.
2) Check that the source exists
The source to be bind mounted must exist.
This fixes issue #2070.
This makes the docker cli reject docker run commands which include bind
mounts like "/:/some/path/in/the/container".
Bind mounting the root directory is a bad idea and the cli should throw
an error right away.
The same check will also be made by the remote API via another commit.
The graph uses a persistent database connection so
a lock is required because our current sqlite3 driver
does not implement retry logic when the ErrBusy is received.
One of my patches was incorporated into Docker, and at that time I didn't know that I was supposed to add myself into this list of contributors.
Commit ID: 7e09c6332d
And a few other patches are already in the pipeline.
+ remove PUBLIC and PRIVATE keywords
+ add the <host_interface>
+ add the -expose flag through a simple link example
+ update the meaning of the EXPOSE command
Added further information regarding how to access BIOS menus on Windows machines (laptops). My laptop required restarting to find the options and this information will undoubtedly help other users.
Fix-up the docker service file description to declare what the service is not what it does.
When a systemd machine starts up the Description of each unit scrolls by instead of the service's filename. Because the current description doesn't say what it is it isn't very friendly:
```
Oct 31 20:40:49 localhost systemd[1]: Started Update Engine.
Oct 31 20:40:49 localhost systemd[1]: Starting Multi-User System.
Oct 31 20:40:49 localhost systemd[1]: Reached target Multi-User System.
Oct 31 20:40:49 localhost systemd[1]: Starting Easily create lightweight, portable, self-sufficient containers from any application!...
Oct 31 20:40:49 localhost systemd[1]: Started Easily create lightweight, portable, self-sufficient containers from any application!.
```
in the example, PORT is assigned to "0.0.0.0:<port>" vs "<port>" as in previous versions of docker (I encountered this documentation discrepancy tonight after installing docker v0.6.5)
* Add line after headers
* Remove space before bullet points
* Fix ' character
* Create sub-headers
* Group add fix etc. Upper-case
* Use head to trac current progress
Go's net/http API does not send headers until the first call to
ResponseWriter.Write(). Ordinarily, this is fine, because in most
cases, responses are returned immediately. However, for the events
API, nothing is written until some event is sent, which causes
timeouts and/or hangs in some HTTP client APIs, which wait for headers
before returning from the "make request" call.
* Implement a new package: engine. It exposes a useful but minimalist job API.
* Refactor main() to instanciate an Engine instead of a Server directly.
* Refactor server.go to register an engine job.
This is the smallest possible refactor which can include the new Engine design
into master. More gradual refactoring will follow.
After a nice long brainstorming session with @shykes on IRC, we decided on using a SHA1 hash of dockerinit compiled into the dynamic docker binary to ensure that we always use the two in a perfect pair, and never mix and match.
This just tries pinging first 127.0.0.1 and then 8.8.8.8.
We can't ping via a dns name, because libresolv.so is missing
from the test image, so dns resolving doesn't work.
This patch allows debootstrap to be run with the http_proxy environment
variable. This is principally useful when you're using apt-cacher and
apt-cacher-ng locally.
I believe that it is helpful to build the binary first. That way,
if you interrupt the tests, you still get a binary to play with.
If you run the tests first and interrupt them, no binary for you!
Also, the second part of this commit is an undeniable proof that
Bash array syntax is nothing else than an elaborate troll by Bash
authors.
This includes the following:
- options to control certain parameters
- an option specifically for creating tarballs directly, esp. for dockerbrew
- the addition of the updates and security repositories for Debian images
- the addition of the universe, updates, and security repositories for Ubuntu images
- more correct tagging of Debian images
- tagging of Ubuntu image versions (12.04, 12.10, etc) and latest tag for LTS
* don't mention outdated PPA that Vagrantfile doesn't use
* point to official Ubuntu installation instructions that Vagrant appears to be following; should be future proof
* added link to edit on GitHub
* Changed image source on homepage
* Made some changes to the structure, added the ability to have l3 navigation. Added warning, note and other styles.
* Fixed an image link, removed the .. :content: links because they were quicky and didn't look good, added pagelinks to current page of other versions.
* Moved the remote client api's to their own doc
add AutoRemove to HostConfig
add -rm flag to docker run
add TestRunAutoRemove to test -rm
docs: add -rm to commandline/command/run
add hostConfig to container monitor
make monitor destroy the container via -rm
This adds support for automatically removing a container after it
exits. The removal of the container is handled on the server side.
This means you're able to set the bits for capabilities on files
inside the container. This is needed for e.g. many fedora packages
as they use finegrained capabilities rather than setuid binaries.
This is safe as we're not adding capabilities really, since the
container is already allowed to create setuid binaries. Setuid
binaries are strictly more powerful that any capabilities (as root implies
all capabilities).
This doesn't mean the container can *gain* capabilities that it
doesn't already have though. The actual set of caps are strictly
decreasing.
We're officially a first-class Gentoo citizen now, which is very exciting. Many thanks to @gregkh for helping us get here.
I started just adapting sections of language in this document, and realized several bits needed to just be rewritten entirely to be more clear.
mostly because I've been failing at getting docker `0.6.2` working with
docker-registry `0.6.0` with regard to login and push. I suspect there
may be some mismatched expectations between `auth.ResolveAuthConfig` and
`registry.ResolveRepositoryName`, but I haven't done enough research or
experimentation to think it's anything more than user error. More tests
are good, no?
We never want the container to be in the same process group as the
daemon, as then the container will receive signals sent to the
process group of the container.
Add simple workaround for #1755 to contrib/mkimage-debian.sh
Add simple echo patch in contrib/mkimage-debian.sh to prevent init scripts from running during apt-get installs/updates
Add `apt-get clean` to mkimage-debian.sh for slightly smaller images
Add more small apt tweaks to mkimage-debian.sh thanks to @jpetazzo and @spahl
Changed "You need to enable namespaces and cgroups, to the extend of what is needed to run LXC containers" to "You need to enable namespaces and cgroups, to the extent of what is needed to run LXC containers"
Without those lines, VitrualBox machines are being very slow with all network operations (tested only in OS X Lion, but they are supposed to work everywhere). See https://github.com/mitchellh/vagrant/issues/1807
To improve the use of docker with a private registry the login
command is extended with a parameter for the server address.
While implementing i noticed that two problems hindered authentication to a
private registry:
1. the resolve of the authentication did not match during push
because the looked up key was for example localhost:8080 but
the stored one would have been https://localhost:8080
Besides The lookup needs to still work if the https->http fallback
is used
2. During pull of an image no authentication is sent, which
means all repositories are expected to be private.
These points are fixed now. The changes are implemented in
a way to be compatible to existing behavior both in the
API as also with the private registry.
Update:
- login does not require the full url any more, you can login
to the repository prefix:
example:
docker logon localhost:8080
Fixed corner corner cases:
- When login is done during pull and push the registry endpoint is used and
not the central index
- When Remote sends a 401 during pull, it is now correctly delegating to
CmdLogin
- After a Login is done pull and push are using the newly entered login data,
and not the previous ones. This one seems to be also broken in master, too.
- Auth config is now transfered in a parameter instead of the body when
/images/create is called.
* 'login_signal' of https://github.com/calavera/docker:
Use flag.StringVar to capture the command line flags.
Fix syscall name.
Remove unused imports.
Simplify term signal handler.
Add the ISIG syscall back to not kill the client withing a shell with ctrl+c.
Print a new line after getting the password from stdin.
Exit if there is any error reading from stdin.
Stop making a raw terminal to ask for registry login credentials.
Allow to generate signals when termios is in raw mode.
Use a more idiomatic syntax to capture the exit.
Exit from `docker login` on SIGTERM and SIGINT.
* master: (23 commits)
Made calling changelog before run return empty. Fixes#1705.
fix error in docker build when param is not a directory
Document FROM <image>:<tag> Dockerfile instruction.
Install Ubuntu raring backported kernel from official archives directly.
Updated "Use -> The Basics" to use ubuntu:12.10.
hide version when not available
added a Dockerfile which installs all deps and builds the docs.
Unable to find image error should print to stderr
remove message during tests
use init function
add TEST env var during tests and silenced parserun during tests
Update python_web_app.rst
Update remaining upstart scripts to wait for lxc-net
Fixed a minor syntax error.
Add privileged flag in documentation for container creation
Fix#1685: Notes on production use. General installation cleanup.
Fix bash completion, remove have
added apt-key finger tip and fingerprint in ubuntu installation page
Improve formatting with 'go fmt' as stated in CONTRIBUTING.md
Start docker after lxc-net to prevent ip forwarding race
...
It only disables echo asking for the password and lets the terminal to handle everything else.
It fixes#1392 since blank spaces are not discarded as they did before.
It also cleans the login code a little bit to improve readability.
As 'go fmt' doesn't support verifying files in multiple directories,
it's probably a good idea to run it on all '*.go' files from time to
time with something like this:
find . -name "*.go" | xargs dirname | sort -u | xargs -n 1 echo go fmt
The dev environment setup procedure fails if mercurial is not installed
on the machine, it cannot download stuff from
http://code.google.com/p/go.net for instance.
During parallel pull of a repostiory it can happen that the same layer
is pulled more than once.
To fix this I have extended the locking code to
- avoid multiple pulls of the same image
- avoid multiple pulls of the same layer
If an error occurs the other layers are awaited before returning as leaving
the scope before the go routines leave causes crashes of the server sometimes
if the download status is updated while the http stream is already closed
Beside this I have extended status display.
For consistency the Buildfile should have the option to
set the working directory.
Of course that is one option more to the buildfile,
so please tell me if we really want this to happen.
This makes it possible to simply wrap a command inside a container. This makes
it easier to use a container as an unified build environment.
Examples:
~/workspace/docker
$ docker run -v `pwd`:`pwd` -w `pwd` -i -t ubuntu ls
AUTHORS Makefile archive.go changes.go docker
[...]
docker run -v `pwd`:`pwd` -w `pwd` -i -t ubuntu pwd
/home/marco/workspace/docker
- Added parmalinks (closes#1527)
- Changed the 'fork us on github' button to 'Edit this page on github', so people can edit quickly (closes#1532)
- Changed the favicon
Sometimes `ip route` will show mask-less IPs, so net.ParseCIDR will fail. If it does we check if we can net.ParseIP, and fail only if we can't.
Fixes#1214Fixes#362
It was confusing to me as a first-time user that my docker attach command failed; I was expecting the container to run continuously, and when I couldn't attach to it, I assumed something was wrong. ps -a shows that the container still exists, which gives the user confidence to go to the next step.
When memory cgroup is absent, there was not attempt to detect if IPv4
forwarding was enabled and therefore, docker was printing a warning
for each command spawning a new container. The test for IPv4
forwarding was guarded by the test for memory cgroup.
Removes the error when a container already has a volume that would otherwise
be created by `Config.VolumesFrom`. Allows restarting containers with a
`Config.VolumesFrom` set.
Copies the volumes from the container specified in `Config.VolumesFrom` before
creating volumes from `Config.Volumes`. Skips any preexisting volumes when
processing `Config.Volumes`. Fixes#1351
stepN is only used in the log line, so if we only produce the log line
when there's a message, it should do the right thing.
If it's *not* a valid instruction, it gets a line as well, so there's no
reason to double up.
It only disables echo asking for the password and lets the terminal to handle everything else.
It fixes#1392 since blank spaces are not discarded as they did before.
It also cleans the login code a little bit to improve readability.
* master: (54 commits)
Return JSONError for HTTPResponse error
Fix TestEnv
Revert "Bind daemon to 0.0.0.0 in Vagrant. Fixes#1304"
Add unit tests for build no cache
Add no cache for docker build
Move note about officially supported kernel
Solved the logo being squished in Safari
Add hostname to the container environment.
fix same issue in api.go
improve tests
Return registy status code in error
update http://get.docker.io/latest
Add check that the request is good
fix tests about refactor checksums
add ufw doc
Fixed a couple of minor syntax errors.
Updated the description of run -d
Make sure the index also receives the checksums
Switch json/payload order
Remove unused parameter
...
Make clear this documentation is about installing docker on EC2 with the
help of vagrant, which doesn't make it easy to start multiple instances,
instead of plain vanilla EC2 installation.
Sometimes `ip route` will show mask-less IPs, so net.ParseCIDR will fail. If it does we check if we can net.ParseIP, and fail only if we can't.
Fixes#1214Fixes#362
The goal is to make it more clear this will give you the container id after run completes.
Since stdout is now standard on run, "docker run -d" is the best (or only) way to get the container ID returned from docker after a plain run, but the description (help) does not hint any such thing.
* changed conf.py to reference toctree.rst instead of index
* Added note to README to upgrade your sphinx to the latest version to prevent a bug with .. note:: blocks.
SaveConfig sets the Username and Password to an empty string
on save. A copy of the authConfigs need to be made so that the
in memory data is not modified.
This change makes docker attempt to create the container ID file and
open it before attempting to create the container. This avoids leaving
a stale container behind if docker has failed to create and open the
container ID file.
The container ID is written to the file after the container is created.
Currently, if you have the following images:
foo/bar 1 23b27d50fb49
foo/bar 2 f2b86ec3fcc4
And you issue the following command:
docker tag foo/bar:2 foo/bar latest
docker will tag the "wrong" image, because the image id for foo/bar:1 starts
with a "2". That is, you'll end up with the following:
foo/bar 1 23b27d50fb49
foo/bar 2 f2b86ec3fcc4
foo/bar latest 23b27d50fb49
This commit reverses the priority given to tags vs. image ids in the
construction `<user>/<repo>:<tagOrId>`, meaning that if a tag that is an exact
match for the specified `tagOrId`, it will be tagged in preference to an image
with an id that happens to start with the correct character sequence.
Prior this commit, 'docker images' and other cmd's, which used utils.HumanSize(),
showed unnecessary whitespaces.
Formatting of progress has been moved to FormatProgess(), justifing the string
directly in the template.
- For the TCP test try again if socat wasn't listening yet;
- For the UDP test raise the timeout to a minute to workaround what
seems to be an issue with Linux.
API Changes
-----------
The port notation is extended to support "/udp" or "/tcp" at the *end*
of the specifier string (and defaults to tcp if "/tcp" or "/udp" are
missing)
`docker ps` now shows UDP ports as "frontend->backend/udp". Nothing
changes for TCP ports.
`docker inspect` now displays two sub-dictionaries: "Tcp" and "Udp",
under "PortMapping" in "NetworkSettings".
Theses changes stand true for the values returned by the HTTP API too.
This changeset will definitely break tools built upon the API (or upon
`docker inspect`). A less intrusive way to add UDP ports in `docker
inspect` would be to simply add "/udp" for UDP ports but it will still
break existing applications which tries to convert the whole field to an
integer. I believe that having two TCP/UDP sub-dictionaries is better
because it makes the whole thing more clear and more easy to parse right
away (i.e: you don't have to check the format of the string, split it
and convert the right part to an integer)
Code Changes
------------
Significant changes in network.go:
- A second PortAllocator is instantiated for the UDP range;
- PortMapper maintains separate mapping for TCP and UDP;
- The extPorts array in NetworkInterface is now an array of Nat objects
(so we can know on which protocol a given port was mapped when
NetworkInterface.Release() is called);
- TCP proxying on localhost has been moved away in network_proxy.go.
localhost proxy code rewrite in network_proxy.go:
We have to proxy the traffic between localhost:frontend-port and
container:backend-port because Netfilter doesn't work properly on the
loopback interface and DNAT iptable rules aren't applied there.
- Goroutines in the TCP proxying code are now explicitly stopped when
the proxy is stopped;
- UDP connection tracking using a map (more infos in [1]);
- Support for IPv6 (to be more accurate, the code is transparent to the
Go net package, so you can use, tcp/tcp4/tcp6/udp/udp4/udp6);
- Single Proxy interface for both UDP and TCP proxying;
- Full test suite.
[1] https://github.com/dotcloud/docker/issues/33#issuecomment-20010400
writing out streamed status.
This is caused by a Buffering message that is not in the correct json format:
[...]
{"status"
:"Pushing 6bba11a28f1ca247de9a47071355ce5923a45b8fea3182389f992f4
24b93edae"}Buffering to disk 244/? (n/a)..
{"status":"Pushing",[...]
The "Buffering to disk" message is originated in
srv.runtime.graph.TempLayerArchive
I am now using the StreamFormatter provided by the context from which the
method is called.
For structs protected by a single mutex, embed the mutex for more
concise usage.
Also use a sync.Mutex directly, rather than a pointer, to avoid the
need for initialization (because a Mutex's zero-value is valid and
ready to be used).
- Fix TestGetImagesJSON when there is more than one image in the test
repository;
- Remove an hardcoded constant use in TestGetImagesByName;
- Wait in a loop in TestKillDifferentUser;
- Use env instead of /usr/bin/env in TestEnv;
- Create a daemon user in contrib/mkimage-unittest.sh.
It's a fork of the mkimage-busybox.sh script and it adds socat to the
image. (socat being needed to add udp support, see #33).
This script, like mkimage-busybox.sh, probably only works on
Debian/Ubuntu.
The dockerbuilder Dockerfile was installing one package per apt-get
install operation.
This changes it so that consecutive run apt-get install operations are
batched into a single operation.
As a user who has blown $150 on VMWare Fusion and vagrant-vmware, I
would like to use my new shiny to hack on Docker. Docker already has a
multi-provider Vagrantfile, so adding another one presents little risk.
Known Issues:
- The docker install of a new kernel breaks the Vagrant shared folder.
- This seems to be because the VMWare hgfs module doesn't build
against a 3.8 kernel.
- I don't believe that shared folder support is actually in use
* replaced previously removed concepts/containers and concepts/introcution by a redirect
* moved orphan index/varable to workingwiththerepository
* added favicon to the layout.html
* added redirect_home which is a http refresh redirect. It works like a 301 for google
* fixed an issue in the layout that would make it break when on small screens
into the container instead of copying them as a regular file.
* Builder: ADD uses tar/untar for copies instead of calling 'cp -ar'.
This is more consistent, reduces the number of dependencies, and
fixe #896.
Move creating the router and populating the
routes to a separate function outside of
ListenAndServe to allow unit tests to make
assertions on the configured routes and handler
funcs.
Add the Access-Control-Allow-Methods header so that
DELETE operations are allowed.
Also move the write CORS headers method before
docker writes a 404 not found so that the client
receives the correct response and not an invalid
CORS request.
seperated the registry and index API's into their own docs and merged
in the index search api into the index api. Also renamed the original
registry api to registry_index_spec.
This updates the documentation to mention that:
1. a lot of data may get sent to the docker daemon if there is a lot of
data in the directory passed to docker build
2. ADD doesn't work in the absence of the context
3. running without a context doesn't send file data to the docker
daemon
4. explain that the data sent to the docker daemon will be used by ADD
commands
Add the -api-enable-cors flag when running docker
in daemon mode to allow CORS requests to be made to
the Remote Api. The default value is false for this
flag to not allow cross origin request to be made.
Also added a handler for OPTIONS requests the standard
for cross domain requests is to initially make an
OPTIONS request to the api.
Fall back to image-specified hostname if user doesn't
provide one, instead of only using image-specified
hostname if the user *does* try to set one.
(ditto for username)
Closes#694.
Usage: docker COMMAND [arg...]
A self-sufficient runtime for linux containers.
Commands:
attach Attach to a running container
insert Insert a file in an image
login Register or Login to the docker registry server
export Stream the contents of a container as a tar archive
diff Inspect changes on a container's filesystem
logs Fetch the logs of a container
pull Pull an image or a repository from the docker registry server
restart Restart a running container
build Build a container from Dockerfile or via stdin
history Show the history of an image
kill Kill a running container
rmi Remove an image
start Start a stopped container
tag Tag an image into a repository
commit Create a new image from a container's changes
import Create a new filesystem image from the contents of a tarball
ps List containers
rm Remove a container
run Run a command in a new container
wait Block until a container stops, then print its exit code
images List images
port Lookup the public-facing port which is NAT-ed to PRIVATE_PORT
info Display system-wide information
inspect Return low-level information on a container
push Push an image or a repository to the docker registry server
search Search for an image in the docker index
stop Stop a running container
version Show the docker version information back
- Registry: Use correct auth config when logging in.
- Registry: Fix the error message so it is the same as the regex
## 0.6.3 (2013-09-23)
#### Packaging
* Add 'docker' group on install for ubuntu package
* Update tar vendor dependency
* Download apt key over HTTPS
#### Runtime
- Only copy and change permissions on non-bindmount volumes
* Allow multiple volumes-from
- Fix HTTP imports from STDIN
#### Documentation
* Update section on extracting the docker binary after build
* Update development environment docs for new build process
* Remove 'base' image from documentation
#### Other
- Client: Fix detach issue
- Registry: Update regular expression to match index
## 0.6.2 (2013-09-17)
#### Runtime
+ Add domainname support
+ Implement image filtering with path.Match
* Remove unnecesasry warnings
* Remove os/user dependency
* Only mount the hostname file when the config exists
* Handle signals within the `docker login` command
- UID and GID are now also applied to volumes
-`docker start` set error code upon error
-`docker run` set the same error code as the process started
#### Builder
+ Add -rm option in order to remove intermediate containers
* Allow multiline for the RUN instruction
#### Registry
* Implement login with private registry
- Fix push issues
#### Other
+ Hack: Vendor all dependencies
* Remote API: Bump to v1.5
* Packaging: Break down hack/make.sh into small scripts, one per 'bundle': test, binary, ubuntu etc.
* Documentation: General improvments
## 0.6.1 (2013-08-23)
#### Registry
* Pass "meta" headers in API calls to the registry
#### Packaging
- Use correct upstart script with new build tool
- Use libffi-dev, don`t build it from sources
- Remove duplicate mercurial install command
## 0.6.0 (2013-08-22)
#### Runtime
+ Add lxc-conf flag to allow custom lxc options
+ Add an option to set the working directory
* Add Image name to LogEvent tests
+ Add -privileged flag and relevant tests, docs, and examples
* Add websocket support to /container/<name>/attach/ws
* Add warning when net.ipv4.ip_forwarding = 0
* Add hostname to environment
* Add last stable version in `docker version`
- Fix race conditions in parallel pull
- Fix Graph ByParent() to generate list of child images per parent image.
- Fix typo: fmt.Sprint -> fmt.Sprintf
- Fix small \n error un docker build
* Fix to "Inject dockerinit at /.dockerinit"
* Fix #910. print user name to docker info output
* Use Go 1.1.2 for dockerbuilder
* Use ranged for loop on channels
- Use utils.ParseRepositoryTag instead of strings.Split(name, ":") in server.ImageDelete
- Improve CMD, ENTRYPOINT, and attach docs.
- Improve connect message with socket error
- Load authConfig only when needed and fix useless WARNING
- Show tag used when image is missing
* Apply volumes-from before creating volumes
- Make docker run handle SIGINT/SIGTERM
- Prevent crash when .dockercfg not readable
- Install script should be fetched over https, not http.
* API, issue 1471: Use groups for socket permissions
- Correctly detect IPv4 forwarding
* Mount /dev/shm as a tmpfs
- Switch from http to https for get.docker.io
* Let userland proxy handle container-bound traffic
* Updated the Docker CLI to specify a value for the "Host" header.
- Change network range to avoid conflict with EC2 DNS
- Reduce connect and read timeout when pinging the registry
* Parallel pull
- Handle ip route showing mask-less IP addresses
* Allow ENTRYPOINT without CMD
- Always consider localhost as a domain name when parsing the FQN repos name
* Refactor checksum
#### Documentation
* Add MongoDB image example
* Add instructions for creating and using the docker group
* Add sudo to examples and installation to documentation
* Add ufw doc
* Add a reference to ps -a
* Add information about Docker`s high level tools over LXC.
* Fix typo in docs for docker run -dns
* Fix a typo in the ubuntu installation guide
* Fix to docs regarding adding docker groups
* Update default -H docs
* Update readme with dependencies for building
* Update amazon.rst to explain that Vagrant is not necessary for running Docker on ec2
* PostgreSQL service example in documentation
* Suggest installing linux-headers by default.
* Change the twitter handle
* Clarify Amazon EC2 installation
* 'Base' image is deprecated and should no longer be referenced in the docs.
* Move note about officially supported kernel
- Solved the logo being squished in Safari
#### Builder
+ Add USER instruction do Dockerfile
+ Add workdir support for the Buildfile
* Add no cache for docker build
- Fix docker build and docker events output
- Only count known instructions as build steps
- Make sure ENV instruction within build perform a commit each time
- Forbid certain paths within docker build ADD
- Repository name (and optionally a tag) in build usage
- Make sure ADD will create everything in 0755
#### Remote API
* Sort Images by most recent creation date.
* Reworking opaque requests in registry module
* Add image name in /events
* Use mime pkg to parse Content-Type
* 650 http utils and user agent field
#### Hack
+ Bash Completion: Limit commands to containers of a relevant state
* Add docker dependencies coverage testing into docker-ci
#### Packaging
+ Docker-brew 0.5.2 support and memory footprint reduction
* Add new docker dependencies into docker-ci
- Revert "docker.upstart: avoid spawning a `sh` process"
+ Docker-brew and Docker standard library
+ Release docker with docker
* Fix the upstart script generated by get.docker.io
* Enabled the docs to generate manpages.
* Revert Bind daemon to 0.0.0.0 in Vagrant.
#### Register
* Improve auth push
* Registry unit tests + mock registry
#### Tests
* Improve TestKillDifferentUser to prevent timeout on buildbot
- Fix typo in TestBindMounts (runContainer called without image)
* Improve TestGetContainersTop so it does not rely on sleep
* Relax the lo interface test to allow iface index != 1
* Add registry functional test to docker-ci
* Add some tests in server and utils
#### Other
* Contrib: bash completion script
* Client: Add docker cp command and copy api endpoint to copy container files/folders to the host
* Don`t read from stdout when only attached to stdin
## 0.5.3 (2013-08-13)
#### Runtime
* Use docker group for socket permissions
- Spawn shell within upstart script
- Handle ip route showing mask-less IP addresses
- Add hostname to environment
#### Builder
- Make sure ENV instruction within build perform a commit each time
## 0.5.2 (2013-08-08)
* Builder: Forbid certain paths within docker build ADD
- Runtime: Change network range to avoid conflict with EC2 DNS
* API: Change daemon to listen on unix socket by default
## 0.5.1 (2013-07-30)
#### Runtime
+ Add `ps` args to `docker top`
+ Add support for container ID files (pidfile like)
+ Add container=lxc in default env
+ Support networkless containers with `docker run -n` and `docker -d -b=none`
* Stdout/stderr logs are now stored in the same file as JSON
* Allocate a /16 IP range by default, with fallback to /24. Try 12 ranges instead of 3.
* Change .dockercfg format to json and support multiple auth remote
- Do not override volumes from config
- Fix issue with EXPOSE override
#### API
+ Docker client now sets useragent (RFC 2616)
+ Add /events endpoint
#### Builder
+ ADD command now understands URLs
+ CmdAdd and CmdEnv now respect Dockerfile-set ENV variables
- Create directories with 755 instead of 700 within ADD instruction
#### Hack
* Simplify unit tests with helpers
* Improve docker.upstart event
* Add coverage testing into docker-ci
## 0.5.0 (2013-07-17)
#### Runtime
+ List all processes running inside a container with 'docker top'
+ Host directories can be mounted as volumes with 'docker run -v'
+ Containers can expose public UDP ports (eg, '-p 123/udp')
+ Optionally specify an exact public port (eg. '-p 80:4500')
* 'docker login' supports additional options
- Dont save a container`s hostname when committing an image.
#### Registry
+ New image naming scheme inspired by Go packaging convention allows arbitrary combinations of registries
- Fix issues when uploading images to a private registry
#### Builder
+ ENTRYPOINT instruction sets a default binary entry point to a container
+ VOLUME instruction marks a part of the container as persistent data
* 'docker build' displays the full output of a build by default
## 0.4.8 (2013-07-01)
+ Builder: New build operation ENTRYPOINT adds an executable entry point to the container. - Runtime: Fix a bug which caused 'docker run -d' to no longer print the container ID.
- Tests: Fix issues in the test suite
## 0.4.7 (2013-06-28)
#### Remote API
* The progress bar updates faster when downloading and uploading large files
- Fix a bug in the optional unix socket transport
#### Runtime
* Improve detection of kernel version
+ Host directories can be mounted as volumes with 'docker run -b'
- fix an issue when only attaching to stdin
* Use 'tar --numeric-owner' to avoid uid mismatch across multiple hosts
#### Hack
* Improve test suite and dev environment
* Remove dependency on unit tests on 'os/user'
#### Other
* Registry: easier push/pull to a custom registry
+ Documentation: add terminology section
## 0.4.6 (2013-06-22)
- Runtime: fix a bug which caused creation of empty images (and volumes) to crash.
## 0.4.5 (2013-06-21)
+ Builder: 'docker build git://URL' fetches and builds a remote git repository
* Runtime: 'docker ps -s' optionally prints container size
* Tests: Improved and simplified
- Runtime: fix a regression introduced in 0.4.3 which caused the logs command to fail.
- Builder: fix a regression when using ADD with single regular file.
## 0.4.4 (2013-06-19)
- Builder: fix a regression introduced in 0.4.3 which caused builds to fail on new clients.
## 0.4.3 (2013-06-19)
#### Builder
+ ADD of a local file will detect tar archives and unpack them
* ADD improvements: use tar for copy + automatically unpack local archives
* ADD uses tar/untar for copies instead of calling 'cp -ar'
* Fixed the behavior of ADD to be (mostly) reverse-compatible, predictable and well-documented.
- Fix a bug which caused builds to fail if ADD was the first command
* Nicer output for 'docker build'
#### Runtime
* Remove bsdtar dependency
* Add unix socket and multiple -H support
* Prevent rm of running containers
* Use go1.1 cookiejar
- Fix issue detaching from running TTY container
- Forbid parralel push/pull for a single image/repo. Fixes #311
- Fix race condition within Run command when attaching.
#### Client
* HumanReadable ProgressBar sizes in pull
* Fix docker version`s git commit output
#### API
* Send all tags on History API call
* Add tag lookup to history command. Fixes #882
#### Documentation
- Fix missing command in irc bouncer example
## 0.4.2 (2013-06-17)
- Packaging: Bumped version to work around an Ubuntu bug
## 0.4.1 (2013-06-17)
#### Remote Api
+ Add flag to enable cross domain requests
+ Add images and containers sizes in docker ps and docker images
#### Runtime
+ Configure dns configuration host-wide with 'docker -d -dns'
+ Detect faulty DNS configuration and replace it with a public default
+ Allow docker run <name>:<id>
+ You can now specify public port (ex: -p 80:4500)
* Improved image removal to garbage-collect unreferenced parents
#### Client
* Allow multiple params in inspect
* Print the container id before the hijack in `docker run`
#### Registry
* Add regexp check on repo`s name
* Move auth to the client
- Remove login check on pull
#### Other
* Vagrantfile: Add the rest api port to vagrantfile`s port_forward
* Upgrade to Go 1.1
- Builder: don`t ignore last line in Dockerfile when it doesn`t end with \n
## 0.4.0 (2013-06-03)
#### Builder
+ Introducing Builder
+ 'docker build' builds a container, layer by layer, from a source repository containing a Dockerfile
#### Remote API
+ Introducing Remote API
+ control Docker programmatically using a simple HTTP/json API
#### Runtime
* Various reliability and usability improvements
## 0.3.4 (2013-05-30)
#### Builder
+ 'docker build' builds a container, layer by layer, from a source repository containing a Dockerfile
+ 'docker build -t FOO' applies the tag FOO to the newly built container.
#### Runtime
+ Interactive TTYs correctly handle window resize
* Fix how configuration is merged between layers
#### Remote API
+ Split stdout and stderr on 'docker run'
+ Optionally listen on a different IP and port (use at your own risk)
#### Documentation
* Improved install instructions.
## 0.3.3 (2013-05-23)
- Registry: Fix push regression
-Various bugfixes
-Registry: Fix push regression
- Various bugfixes
## 0.3.2 (2013-05-09)
* Runtime: Store the actual archive on commit
* Registry: Improve the checksum process
* Registry: Use the size to have a good progress bar while pushing
*Registry: Use the actual archive if it exists in order to speed up the push
- Registry: Fix error 400 on push
#### Registry
*Improve the checksum process
* Use the size to have a good progress bar while pushing
* Use the actual archive if it exists in order to speed up the push
- Fix error 400 on push
#### Runtime
* Store the actual archive on commit
## 0.3.1 (2013-05-08)
+ Builder: Implement the autorun capability within docker builder
+ Builder: Add caching to docker builder
+ Builder: Add support for docker builder with native API as top level command
+ Runtime: Add go version to debug infos
+ Builder: Implement ENV within docker builder
+ Registry: Add docker search top level command in order to search a repository
+ Images: output graph of images to dot (graphviz)
+ Documentation: new introduction and high-level overview
+ Documentation: Add the documentation for docker builder
+ Website: new high-level overview
- Makefile: Swap "go get" for "go get -d", especially to compile on go1.1rc
- Images: fix ByParent function
- Builder: Check the command existance prior create and add Unit tests for the case
- Registry: Fix pull for official images with specific tag
- Registry: Fix issue when login in with a different user and trying to push
- Documentation: CSS fix for docker documentation to make REST API docs look better.
- Documentation: Fixed CouchDB example page header mistake
- Documentation: fixed README formatting
* Registry: Improve checksum - async calculation
* Runtime: kernel version - don't show the dash if flavor is empty
*Documentation: updated www.docker.io website.
* Builder: use any whitespaces instead of tabs
* Packaging: packaging ubuntu; issue #510: Use goland-stable PPA package to build docker
#### Builder
+ Implement the autorun capability within docker builder
+ Add caching to docker builder
+ Add support for docker builder with native API as top level command
+ Implement ENV within docker builder
- Check the command existance prior create and add Unit tests for the case
* use any whitespaces instead of tabs
#### Runtime
+ Add go version to debug infos
* Kernel version - don`t show the dash if flavor is empty
#### Registry
+ Add docker search top level command in order to search a repository
- Fix pull for official images with specific tag
- Fix issue when login in with a different user and trying to push
*Improve checksum - async calculation
#### Images
+ Output graph of images to dot (graphviz)
- Fix ByParent function
#### Documentation
+ New introduction and high-level overview
+ Add the documentation for docker builder
- CSS fix for docker documentation to make REST API docs look better.
- Fix CouchDB example page header mistake
- Fix README formatting
* Update www.docker.io website.
#### Other
+ Website: new high-level overview
- Makefile: Swap "go get" for "go get -d", especially to compile on go1.1rc
* Packaging: packaging ubuntu; issue #510: Use goland-stable PPA package to build docker
## 0.3.0 (2013-05-06)
+ Registry: Implement the new registry
+ Documentation: new example: sharing data between 2 couchdb databases
- Runtime: Fix the command existance check
-Runtime: strings.Split may return an empty string on no match
-Runtime: Fix an index out of range crash if cgroup memory is not
* Documentation: Various improvments
* Vagrant: Use only one deb line in /etc/apt
#### Runtime
-Fix the command existance check
-strings.Split may return an empty string on no match
- Fix an index out of range crash if cgroup memory is not
#### Documentation
* Various improvments
+ New example: sharing data between 2 couchdb databases
#### Other
* Vagrant: Use only one deb line in /etc/apt
+ Registry: Implement the new registry
## 0.2.2 (2013-05-03)
+ Support for data volumes ('docker run -v=PATH')
+ Share data volumes between containers ('docker run -volumes-from')
+ Improved documentation
* Upgrade to Go 1.0.3
*Various upgrades to the dev environment for contributors
+ Support for data volumes ('docker run -v=PATH')
+ Share data volumes between containers ('docker run -volumes-from')
+ Improved documentation
*Upgrade to Go 1.0.3
* Various upgrades to the dev environment for contributors
## 0.2.1 (2013-05-01)
+ 'docker commit -run' bundles a layer with default runtime options: command, ports etc.
* Improve install process on Vagrant
+ New Dockerfile operation: "maintainer"
+ New Dockerfile operation: "expose"
+ New Dockerfile operation: "cmd"
+ Contrib script to build a Debian base layer
+ 'docker -d -r': restart crashed containers at daemon startup
* Runtime: improve test coverage
+ 'docker commit -run' bundles a layer with default runtime options: command, ports etc.
* Improve install process on Vagrant
+ New Dockerfile operation: "maintainer"
+ New Dockerfile operation: "expose"
+ New Dockerfile operation: "cmd"
+ Contrib script to build a Debian base layer
+ 'docker -d -r': restart crashed containers at daemon startup
* Runtime: improve test coverage
## 0.2.0 (2013-04-23)
- Runtime: ghost containers can be killed and waited for
* Documentation: update install intructions
- Packaging: fix Vagrantfile
-Development: automate releasing binaries and ubuntu packages
+ Add a changelog
- Various bugfixes
- Runtime: ghost containers can be killed and waited for
* Documentation: update install intructions
-Packaging: fix Vagrantfile
- Development: automate releasing binaries and ubuntu packages
+ Add a changelog
- Various bugfixes
## 0.1.8 (2013-04-22)
- Dynamically detect cgroup capabilities
-Issue stability warning on kernels <3.8
-'docker push' buffers on disk instead of memory
- Fix 'docker diff' for removed files
- Fix 'docker stop' for ghost containers
- Fix handling of pidfile
-Various bugfixes and stability improvements
-Dynamically detect cgroup capabilities
-Issue stability warning on kernels <3.8
- 'docker push' buffers on disk instead of memory
- Fix 'docker diff' for removed files
- Fix 'docker stop' for ghost containers
-Fix handling of pidfile
- Various bugfixes and stability improvements
## 0.1.7 (2013-04-18)
- Container ports are available on localhost
-'docker ps' shows allocated TCP ports
-Contributors can run 'make hack' to start a continuous integration VM
-Streamline ubuntu packaging & uploading
-Various bugfixes and stability improvements
-Container ports are available on localhost
-'docker ps' shows allocated TCP ports
-Contributors can run 'make hack' to start a continuous integration VM
-Streamline ubuntu packaging & uploading
- Various bugfixes and stability improvements
## 0.1.6 (2013-04-17)
- Record the author an image with 'docker commit -author'
- Record the author an image with 'docker commit -author'
## 0.1.5 (2013-04-17)
- Disable standalone mode
-Use a custom DNS resolver with 'docker -d -dns'
-Detect ghost containers
-Improve diagnosis of missing system capabilities
-Allow disabling memory limits at compile time
- Add debian packaging
-Documentation: installing on Arch Linux
- Documentation: running Redis on docker
-Fixed lxc 0.9 compatibility
-Automatically load aufs module
-Various bugfixes and stability improvements
-Disable standalone mode
-Use a custom DNS resolver with 'docker -d -dns'
-Detect ghost containers
-Improve diagnosis of missing system capabilities
- Allow disabling memory limits at compile time
-Add debian packaging
- Documentation: installing on Arch Linux
-Documentation: running Redis on docker
-Fixed lxc 0.9 compatibility
-Automatically load aufs module
- Various bugfixes and stability improvements
## 0.1.4 (2013-04-09)
- Full support for TTY emulation
- Detach from a TTY session with the escape sequence `C-p C-q`
-Various bugfixes and stability improvements
- Minor UI improvements
-Automatically create our own bridge interface 'docker0'
- Full support for TTY emulation
-Detach from a TTY session with the escape sequence `C-p C-q`
- Various bugfixes and stability improvements
-Minor UI improvements
- Automatically create our own bridge interface 'docker0'
## 0.1.3 (2013-04-04)
- Choose TCP frontend port with '-p :PORT'
-Layer format is versioned
-Major reliability improvements to the process manager
-Various bugfixes and stability improvements
-Choose TCP frontend port with '-p :PORT'
-Layer format is versioned
-Major reliability improvements to the process manager
- Various bugfixes and stability improvements
## 0.1.2 (2013-04-03)
- Set container hostname with 'docker run -h'
- Selective attach at run with 'docker run -a [stdin[,stdout[,stderr]]]'
- Various bugfixes and stability improvements
- UI polish
-Progress bar on push/pull
-Use XZ compression by default
-Make IP allocator lazy
- Set container hostname with 'docker run -h'
- Selective attach at run with 'docker run -a [stdin[,stdout[,stderr]]]'
A common method for distributing applications and sandbox their execution is to use virtual machines, or VMs. Typical VM formats
are VMWare's vmdk, Oracle Virtualbox's vdi, and Amazon EC2's ami. In theory these formats should allow every developer to
automatically package their application into a "machine" for easy distribution and deployment. In practice, that almost never
happens, for a few reasons:
A common method for distributing applications and sandbox their
execution is to use virtual machines, or VMs. Typical VM formats are
VMWare's vmdk, Oracle Virtualbox's vdi, and Amazon EC2's ami. In
theory these formats should allow every developer to automatically
package their application into a "machine" for easy distribution and
deployment. In practice, that almost never happens, for a few reasons:
* *Size*: VMs are very large which makes them impractical to store and transfer.
* *Performance*: running VMs consumes significant CPU and memory, which makes them impractical in many scenarios, for example local development of multi-tier applications, and
large-scale deployment of cpu and memory-intensive applications on large numbers of machines.
* *Portability*: competing VM environments don't play well with each other. Although conversion tools do exist, they are limited and add even more overhead.
* *Hardware-centric*: VMs were designed with machine operators in mind, not software developers. As a result, they offer very limited tooling for what developers need most:
building, testing and running their software. For example, VMs offer no facilities for application versioning, monitoring, configuration, logging or service discovery.
* *Size*: VMs are very large which makes them impractical to store
and transfer.
* *Performance*: running VMs consumes significant CPU and memory,
which makes them impractical in many scenarios, for example local
development of multi-tier applications, and large-scale deployment
of cpu and memory-intensive applications on large numbers of
machines.
* *Portability*: competing VM environments don't play well with each
other. Although conversion tools do exist, they are limited and
add even more overhead.
* *Hardware-centric*: VMs were designed with machine operators in
mind, not software developers. As a result, they offer very
limited tooling for what developers need most: building, testing
and running their software. For example, VMs offer no facilities
for application versioning, monitoring, configuration, logging or
service discovery.
By contrast, Docker relies on a different sandboxing method known as*containerization*. Unlike traditional virtualization,
containerization takes place at the kernel level. Most modern operating system kernels now support the primitives necessary
for containerization, including Linux with [openvz](http://openvz.org), [vserver](http://linux-vserver.org) and more recently [lxc](http://lxc.sourceforge.net),
Solaris with [zones](http://docs.oracle.com/cd/E26502_01/html/E29024/preface-1.html#scrolltoc) and FreeBSD with [Jails](http://www.freebsd.org/doc/handbook/jails.html).
By contrast, Docker relies on a different sandboxing method known as
*containerization*. Unlike traditional virtualization,
containerization takes place at the kernel level. Most modern
operating system kernels now support the primitives necessary for
containerization, including Linux with [openvz](http://openvz.org),
[vserver](http://linux-vserver.org) and more recently
Docker builds on top of these low-level primitives to offer developers a portable format and runtime environment that solves
all 4 problems. Docker containers are small (and their transfer can be optimized with layers), they have basically zero memory and cpu overhead,
they are completely portable and are designed from the ground up with an application-centric design.
Docker builds on top of these low-level primitives to offer developers
a portable format and runtime environment that solves all 4
problems. Docker containers are small (and their transfer can be
optimized with layers), they have basically zero memory and cpu
overhead, they are completely portable and are designed from the
ground up with an application-centric design.
The best part: because docker operates at the OS level, it can still be run inside a VM!
The best part: because ``docker`` operates at the OS level, it can
still be run inside a VM!
## Plays well with others
Docker does not require that you buy into a particular programming language, framework, packaging system or configuration language.
Docker does not require that you buy into a particular programming
language, framework, packaging system or configuration language.
Is your application a unix process? Does it use files, tcp connections, environment variables, standard unix streams and command-line
arguments as inputs and outputs? Then docker can run it.
Is your application a Unix process? Does it use files, tcp
connections, environment variables, standard Unix streams and
command-line arguments as inputs and outputs? Then ``docker`` can run
it.
Can your application's build be expressed as a sequence of such commands? Then docker can build it.
Can your application's build be expressed as a sequence of such
commands? Then ``docker`` can build it.
## Escape dependency hell
A common problem for developers is the difficulty of managing all their application's dependencies in a simple and automated way.
A common problem for developers is the difficulty of managing all
their application's dependencies in a simple and automated way.
This is usually difficult for several reasons:
* *Cross-platform dependencies*. Modern applications often depend on a combination of system libraries and binaries, language-specific packages, framework-specific modules,
internal components developed for another project, etc. These dependencies live in different "worlds" and require different tools - these tools typically don't work
well with each other, requiring awkward custom integrations.
* *Cross-platform dependencies*. Modern applications often depend on
a combination of system libraries and binaries, language-specific
developed for another project, etc. These dependencies live in
different "worlds" and require different tools - these tools
typically don't work well with each other, requiring awkward
custom integrations.
* Conflicting dependencies. Different applications may depend on different versions of the same dependency. Packaging tools handle these situations with various degrees of ease -
but they all handle them in different and incompatible ways, which again forces the developer to do extra work.
* Conflicting dependencies. Different applications may depend on
different versions of the same dependency. Packaging tools handle
these situations with various degrees of ease - but they all
handle them in different and incompatible ways, which again forces
the developer to do extra work.
* Custom dependencies. A developer may need to prepare a custom version of his application's dependency. Some packaging systems can handle custom versions of a dependency,
others can't - and all of them handle it differently.
* Custom dependencies. A developer may need to prepare a custom
version of their application's dependency. Some packaging systems
can handle custom versions of a dependency, others can't - and all
of them handle it differently.
Docker solves dependency hell by giving the developer a simple way to express *all* his application's dependencies in one place,
and streamline the process of assembling them. If this makes you think of [XKCD 927](http://xkcd.com/927/), don't worry. Docker doesn't
*replace* your favorite packaging systems. It simply orchestrates their use in a simple and repeatable way. How does it do that? With layers.
Docker solves dependency hell by giving the developer a simple way to
express *all* their application's dependencies in one place, and
streamline the process of assembling them. If this makes you think of
*replace* your favorite packaging systems. It simply orchestrates
their use in a simple and repeatable way. How does it do that? With
layers.
Docker defines a build as running a sequence of unix commands, one after the other, in the same container. Build commands modify the contents of the container
(usually by installing new files on the filesystem), the next command modifies it some more, etc. Since each build command inherits the result of the previous
commands, the *order* in which the commands are executed expresses *dependencies*.
Docker defines a build as running a sequence of Unix commands, one
after the other, in the same container. Build commands modify the
contents of the container (usually by installing new files on the
filesystem), the next command modifies it some more, etc. Since each
build command inherits the result of the previous commands, the
*order* in which the commands are executed expresses *dependencies*.
Here's a typical docker build process:
Here's a typical Docker build process:
```bash
from ubuntu:12.10
@@ -87,288 +136,66 @@ run curl -L https://github.com/shykes/helloflask/archive/master.tar.gz | tar -xz
run cd helloflask-master && pip install -r requirements.txt
```
Note that Docker doesn't care *how* dependencies are built - as long as they can be built by running a unix command in a container.
Note that Docker doesn't care *how* dependencies are built - as long
as they can be built by running a Unix command in a container.
Install instructions
==================
Getting started
===============
Quick install on Ubuntu 12.04 and 12.10
---------------------------------------
Docker can be installed on your local machine as well as servers - both bare metal and virtualized.
It is available as a binary on most modern Linux systems, or as a VM on Windows, Mac and other systems.
```bash
curl get.docker.io | sh -x
```
We also offer an interactive tutorial for quickly learning the basics of using Docker.
Binary installs
----------------
Docker supports the following binary installation methods.
Note that some methods are community contributions and not yet officially supported.
For up-to-date install instructions and online tutorials, see the [Getting Started page](http://www.docker.io/gettingstarted/).
* [Ubuntu 12.04 and 12.10 (officially supported)](http://docs.docker.io/en/latest/installation/ubuntulinux/)
You can find a [list of real-world examples](http://docs.docker.io/en/latest/examples/) in the documentation.
Under the hood
--------------
Under the hood, Docker is built on the following components:
* The [cgroup](http://blog.dotcloud.com/kernel-secrets-from-the-paas-garage-part-24-c) and [namespacing](http://blog.dotcloud.com/under-the-hood-linux-kernels-on-dotcloud-part) capabilities of the Linux kernel;
* [AUFS](http://aufs.sourceforge.net/aufs.html), a powerful union filesystem with copy-on-write capabilities;
Run the `go install` command (above) to recompile docker.
What is a Standard Container?
=============================
Docker defines a unit of software delivery called a Standard Container. The goal of a Standard Container is to encapsulate a software component and all its dependencies in
a format that is self-describing and portable, so that any compliant runtime can run it without extra dependencies, regardless of the underlying machine and the contents of the container.
The spec for Standard Containers is currently a work in progress, but it is very straightforward. It mostly defines 1) an image format, 2) a set of standard operations, and 3) an execution environment.
A great analogy for this is the shipping container. Just like how Standard Containers are a fundamental unit of software delivery, shipping containers (http://bricks.argz.com/ins/7823-1/12) are a fundamental unit of physical delivery.
### 1. STANDARD OPERATIONS
Just like shipping containers, Standard Containers define a set of STANDARD OPERATIONS. Shipping containers can be lifted, stacked, locked, loaded, unloaded and labelled. Similarly, standard containers can be started, stopped, copied, snapshotted, downloaded, uploaded and tagged.
### 2. CONTENT-AGNOSTIC
Just like shipping containers, Standard Containers are CONTENT-AGNOSTIC: all standard operations have the same effect regardless of the contents. A shipping container will be stacked in exactly the same way whether it contains Vietnamese powder coffee or spare Maserati parts. Similarly, Standard Containers are started or uploaded in the same way whether they contain a postgres database, a php application with its dependencies and application server, or Java build artifacts.
### 3. INFRASTRUCTURE-AGNOSTIC
Both types of containers are INFRASTRUCTURE-AGNOSTIC: they can be transported to thousands of facilities around the world, and manipulated by a wide variety of equipment. A shipping container can be packed in a factory in Ukraine, transported by truck to the nearest routing center, stacked onto a train, loaded into a German boat by an Australian-built crane, stored in a warehouse at a US facility, etc. Similarly, a standard container can be bundled on my laptop, uploaded to S3, downloaded, run and snapshotted by a build server at Equinix in Virginia, uploaded to 10 staging servers in a home-made Openstack cluster, then sent to 30 production instances across 3 EC2 regions.
### 4. DESIGNED FOR AUTOMATION
Because they offer the same standard operations regardless of content and infrastructure, Standard Containers, just like their physical counterpart, are extremely well-suited for automation. In fact, you could say automation is their secret weapon.
Many things that once required time-consuming and error-prone human effort can now be programmed. Before shipping containers, a bag of powder coffee was hauled, dragged, dropped, rolled and stacked by 10 different people in 10 different locations by the time it reached its destination. 1 out of 50 disappeared. 1 out of 20 was damaged. The process was slow, inefficient and cost a fortune - and was entirely different depending on the facility and the type of goods.
Similarly, before Standard Containers, by the time a software component ran in production, it had been individually built, configured, bundled, documented, patched, vendored, templated, tweaked and instrumented by 10 different people on 10 different computers. Builds failed, libraries conflicted, mirrors crashed, post-it notes were lost, logs were misplaced, cluster updates were half-broken. The process was slow, inefficient and cost a fortune - and was entirely different depending on the language and infrastructure provider.
### 5. INDUSTRIAL-GRADE DELIVERY
There are 17 million shipping containers in existence, packed with every physical good imaginable. Every single one of them can be loaded onto the same boats, by the same cranes, in the same facilities, and sent anywhere in the World with incredible efficiency. It is embarrassing to think that a 30 ton shipment of coffee can safely travel half-way across the World in *less time* than it takes a software team to deliver its code from one datacenter to another sitting 10 miles away.
With Standard Containers we can put an end to that embarrassment, by making INDUSTRIAL-GRADE DELIVERY of software a reality.
Standard Container Specification
--------------------------------
(TODO)
### Image format
### Standard operations
* Copy
* Run
* Stop
* Wait
* Commit
* Attach standard streams
* List filesystem changes
* ...
### Execution environment
#### Root filesystem
#### Environment variables
#### Process arguments
#### Networking
#### Process namespacing
#### Resource limits
#### Process monitoring
#### Logging
#### Signals
#### Pseudo-terminal allocation
#### Security
*Brought to you courtesy of our legal counsel. For more context,
please see the Notice document.*
Use and transfer of Docker may be subject to certain restrictions by the
United States and other governments.
It is your responsibility to ensure that your use and/or transfer does not
violate applicable laws.
For more information, please see http://www.bis.doc.gov
Data volumes (issue #111) are a much-requested feature which trigger much discussion and debate. Below is the current authoritative spec for implementing data volumes.
This spec will be deprecated once the feature is fully implemented.
Discussion, requests, trolls, demands, offerings, threats and other forms of supplications concerning this spec should be addressed to Solomon here: https://github.com/dotcloud/docker/issues/111
### 1. Creating data volumes
At container creation, parts of a container's filesystem can be mounted as separate data volumes. Volumes are defined with the -v flag.
For example:
```bash
$ docker run -v /var/lib/postgres -v /var/log postgres /usr/bin/postgres
```
In this example, a new container is created from the 'postgres' image. At the same time, docker creates 2 new data volumes: one will be mapped to the container at /var/lib/postgres, the other at /var/log.
2 important notes:
1) Volumes don't have top-level names. At no point does the user provide a name, or is a name given to him. Volumes are identified by the path at which they are mounted inside their container.
2) The user doesn't choose the source of the volume. Docker only mounts volumes it created itself, in the same way that it only runs containers that it created itself. That is by design.
### 2. Sharing data volumes
Instead of creating its own volumes, a container can share another container's volumes. For example:
```bash
$ docker run --volumes-from $OTHER_CONTAINER_ID postgres /usr/local/bin/postgres-backup
```
In this example, a new container is created from the 'postgres' example. At the same time, docker will *re-use* the 2 data volumes created in the previous example. One volume will be mounted on the /var/lib/postgres of *both* containers, and the other will be mounted on the /var/log of both containers.
### 3. Under the hood
Docker stores volumes in /var/lib/docker/volumes. Each volume receives a globally unique ID at creation, and is stored at /var/lib/docker/volumes/ID.
At creation, volumes are attached to a single container - the source of truth for this mapping will be the container's configuration.
Mounting a volume consists of calling "mount --bind" from the volume's directory to the appropriate sub-directory of the container mountpoint. This may be done by Docker itself, or farmed out to lxc (which supports mount-binding) if possible.
### 4. Backups, transfers and other volume operations
Volumes sometimes need to be backed up, transfered between hosts, synchronized, etc. These operations typically are application-specific or site-specific, eg. rsync vs. S3 upload vs. replication vs...
Rather than attempting to implement all these scenarios directly, Docker will allow for custom implementations using an extension mechanism.
### 5. Custom volume handlers
Docker allows for arbitrary code to be executed against a container's volumes, to implement any custom action: backup, transfer, synchronization across hosts, etc.
Here's an example:
```bash
$ DB=$(docker run -d -v /var/lib/postgres -v /var/log postgres /usr/bin/postgres)
$ BACKUP_JOB=$(docker run -d --volumes-from $DB shykes/backuper /usr/local/bin/backup-postgres --s3creds=$S3CREDS)
$ docker wait$BACKUP_JOB
```
Congratulations, you just implemented a custom volume handler, using Docker's built-in ability to 1) execute arbitrary code and 2) share volumes between containers.
docker-build is a script to build docker images from source. It will be deprecated once the 'build' feature is incorporated into docker itself (See https://github.com/dotcloud/docker/issues/278)
Author: Solomon Hykes <solomon@dotcloud.com>
## Install
docker-builder requires:
1) A reasonably recent Python setup (tested on 2.7.2).
2) A running docker daemon at version 0.1.4 or more recent (http://www.docker.io/gettingstarted)
## Usage
First create a valid Changefile, which defines a sequence of changes to apply to a base image.
$ cat Changefile
# Start build from a know base image
from base:ubuntu-12.10
# Update ubuntu sources
run echo 'deb http://archive.ubuntu.com/ubuntu quantal main universe multiverse' > /etc/apt/sources.list
run apt-get update
# Install system packages
run DEBIAN_FRONTEND=noninteractive apt-get install -y -q git
run DEBIAN_FRONTEND=noninteractive apt-get install -y -q curl
run DEBIAN_FRONTEND=noninteractive apt-get install -y -q golang
# Insert files from the host (./myscript must be present in the current directory)
copy myscript /usr/local/bin/myscript
Run docker-build, and pass the contents of your Changefile as standard input.
$ IMG=$(./docker-build < Changefile)
This will take a while: for each line of the changefile, docker-build will:
1. Create a new container to execute the given command or insert the given file
2. Wait for the container to complete execution
3. Commit the resulting changes as a new image
4. Use the resulting image as the input of the next step
If all the steps succeed, the result will be an image containing the combined results of each build step.
You can trace back those build steps by inspecting the image's history:
$ docker history $IMG
ID CREATED CREATED BY
1e9e2045de86 A few seconds ago /bin/sh -c cat > /usr/local/bin/myscript; chmod +x /usr/local/bin/git
77db140aa62a A few seconds ago /bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -q golang
77db140aa62a A few seconds ago /bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -q curl
77db140aa62a A few seconds ago /bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -q git
83e85d155451 A few seconds ago /bin/sh -c apt-get update
bfd53b36d9d3 A few seconds ago /bin/sh -c echo 'deb http://archive.ubuntu.com/ubuntu quantal main universe multiverse' > /etc/apt/sources.list
base 2 weeks ago /bin/bash
27cf78414709 2 weeks ago
Note that your build started from 'base', as instructed by your Changefile. But that base image itself seems to have been built in 2 steps - hence the extra step in the history.
You can use this build technique to create any image you want: a database, a web application, or anything else that can be build by a sequence of unix commands - in other words, anything else.
return docker(["commit"] + (["-author", author] if author else []) + (["-run", json.dumps(run)] if run is not None else []) + [run_id]).read().rstrip()
def insert(base, src, dst, author=None):
print "COPY {} to {} in {}".format(src, dst, base)
The Index is responsible for centralizing information about:
- User accounts
- Checksums of the images
- Public namespaces
The Index has different components:
- Web UI
- Meta-data store (comments, stars, list public repositories)
- Authentication service
- Tokenization
The index is authoritative for those information.
We expect that there will be only one instance of the index, run and managed by dotCloud.
1.2 Registry
------------
- This is the REST API for the Docker Registry
- It stores the images and the graph for a set of repositories
- It does not have user accounts data
- It has no notion of user accounts or authorization
@@ -60,414 +39,462 @@ We expect that there will be multiple registries out there. To help to grasp the
The latter would only require two new commands in docker, e.g. “registryget” and “registryput”, wrapping access to the local filesystem (and optionally doing consistency checks). Authentication and authorization are then delegated to SSH (e.g. with public keys).
1.3 Docker
2. Endpoints
============
2.1 Images
----------
On top of being a runtime for LXC, Docker is the Registry client. It supports:
1. Contact the Index to know where I should download “samalba/busybox”
2. Index replies:
a. “samalba/busybox” is on Registry A
b. here are the checksums for “samalba/busybox” (for all layers)
c. token
3. Contact Registry A to receive the layers for “samalba/busybox” (all of them to the base image). Registry A is authoritative for “samalba/busybox” but keeps a copy of all inherited layers and serve them all from the same location.
4. registry contacts index to verify if token/user is allowed to download images
5. Index returns true/false lettings registry know if it should proceed or error out
6. Get the payload for all layers
get all of the tags for the given repo.
It’s possible to run docker pull \https://<registry>/repositories/samalba/busybox. In this case, docker bypasses the Index. However the security is not guaranteed (in case Registry A is corrupted) because there won’t be any checksum checks.
**Example Request**:
Currently registry redirects to s3 urls for downloads, going forward all downloads need to be streamed through the registry. The Registry will then abstract the calls to S3 by a top-level class which implements sub-classes for S3 and local storage.
..sourcecode::http
Token is only returned when the 'X-Docker-Token' header is sent with request.
Basic Auth is required to pull private repos. Basic auth isn't required for pulling public repos, but if one is provided, it needs to be valid and for an active account.
API (pulling repository foo/bar):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1. (Docker -> Index) GET /v1/repositories/foo/bar/images
**Headers**:
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
X-Docker-Token: true
**Action**:
(looking up the foo/bar in db and gets images and checksums for that repo (all if no tag is specified, if tag, only checksums for those tags) see part 4.4.1)
5. (Docker -> Registry) GET /v1/images/928374982374/ancestry
**Action**:
(for each image id returned in the registry, fetch /json + /layer)
..note::
If someone makes a second request, then we will always give a new token, never reuse tokens.
2.2 Push
--------
..image:: /static_files/docker_push_chart.png
1. Contact the index to allocate the repository name “samalba/busybox” (authentication required with user credentials)
2. If authentication works and namespace available, “samalba/busybox” is allocated and a temporary token is returned (namespace is marked as initialized in index)
3. Push the image on the registry (along with the token)
4. Registry A contacts the Index to verify the token (token must corresponds to the repository name)
5. Index validates the token. Registry A starts reading the stream pushed by docker and store the repository (with its images)
6. docker contacts the index to give checksums for upload images
..note::
**It’s possible not to use the Index at all!** In this case, a deployed version of the Registry is deployed to store and serve images. Those images are not authentified and the security is not guaranteed.
..note::
**Index can be replaced!** For a private Registry deployed, a custom Index can be used to serve and validate token according to different policies.
Docker computes the checksums and submit them to the Index at the end of the push. When a repository name does not have checksums on the Index, it means that the push is in progress (since checksums are submitted at the end).
API (pushing repos foo/bar):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1. (Docker -> Index) PUT /v1/repositories/foo/bar/
**Headers**:
Authorization: Basic sdkjfskdjfhsdkjfh==
X-Docker-Token: true
**Action**::
- in index, we allocated a new repository, and set to initialized
**Body**::
(The body contains the list of images that are going to be pushed, with empty checksums. The checksums will be set at the end of the push)::
If push fails and they need to start again, what happens in the index, there will already be a record for the namespace/name, but it will be initialized. Should we allow it, or mark as name already used? One edge case could be if someone pushes the same thing at the same time with two different shells.
If it's a retry on the Registry, Docker has a cookie (provided by the registry after token validation). So the Index won’t have to provide a new token.
- Authenticate a user as a repos owner (for a central referenced repository)
..sourcecode::http
3.1 Without an Index
--------------------
Using the Registry without the Index can be useful to store the images on a private network without having to rely on an external entity controlled by dotCloud.
PUT/v1/repositories/foo/bar/tags/latestHTTP/1.1
Host: registry-1.docker.io
Accept: application/json
Content-Type: application/json
Cookie: (Cookie provided by the Registry)
In this case, the registry will be launched in a special mode (--standalone? --no-index?). In this mode, the only thing which changes is that Registry will never contact the Index to verify a token. It will be the Registry owner responsibility to authenticate the user who pushes (or even pulls) an image using any mechanism (HTTP auth, IP based, etc...).
In this scenario, the Registry is responsible for the security in case of data corruption since the checksums are not delivered by a trusted entity.
:parameter namespace:namespace for the repo
:parameter repository:name for the repo
:parameter tag:name of tag you want to add
As hinted previously, a standalone registry can also be implemented by any HTTP server handling GET/PUT requests (or even only GET requests if no write access is necessary).
**Example Response**:
3.2 With an Index
-----------------
..sourcecode::http
The Index data needed by the Registry are simple:
- Serve the checksums
- Provide and authorize a Token
HTTP/1.1200
Vary: Accept
Content-Type: application/json
X-Docker-Registry-Version: 0.6.0
In the scenario of a Registry running on a private network with the need of centralizing and authorizing, it’s easy to use a custom Index.
""
The only challenge will be to tell Docker to contact (and trust) this custom Index. Docker will be configurable at some point to use a specific Index, it’ll be the private entity responsibility (basically the organization who uses Docker in a private environment) to maintain the Index and the Docker’s configuration among its consumers.
:statuscode 200:OK
:statuscode 400:Invalid data
:statuscode 401:Requires authorization
:statuscode 404:Image not found
4. The API
==========
2.3 Repositories
----------------
The first version of the api is available here: https://github.com/jpetazzo/docker/blob/acd51ecea8f5d3c02b00a08176171c59442df8b3/docs/images-repositories-push-pull.md
The format returned in the images is not defined here (for layer and json), basically because Registry stores exactly the same kind of information as Docker uses to manage them.
..http:get:: /v1/_ping
The format of ancestry is a line-separated list of image ids, in age order. I.e. the image’s parent is on the last line, the parent of the parent on the next-to-last line, etc.; if the image has no parent, the file is empty.
Check status of the registry. This endpoint is also used to determine if
- **username** : min 4 character, max 30 characters, all lowercase no special characters.
- **password**: min 5 characters
..sourcecode::http
**Valid**: return HTTP 200
HTTP/1.1200
Vary: Accept
Content-Type: application/json
X-Docker-Registry-Version: 0.6.0
Errors: HTTP 400 (we should create error codes for possible errors)
- invalid json
- missing field
- wrong format (username, password, email, etc)
- forbidden name
- name already exists
""
..note::
:statuscode 200:OK
A user account will be valid only if the email has been validated (a validation link is sent to the email address).
4.2.2 Update a user (Index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
3 Authorization
===============
This is where we describe the authorization process, including the tokens and cookies.
PUT /v1/users/<username>
**Body**:
{"password": "toto"}
..note::
We can also update email address, if they do, they will need to reverify their new email address.
4.2.3 Login (Index)
^^^^^^^^^^^^^^^^^^^
Does nothing else but asking for a user authentication. Can be used to validate credentials. HTTP Basic Auth for now, maybe change in future.
GET /v1/users
**Return**:
- Valid: HTTP 200
- Invalid login: HTTP 401
- Account inactive: HTTP 403 Account is not Active
4.3 Tags (Registry)
-------------------
The Registry does not know anything about users. Even though repositories are under usernames, it’s just a namespace for the registry. Allowing us to implement organizations or different namespaces per user later, without modifying the Registry’s API.
4.3.1 Get all tags
^^^^^^^^^^^^^^^^^^
GET /v1/repositories/<namespace>/<repository_name>/tags
For the Index to “resolve” the repository name to a Registry location, it uses the X-Docker-Endpoints header. In other terms, this requests always add a “X-Docker-Endpoints” to indicate the location of the registry which hosts this repository.
4.4.1 Get the images
^^^^^^^^^^^^^^^^^^^^^
GET /v1/repositories/<namespace>/<repo_name>/images
Usually, the Registry provides a Cookie when a Token verification succeeded. Every time the Registry passes a Cookie, you have to pass it back the same cookie.::
:description:Documentation for docker Registry and Registry API
:keywords:docker, registry, api, index
.._registryindexspec:
=====================
Registry & Index Spec
=====================
1. The 3 roles
===============
1.1 Index
---------
The Index is responsible for centralizing information about:
- User accounts
- Checksums of the images
- Public namespaces
The Index has different components:
- Web UI
- Meta-data store (comments, stars, list public repositories)
- Authentication service
- Tokenization
The index is authoritative for those information.
We expect that there will be only one instance of the index, run and managed by dotCloud.
1.2 Registry
------------
- It stores the images and the graph for a set of repositories
- It does not have user accounts data
- It has no notion of user accounts or authorization
- It delegates authentication and authorization to the Index Auth service using tokens
- It supports different storage backends (S3, cloud files, local FS)
- It doesn’t have a local database
- It will be open-sourced at some point
We expect that there will be multiple registries out there. To help to grasp the context, here are some examples of registries:
-**sponsor registry**: such a registry is provided by a third-party hosting infrastructure as a convenience for their customers and the docker community as a whole. Its costs are supported by the third party, but the management and operation of the registry are supported by dotCloud. It features read/write access, and delegates authentication and authorization to the Index.
-**mirror registry**: such a registry is provided by a third-party hosting infrastructure but is targeted at their customers only. Some mechanism (unspecified to date) ensures that public images are pulled from a sponsor registry to the mirror registry, to make sure that the customers of the third-party provider can “docker pull” those images locally.
-**vendor registry**: such a registry is provided by a software vendor, who wants to distribute docker images. It would be operated and managed by the vendor. Only users authorized by the vendor would be able to get write access. Some images would be public (accessible for anyone), others private (accessible only for authorized users). Authentication and authorization would be delegated to the Index. The goal of vendor registries is to let someone do “docker pull basho/riak1.3” and automatically push from the vendor registry (instead of a sponsor registry); i.e. get all the convenience of a sponsor registry, while retaining control on the asset distribution.
-**private registry**: such a registry is located behind a firewall, or protected by an additional security layer (HTTP authorization, SSL client-side certificates, IP address authorization...). The registry is operated by a private entity, outside of dotCloud’s control. It can optionally delegate additional authorization to the Index, but it is not mandatory.
..note::
Mirror registries and private registries which do not use the Index don’t even need to run the registry code. They can be implemented by any kind of transport implementing HTTP GET and PUT. Read-only registries can be powered by a simple static HTTP server.
..note::
The latter implies that while HTTP is the protocol of choice for a registry, multiple schemes are possible (and in some cases, trivial):
- HTTP with GET (and PUT for read-write registries);
- local mount point;
- remote docker addressed through SSH.
The latter would only require two new commands in docker, e.g. “registryget” and “registryput”, wrapping access to the local filesystem (and optionally doing consistency checks). Authentication and authorization are then delegated to SSH (e.g. with public keys).
1.3 Docker
----------
On top of being a runtime for LXC, Docker is the Registry client. It supports:
- Push / Pull on the registry
- Client authentication on the Index
2. Workflow
===========
2.1 Pull
--------
..image:: /static_files/docker_pull_chart.png
1. Contact the Index to know where I should download “samalba/busybox”
2. Index replies:
a. “samalba/busybox” is on Registry A
b. here are the checksums for “samalba/busybox” (for all layers)
c. token
3. Contact Registry A to receive the layers for “samalba/busybox” (all of them to the base image). Registry A is authoritative for “samalba/busybox” but keeps a copy of all inherited layers and serve them all from the same location.
4. registry contacts index to verify if token/user is allowed to download images
5. Index returns true/false lettings registry know if it should proceed or error out
6. Get the payload for all layers
It’s possible to run docker pull \https://<registry>/repositories/samalba/busybox. In this case, docker bypasses the Index. However the security is not guaranteed (in case Registry A is corrupted) because there won’t be any checksum checks.
Currently registry redirects to s3 urls for downloads, going forward all downloads need to be streamed through the registry. The Registry will then abstract the calls to S3 by a top-level class which implements sub-classes for S3 and local storage.
Token is only returned when the 'X-Docker-Token' header is sent with request.
Basic Auth is required to pull private repos. Basic auth isn't required for pulling public repos, but if one is provided, it needs to be valid and for an active account.
API (pulling repository foo/bar):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1. (Docker -> Index) GET /v1/repositories/foo/bar/images
**Headers**:
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
X-Docker-Token: true
**Action**:
(looking up the foo/bar in db and gets images and checksums for that repo (all if no tag is specified, if tag, only checksums for those tags) see part 4.4.1)
5. (Docker -> Registry) GET /v1/images/928374982374/ancestry
**Action**:
(for each image id returned in the registry, fetch /json + /layer)
..note::
If someone makes a second request, then we will always give a new token, never reuse tokens.
2.2 Push
--------
..image:: /static_files/docker_push_chart.png
1. Contact the index to allocate the repository name “samalba/busybox” (authentication required with user credentials)
2. If authentication works and namespace available, “samalba/busybox” is allocated and a temporary token is returned (namespace is marked as initialized in index)
3. Push the image on the registry (along with the token)
4. Registry A contacts the Index to verify the token (token must corresponds to the repository name)
5. Index validates the token. Registry A starts reading the stream pushed by docker and store the repository (with its images)
6. docker contacts the index to give checksums for upload images
..note::
**It’s possible not to use the Index at all!** In this case, a deployed version of the Registry is deployed to store and serve images. Those images are not authenticated and the security is not guaranteed.
..note::
**Index can be replaced!** For a private Registry deployed, a custom Index can be used to serve and validate token according to different policies.
Docker computes the checksums and submit them to the Index at the end of the push. When a repository name does not have checksums on the Index, it means that the push is in progress (since checksums are submitted at the end).
API (pushing repos foo/bar):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1. (Docker -> Index) PUT /v1/repositories/foo/bar/
**Headers**:
Authorization: Basic sdkjfskdjfhsdkjfh==
X-Docker-Token: true
**Action**::
- in index, we allocated a new repository, and set to initialized
**Body**::
(The body contains the list of images that are going to be pushed, with empty checksums. The checksums will be set at the end of the push)::
If push fails and they need to start again, what happens in the index, there will already be a record for the namespace/name, but it will be initialized. Should we allow it, or mark as name already used? One edge case could be if someone pushes the same thing at the same time with two different shells.
If it's a retry on the Registry, Docker has a cookie (provided by the registry after token validation). So the Index won’t have to provide a new token.
2.3 Delete
----------
If you need to delete something from the index or registry, we need a nice clean way to do that. Here is the workflow.
1. Docker contacts the index to request a delete of a repository “samalba/busybox” (authentication required with user credentials)
2. If authentication works and repository is valid, “samalba/busybox” is marked as deleted and a temporary token is returned
3. Send a delete request to the registry for the repository (along with the token)
4. Registry A contacts the Index to verify the token (token must corresponds to the repository name)
5. Index validates the token. Registry A deletes the repository and everything associated to it.
6. docker contacts the index to let it know it was removed from the registry, the index removes all records from the database.
..note::
The Docker client should present an "Are you sure?" prompt to confirm the deletion before starting the process. Once it starts it can't be undone.
- Authenticate a user as a repos owner (for a central referenced repository)
3.1 Without an Index
--------------------
Using the Registry without the Index can be useful to store the images on a private network without having to rely on an external entity controlled by dotCloud.
In this case, the registry will be launched in a special mode (--standalone? --no-index?). In this mode, the only thing which changes is that Registry will never contact the Index to verify a token. It will be the Registry owner responsibility to authenticate the user who pushes (or even pulls) an image using any mechanism (HTTP auth, IP based, etc...).
In this scenario, the Registry is responsible for the security in case of data corruption since the checksums are not delivered by a trusted entity.
As hinted previously, a standalone registry can also be implemented by any HTTP server handling GET/PUT requests (or even only GET requests if no write access is necessary).
3.2 With an Index
-----------------
The Index data needed by the Registry are simple:
- Serve the checksums
- Provide and authorize a Token
In the scenario of a Registry running on a private network with the need of centralizing and authorizing, it’s easy to use a custom Index.
The only challenge will be to tell Docker to contact (and trust) this custom Index. Docker will be configurable at some point to use a specific Index, it’ll be the private entity responsibility (basically the organization who uses Docker in a private environment) to maintain the Index and the Docker’s configuration among its consumers.
4. The API
==========
The first version of the api is available here: https://github.com/jpetazzo/docker/blob/acd51ecea8f5d3c02b00a08176171c59442df8b3/docs/images-repositories-push-pull.md
4.1 Images
----------
The format returned in the images is not defined here (for layer and json), basically because Registry stores exactly the same kind of information as Docker uses to manage them.
The format of ancestry is a line-separated list of image ids, in age order. I.e. the image’s parent is on the last line, the parent of the parent on the next-to-last line, etc.; if the image has no parent, the file is empty.
- **username**: min 4 character, max 30 characters, must match the regular
expression [a-z0-9\_].
- **password**: min 5 characters
**Valid**: return HTTP 200
Errors: HTTP 400 (we should create error codes for possible errors)
- invalid json
- missing field
- wrong format (username, password, email, etc)
- forbidden name
- name already exists
..note::
A user account will be valid only if the email has been validated (a validation link is sent to the email address).
4.2.2 Update a user (Index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
PUT /v1/users/<username>
**Body**:
{"password": "toto"}
..note::
We can also update email address, if they do, they will need to reverify their new email address.
4.2.3 Login (Index)
^^^^^^^^^^^^^^^^^^^
Does nothing else but asking for a user authentication. Can be used to validate credentials. HTTP Basic Auth for now, maybe change in future.
GET /v1/users
**Return**:
- Valid: HTTP 200
- Invalid login: HTTP 401
- Account inactive: HTTP 403 Account is not Active
4.3 Tags (Registry)
-------------------
The Registry does not know anything about users. Even though repositories are under usernames, it’s just a namespace for the registry. Allowing us to implement organizations or different namespaces per user later, without modifying the Registry’s API.
The following naming restrictions apply:
- Namespaces must match the same regular expression as usernames (See 4.2.1.)
- Repository names must match the regular expression [a-zA-Z0-9-_.]
4.3.1 Get all tags
^^^^^^^^^^^^^^^^^^
GET /v1/repositories/<namespace>/<repository_name>/tags
For the Index to “resolve” the repository name to a Registry location, it uses the X-Docker-Endpoints header. In other terms, this requests always add a “X-Docker-Endpoints” to indicate the location of the registry which hosts this repository.
4.4.1 Get the images
^^^^^^^^^^^^^^^^^^^^^
GET /v1/repositories/<namespace>/<repo_name>/images
Usually, the Registry provides a Cookie when a Token verification succeeded. Every time the Registry passes a Cookie, you have to pass it back the same cookie.::
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Victor Vieux