This patch adds the needed pkgs to docker/docker development Dockerfile
in order to test the clr execdriver in the development container.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
- the /etc/hosts read caveat due to dynamic update
- information about docker_gwbridge
- Carries and closes#17654
- Updating with last change by Madhu
- Updating with the IPAM api 1.22
Signed-off-by: Mary Anthony <mary@docker.com>
(cherry picked from commit 39dfc536d444df8745124ffa38504241a0a985f7)
Conflicts:
docs/reference/api/docker_remote_api_v1.22.md
updating after assignment for Nigel
Adding in some notes from Nigel work
Updating with the storage driver content Nigel added
Updating with Nigel's polishing tech
Adding in Nigel graphics
First pass of aufs material
Capturing Nigel's latest
Comments back to Nigel on devicemapper
Incorporating Nigel's comments v3
Converting images for dm
Entering comments into aufs page
Adding the btfs storage driver
Moving to userguide
Adding in two new driver articles from Nigel
Optimized images
Updating with comments
Signed-off-by: Mary Anthony <mary@docker.com>
(cherry picked from commit 950fbf99b1214e77c0dcc27e8f9a16e01b2c630a)
Removing old networking.md
Updating dockernetworks.md with images
Adding information on network plugins
Adding blurb about links to docker networking
Updating the working documentation
Adding Overlay Getting Started
Downplaying links by removing refs/examples, adding refs/examples for network.
Updating getting started to reflect networks not links
Pulling out old network material
Updating per discussion with Madhu to add Default docs section
Updating with bridge default
Fix bad merge
Updating with new cluster-advertise behavior
Update working and NetworkSettings examples
Correcting example for default bridge discovery behavior
Entering comments
Fixing broken Markdown Syntax
Updating with comments
Updating all the links
Signed-off-by: Mary Anthony <mary@docker.com>
(cherry picked from commit 9ef855f9e5fa8077468bda5ce43155318c58e60e)
Fixing the weight
Signed-off-by: Mary Anthony <mary@docker.com>
(cherry picked from commit 5ce093e94565a5f32df9adf78dcdb6deeda75f79)
Conflicts:
docs/reference/api/docker_remote_api.md
docs/reference/api/docker_remote_api_v1.21.md
docs/reference/api/docker_remote_api_v1.22.md
For graceful restart case it was done when the container was brought
down. But for ungraceful cases, the persistence is missing for nw
connect
Signed-off-by: Madhu Venugopal <madhu@docker.com>
(cherry picked from commit 401632c7568408ee9689adc1da855cfb1409e906)
This fixes a bug introduced in #15786:
* if a pre-v1.20 client requested docker stats, the daemon
would return both an API-compatible JSON blob *and* an API-incompatible JSON
blob: see https://gist.github.com/donhcd/338a5b3681cd6a071629
Signed-off-by: Donald Huang <don.hcd@gmail.com>
(cherry picked from commit d2c04f844b8258d712da4b8feac25df7590b037c)
The commit title wrongfully mentioned API v1.22, when it meant to mention v1.21.
- So that it complies with docker convention for inspect
Signed-off-by: Alessandro Boch <aboch@docker.com>
(cherry picked from commit d795bc7d53db4b9e9194d0570d65e22d39f58384)
- Keep old fields in NetworkSetting to respect the deprecation policy.
Signed-off-by: David Calavera <david.calavera@gmail.com>
(cherry picked from commit f301c5765a0d7f4b6866cedfdface6f87874ff53)
- So that they comply with docker inspect convention
Which is allowing camel case for json field names
Signed-off-by: Alessandro Boch <aboch@docker.com>
(cherry picked from commit b2d0b75018c5be5c671a15950546a79cda307cf2)
You cannot do this for individual cgroups for all the containers. Only
set the reservation if the user requested it. The error you will
receive is an EINTVAL when you try to set a large limit like we were in
the memory limit.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
(cherry picked from commit ecb87ed0a5128a3d9ab3b0a620463b7b56e82bab)
When a container restarts after a ungraceful daemon restart, first
cleanup any unclean sandbox before trying to allocate network resources.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
(cherry picked from commit 0c07096b7da6dc28df10df21283fc21b3b052e1a)
That way swarm can understand the user's intention.
Signed-off-by: David Calavera <david.calavera@gmail.com>
(cherry picked from commit 34668ad68bb42d7488649e41b32818f3f1a7cff2)
- As the retrieved info may not be available at
Endpoint creation time for certain network drivers
- Also retrieve the MAC address from Endpoint.Info().Iface()
Signed-off-by: Alessandro Boch <aboch@docker.com>
(cherry picked from commit e03daebb48221bae84bfc48b0f9b652ced6d93f3)
When an image has multiple tags and rmi is called with force on a tag, only the single tag should be removed.
The current behavior is broken and removes all tags and the image.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
(cherry picked from commit 48e7f7963e142a0a45b583c5e65015d896c59433)
Added Note to show users that signals will not propagate to the container if the preferred exec form isn't used.
Signed-off-by: Eric Rosenberg <ehaydenr@gmail.com>
(cherry picked from commit c1a5ee53c19d62918f5ef245b5d2268fb9c89ca8)
--cluster-advertise daemon option is enahanced to support <interface-name>
in addition to <ip-address> in order to amke it automation friendly using
docker-machine.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Conflicts:
integration-cli/docker_cli_info_test.go
- Only user named containers will be published into
other containers' etc/hosts file.
- Also block linking to containers which are not
connected to the default network
Signed-off-by: Alessandro Boch <aboch@docker.com>
Adding Related information blocks
Final first draft pass: ready for review
Review comments
Entering comments from the gang
Updating connect to include paused
Signed-off-by: Mary Anthony <mary@docker.com>
While creating multiple containers the second
container could remove the first one from graph
and not produce an error.
Fixes#15995
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Without this buffering, the compressor was outputting 64 bytes at a
time to the HTTP stream, which was resulting in absurdly small chunk
sizes and a lot of extra overhead. The buffering restores the chunk size
to 32768 bytes, which matches the behavior with 1.8.2.
Times pushing to a local registry:
1.8.2: 0m18.934s
master: 0m20.564s
master+this commit: 0m17.593s
Fixes: #17038
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
- Do not execute the template directly in the cli outout, go is not atomic
in this operation and can send bytes before failing the execution.
- Fail after evaluating a raw interface if the typed execution also
failed, assuming there is a template parsing error.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This carries fixes for
- Internal racy /etc/hosts updates within container during SD
- Renable SD service record watch after cluster-store restarts
- Fix to allow remote IPAM driver to return no IP if the user prefers
- Fix to allow --fixed-cidr and --bip to be in same range
Signed-off-by: Madhu Venugopal <madhu@docker.com>
PR #15493 removed compression of layers when pushing them to a V2
registry. This this makes layer uploads larger than they should be.
This commit restores the compression. It uses an io.Pipe to turn the
gzip compressor output Writer into a Reader, so the ReadFrom method can
be used on the BlobWriter (which is very important for avoiding many
PATCH requests per layer).
Fixes#17209Fixes#17038
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Also remove unused func `newTempFile` and prevent a possible deadlock
between pull_v2 `attemptIDReuse` and graph `register`
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Lookup the graph parent reference to detect a builder cache miss before
looping the whole graph image index to build a parent-children tree.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Delegate validation tasks to the volume drivers. It's up to them
to decide whether a name is valid or not.
Restrict volume names for the local driver to prevent creating
mount points outside docker's volumes directory.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Since #15862, containers fail to start when started with --ipc=host if
/dev/mqueue is not present. This change causes docker to create
container-local mounts for --ipc=host containers as well as in the
default case.
Signed-off-by: Burke Libbey <burke.libbey@shopify.com>
To be consistent with other inspect command (on container and images),
add the possiblity to pass multiple network to the network inspect
commands.
`docker network inspect host bridge none` is possible now.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
After v1.8.3 layer checksum is used for image ID
validation. Rewriting the checksums on push would
mean that next pulls will get different image IDs
and pulls may fail if its detected that same
manifest digest can now point to new image ID.
Fixes#17178
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
cleanupDeleted() takes devices.Lock() but does not drop it if there are
no deleted devices. Hence docker deadlocks if one is using deferred
device deletion feature. (--storage-opt dm.use_deferred_deletion=true).
Fix it. Drop the lock before returning.
Also added a unit test case to make sure in future this can be easily
detected if somebody changes the function.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
This will prevent the api from trying to serve network requests in
systems where libnetwork is not enabled, returning 404 responses in any
case.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Ensure v2 registries are given more than 5 seconds to return a ping and avoid an unnecessary fallback to v1.
Elevates log level about failed v2 ping to a warning to match the warning related to using v1 registries.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Instead of returning only the container ID, starting a container may
also return a warning:
"WARNING: Your kernel does not support swap
limit capabilities, memory limited without
swap.\nff6ebd9f7a8d035d17bb9a61eb9d3f0a5d563160cc43471a9d7ac9f71945d061"
The test assumes that only the container ID is returned and uses the
entire message as the name for the inspect command. To avoid the need to
parse the container ID from the output after the run command, give the
container a name and use that instead.
Signed-off-by: Christy Perez <christy@linux.vnet.ibm.com>
`service` is deprecated from Fedora v21.
It's important to enable and start the `docker` daemon using `systemctl`.
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
This fixes the indentation of the storage driver
options list.
Also wraps/reformats some examples to prevent
horizontal scrollbars on the rendered HTML
Fixes#17140
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The race is between pools.Put which calls buf.Reset and exec.Cmd
doing io.Copy from the buffer; it caused a runtime crash, as
described in #16924:
``` docker-daemon cat the-tarball.xz | xz -d -c -q | docker-untar /path/to/... (aufs ) ```
When docker-untar side fails (like try to set xattr on aufs, or a broken
tar), invokeUnpack will be responsible to exhaust all input, otherwise
`xz` will be write pending for ever.
this change add a receive only channel to cmdStream, and will close it
to notify it's now safe to close the input stream;
in CmdStream the change to use Stdin / Stdout / Stderr keeps the
code simple, os/exec.Cmd will spawn goroutines and call io.Copy automatically.
the CmdStream is actually called in the same file only, change it
lowercase to mark as private.
[...]
INFO[0000] Docker daemon commit=0a8c2e3 execdriver=native-0.2 graphdriver=aufs version=1.8.2
DEBU[0006] Calling POST /build
INFO[0006] POST /v1.20/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=gentoo-x32&ulimits=null
DEBU[0008] [BUILDER] Cache miss
DEBU[0009] Couldn't untar /home/lib-docker-v1.8.2-tmp/tmp/docker-build316710953/stage3-x32-20151004.tar.xz to /home/lib-docker-v1.8.2-tmp/aufs/mnt/d909abb87150463939c13e8a349b889a72d9b14f0cfcab42a8711979be285537: Untar re-exec error: exit status 1: output: operation not supported
DEBU[0009] CopyFileWithTar(/home/lib-docker-v1.8.2-tmp/tmp/docker-build316710953/stage3-x32-20151004.tar.xz, /home/lib-docker-v1.8.2-tmp/aufs/mnt/d909abb87150463939c13e8a349b889a72d9b14f0cfcab42a8711979be285537/)
panic: runtime error: slice bounds out of range
goroutine 42 [running]:
bufio.(*Reader).fill(0xc208187800)
/usr/local/go/src/bufio/bufio.go:86 +0x2db
bufio.(*Reader).WriteTo(0xc208187800, 0x7ff39602d150, 0xc2083f11a0, 0x508000, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:449 +0x27e
io.Copy(0x7ff39602d150, 0xc2083f11a0, 0x7ff3960261f8, 0xc208187800, 0x0, 0x0, 0x0)
/usr/local/go/src/io/io.go:354 +0xb2
github.com/docker/docker/pkg/archive.func·006()
/go/src/github.com/docker/docker/pkg/archive/archive.go:817 +0x71
created by github.com/docker/docker/pkg/archive.CmdStream
/go/src/github.com/docker/docker/pkg/archive/archive.go:819 +0x1ec
goroutine 1 [chan receive]:
main.(*DaemonCli).CmdDaemon(0xc20809da30, 0xc20800a020, 0xd, 0xd, 0x0, 0x0)
/go/src/github.com/docker/docker/docker/daemon.go:289 +0x1781
reflect.callMethod(0xc208140090, 0xc20828fce0)
/usr/local/go/src/reflect/value.go:605 +0x179
reflect.methodValueCall(0xc20800a020, 0xd, 0xd, 0x1, 0xc208140090, 0x0, 0x0, 0xc208140090, 0x0, 0x45343f, ...)
/usr/local/go/src/reflect/asm_amd64.s:29 +0x36
github.com/docker/docker/cli.(*Cli).Run(0xc208129fb0, 0xc20800a010, 0xe, 0xe, 0x0, 0x0)
/go/src/github.com/docker/docker/cli/cli.go:89 +0x38e
main.main()
/go/src/github.com/docker/docker/docker/docker.go:69 +0x428
goroutine 5 [syscall]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
/usr/local/go/src/os/signal/signal_unix.go:27 +0x35
Signed-off-by: Derek Ch <denc716@gmail.com>
When running LXC dind (outer docker is started with native driver)
cgroup paths point to `/docker/CID` inside `/proc/self/mountinfo` but
these paths aren't mounted (root is wrong). This fix just discard the
cgroup dir from mountinfo and set it to root `/`.
This patch fixes/skip OOM LXC tests that were failing.
Fix#16520
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Signed-off-by: Antonio Murdaca <amurdaca@redhat.com>
On LXC memory swap was only set to memory_limit*2 even if a value for
memory swap was provided. This patch fix this behavior to be the same
as the native driver and set correct memory swap in the template.
Also add a test specifically for LXC but w/o adding a new test
requirement.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
The latest libkv uses a different etcd library. Unfortunately
that library uses some funky import paths, so I've added a new cleanup
routine for our vendor scripts to be able to normalize the imports
to be consistent with how imports work in this tree.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
introduced --subnet, --ip-range and --gateway options in docker network
command. Also, user can allocate driver specific ip-address if any using
the --aux-address option.
Supports multiple subnets per network and also sharing ip range
across networks if the network-driver and ipam-driver supports it.
Example, Bridge driver doesnt support sharing same ip range across
networks.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
* Made use of IPAM driver primitives for legacy IP configurations
* Replaced custom Generics with backend labels
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Tags and digests are kept in the same storage. We want to make sure that they are completely separated - tags are something users set and digests can only be set by pull-by-digest code path.
Reverts #14664
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
It is possible that network files do not exist, especially in the case
of `--net=host` where a host OS (like CoreOS) does not use certain
standard network files. This patch verifies that the source file of a
network mount point exists before adding it to the list of mount points
for bind mounting from the container's metadata directory.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
this allows jsonfile logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`.
Extra attributes are saved into `attrs` attributes for each log data.
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
this allows journald logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
this allows fluentd logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
this allows gelf logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`
Additional log field will be prefixed with `_` as per gelf protocol
https://www.graylog.org/resources/gelf/
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
Add a unit test for validateManifest which ensures extra data can't be
injected by adding data to the JSON object outside the payload area.
This also removes validation of legacy signatures at pull time. This
starts the path of deprecating legacy signatures, whose presence in the
very JSON document they attempt to sign is problematic. These
signatures were only checked for official images, and since they only
caused a weakly-worded message to be printed, removing the verification
should not cause impact.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Generate a hash chain involving the image configuration, layer digests,
and parent image hashes. Use the digests to compute IDs for each image
in a manifest, instead of using the remotely specified IDs.
To avoid breaking users' caches, check for images already in the graph
under old IDs, and avoid repulling an image if the version on disk under
the legacy ID ends up with the same digest that was computed from the
manifest for that image.
When a calculated ID already exists in the graph but can't be verified,
continue trying SHA256(digest) until a suitable ID is found.
"save" and "load" are not changed to use a similar scheme. "load" will
preserve the IDs present in the tar file.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Docker 1.7 got support for specifying the CFS scheduler's CPU quota and
period. The work appears to have started around commit
dcc50e1d59. This change updates the API
documentation to reflect the existing support.
Signed-off-by: Victor Costan <costan@gmail.com>
All the go-lint work forced any existing "Uid" -> "UID", but seems to
not have the same rules for Gid, so stat package has calls UID() and
Gid().
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
The first param on opts.ParseHost() wasn't being used for anything.
Once we get rid of that param we can then also clean-up some code
that calls ParseHost() because the param that was passed in wasn't
being used for anything else.
Signed-off-by: Doug Davis <dug@us.ibm.com>
`docker network` is the second command with subcommands.
This refactoring pulls out parsing and processing of subcommands
from `docker volume` completion and thus makes its logic available
for other commands.
Also enables `__docker_pos_first_nonflag` for subcommand completion.
Signed-off-by: Harald Albers <github@albersweb.de>
GitHub flavored markdown is now supported for links and images. Also, ran LinkChecker and FileResolver. Yay!
Fixes from Spider check
Output for docker/docker now goes into engine directory
Signed-off-by: Mary Anthony <mary@docker.com>
This adds the capability to turn on user namespace support when using an
experimental build Docker daemon binary using the `--userns-remap` flag.
Also documentation is added to the experimental docs.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Refactor so that the Host flag validation doesn't destroy the user's input,
and then post process the flags when we know the TLS options
Signed-off-by: Sven Dowideit <SvenDowideit@home.org.au>
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.
Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
The `pkg/idtools` package supports the creation of user(s) for
retrieving /etc/sub{u,g}id ranges and creation of the UID/GID mappings
provided to clone() to add support for user namespaces in Docker.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
- add daemon methods Authenticate & SearchRegistryForImages
- use new methods instead of directly accessing RegistryService
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
add needed clarity for
1) using STDIN to pass build context
2) --cpu-shares flag use
also a few typos
Signed-off-by: Sally O'Malley <somalley@redhat.com>
When `-s` is not specified, there is no need to ask if there is a plugin
with the specified name.
This speeds up unit tests dramatically since they don't need to wait the
timeout period for each call to `graphdriver.New`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
If I have some unsupported sysinfo, it's warning on daemon
side every time I use `docker info`, it seems unnecessay and
annoying to me, let's keep it quiet.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
Pick up name regexp change in distribution to allow matching of hostnames as a valid component of a repository.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
A recent change made `graphdriver.NaiveDiffDriver` from a function to a
struct. New function is `graphdriver.NewNaiveDiffDriver.
The graphdriver plugin PR was created (and tests run) before this change
was introduced causing the failure to not be seen until after merge.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- use daemon member directly rather than through a function call
- create GetImage method for use external to deamon
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
Fixes a race when starting a container when there is an error, the stdio
streams are not always written.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
There is no need to call `os.Stat` on the driver filesystem path of a
container as `os.RemoveAll` already handles (properly) the case where
the path no longer exists.
Given the results of the stat() were not even being used, there is no
value in erroring out because of the stat call failure, and worse, it
prevents daemon cleanup of containers in "Dead" state unless you re-create
directories that were already removed via a manual cleanup after a
failure. This brings removal in overlay in line with aufs/devicemapper
drivers which don't error out if the filesystem path no longer exists.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
The registry doesn't have containers, only images.
Also updated the Docker Hub link to the new location.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This leverages recent additions to libkv enabling client
authentication via TLS so the discovery back-end can be locked
down with mutual TLS. Example usage:
docker daemon [other args] \
--cluster-advertise 192.168.122.168:2376 \
--cluster-store etcd://192.168.122.168:2379 \
--cluster-store-opt kv.cacertfile=/path/to/ca.pem \
--cluster-store-opt kv.certfile=/path/to/cert.pem \
--cluster-store-opt kv.keyfile=/path/to/key.pem
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
- refactor to make it easier to split the api in the future
- additional tests for non existent container case
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
- create pass through in daemon for access to functions on daemon
member
- import image
push image
export image and corrections
lookup image & comments
load image
list images
image history & comments
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
* fixed weird logic with "*"
* return error if fdNum is failed to parse
* check if listener at offset is nil
* close unused listeners
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Now we're start to serve early, but all Accept calls are intercepted by
listenbuffer or systemd socket.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Right now we check for the existence of device but don't make sure it is
a thin pool device. We assume it is a thin pool device and call poolStatus()
on the device which returns an error EOF. And that error does not tell
anything.
So before we reach the stage of calling poolStatus() make sure we are working
with a thin pool device otherwise error out.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Before, typing `docker volume` with no args would forward to the handler
for `docker volume ls`, except the flags for the `ls` subcommand were
not supported.
Instead just print the cmd usage.
This makes the behavior of the `docker volume` subcommand behave exactly
like the `docker network` subcommand.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
I disagree with #14546 that pushed the help text past 80 chars.
Aside from it now making the help text look ugly on 80 char displays,
which I use, one thing I like about the previous limitation is that it
forced us to keep our options down to more reasonable phrases/words.
For example, I think
` --disable-content-trust=true`
could have been:
` --disable-trust=true`
or even:
` --disable-ctrust=true`
But regardless, let's at least make the comments match what the code does.
Signed-off-by: Doug Davis <dug@us.ibm.com>
* Exiting experimental network UX
* removed experimental service UX
* integrated with the new network remote API
Signed-off-by: Madhu Venugopal <madhu@docker.com>
* Moving Network Remote APIs out of experimental
* --net can now accept user created networks using network drivers/plugins
* Removed the experimental services concept and --default-network option
* Neccessary backend changes to accomodate multiple networks per container
* Integration Tests
Signed-off-by: David Calavera <david.calavera@gmail.com>
Signed-off-by: Madhu Venugopal <madhu@docker.com>
progressreader.Broadcaster becomes broadcaster.Buffered and
broadcastwriter.Writer becomes broadcaster.Unbuffered.
The package broadcastwriter is thus renamed to broadcaster.
Signed-off-by: Tibor Vass <tibor@docker.com>
This patch creates interfaces in builder/ for building Docker images.
It is a first step in a series of patches to remove the daemon
dependency on builder and later allow a client-side Dockerfile builder
as well as potential builder plugins.
It is needed because we cannot remove the /build API endpoint, so we
need to keep the server-side Dockerfile builder, but we also want to
reuse the same Dockerfile parser and evaluator for both server-side and
client-side.
builder/dockerfile/ and api/server/builder.go contain implementations
of those interfaces as a refactoring of the current code.
Signed-off-by: Tibor Vass <tibor@docker.com>
Start a goroutine which runs every 30 seconds and if there are deferred
deleted devices, it tries to clean those up.
Also it moves the call to cleanupDeletedDevices() into goroutine and
moves the locking completely inside the function. Now function does not
assume that device lock is held at the time of entry.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Finally here is the patch to implement deferred deletion functionality.
Deferred deleted devices are marked as "Deleted" in device meta file.
First we try to delete the device and only if deletion fails and user has
enabled deferred deletion, device is marked for deferred deletion.
When docker starts up again, we go through list of deleted devices and
try to delete these again.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Provide a command line option dm.use_deferred_deletion to enable deferred
device deletion feature. By default feature will be turned off.
Not sure if there is much value in deferred deletion being turned on
without deferred removal being turned on. So for now, this feature can
be enabled only if deferred removal is on.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Currently during startup we walk through all the device files and read
their device ID and mark in a bitmap that device id is used.
We are anyway going through all device files. So we can as well load all
that data into device hash map. This will save us little time when
container is actually launched later.
Also this will help with later patches where cleanup deferred device
wants to go through all the devices and see which have been marked for
deletion and delete these.
So re-organize the code a bit.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Simplify setupBaseImage() even further. Move some more code in a separate
function. Pure code reorganization. No functionality change.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Move thin pool related checks in a separate function. Pure code reorganization.
Makes reading code easier.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
This moves base device creation function in a separate function. Pure
code reorganization. Makes reading code little easier.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
W/o this PR the logs would show:
```
ERRO[0000] API listen on [::]:2375
ERRO[0000] API listen on /var/run/docker.sock
```
which is misleading since these aren't "errors" they're just informational.
Signed-off-by: Doug Davis <dug@us.ibm.com>
It prevents occupying of those resources (ports, unix-sockets) by
containers.
Also fixed false-positive test for that case.
Fix#15912
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
This patch does three things. Following are the descriptions.
===
Create a separate function for delete transactions so that parent function
is little smaller.
Also close transaction if an error happens.
===
When docker is being shutdown, save deviceset metadata first before
trying to remove the devices. Generally caller gives only 10 seconds
for shutdown to complete and then kills it after that. So if some device
is busy, we will wait 20 seconds for it removal and never be able to save
metadata. So first save metadata and then deal with device removal.
===
Move issue discard operation in a separate function. This makes reading code
little easier.
Also don't issue discards if device is still open. That means devices is
still probably being used and issuing discards is not a good idea.
This is especially true in case of deferred deletion. We want to issue
discards when device is not open. At that time device can be deleted too.
Otherwise we will issue discards and deletion will actually fail. Later
we will try deletion again and issue discards again and deletion will
fail again as device is open and busy.
So this will ensure that discards are issued once when device is not open
and it can actually be deleted.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
- missing help option in `docs/reference/commandline/*.md` (some files
have it, the other I fixed didn't)
- missing `[OPTIONS]` in Usage description
- missing options
- formatting
- start/stop idempotence
Signed-off-by: Antonio Murdaca <amurdaca@redhat.com>
Search terms shouldn't be restricted to only full valid repository
names. It should be perfectly valid to search using a part of a name,
even if it ends with a period, dash or underscore.
Signed-off-by: Hu Keping <hukeping@huawei.com>
Fixes an issue where a `Dead` container has no names so the API returns
`null` instead of an empty array.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This commit brings in end to end integration of Docker Discovery with
libnetwork multi-host networking.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Exec start was sending HTTP 500 for every error.
Fixed an error where pausing a container and then calling exec start
caused the daemon to freeze.
Updated API docs which incorrectly showed that a successful exec start
was an HTTP 201, in reality it is HTTP 200.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Right now we seem to have 3 locks.
- devinfo.lock
This is a per device lock
- metaData.devicesLock
This is supposedely protecting map of devices.
- Global DeviceSet lock
This is protecting map of devices as well as serializing calls to libdevmapper.
Semantics of per devices lock and global deviceset lock seem to be very clear.
Even ordering between these two locks has been defined properly.
What is not clear is the need and ordering of metaData.devicesLock. Looks like
this lock is not necessary and global DeviceSet lock should be used to
protect map of devices as it is part of DeviceSet.
This patchset gets rid of metaData.devicesLock and instead uses DeviceSet
lock to protect map of devices.
Also at couple of places during initialization takes devices.Lock(). That
is not strictly necessary as there is supposed to be one thread of execution
during initializaiton. Still it makes the code clearer.
I think this makes code more clear and easier to understand and easier to
make further changes.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
maxDeviceID is upper limit on device Id thin pool can support. Right now
we have this check only during startup. It is a good idea to move this
check in loadMetadata so that any time a device file is loaded and if it
is corrupted and device Id is more than maxDevieceID, it will be detected
right then and there.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Use deactivateDevice() instead of removeDevice() directly. This will make
sure for device deletion, deferred removal is used if user has configured
it in. Also this makes reading code litle easier as there is single function
to remove a device and that is deactivateDevice().
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Fixing user namespaces (again) with a vendor update from runc
(specifically, the remount() only if special flags change)
Other changes are very minimal.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
If a device is still mounted at the time of DeleteDevice(), that means
higher layers have not called Put() properly on the device and are trying
to delete it. This is a bug in the code where Get() and Put() have not been
properly paired up. Fail device deletion if it is still mounted.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Exists() and HasDevice() just check if device file exists or not. It does
not say anything about if device is mounted or not. Fix comments.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
device has map (device.Devices), contains valid devices and we skip all
the files which are not device files. transaction metadata file is not
device file. Skip this file when devices files are being read and loaded
into map.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Implement basic interfaces to write custom routers that can be plugged
to the server. Remove server coupling with the daemon.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This passes through the container hostname to HCS, which in Windows Server
2016 TP4 will set the container's hostname in the registry before starting
it. This will be silently ignored by TP3.
Signed-off-by: John Starks <jostarks@microsoft.com>
Although having a request ID available throughout the codebase is very
valuable, the impact of requiring a Context as an argument to every
function in the codepath of an API request, is too significant and was
not properly understood at the time of the review.
Furthermore, mixing API-layer code with non-API-layer code makes the
latter usable only by API-layer code (one that has a notion of Context).
This reverts commit de41640435, reversing
changes made to 7daeecd42d.
Signed-off-by: Tibor Vass <tibor@docker.com>
Conflicts:
api/server/container.go
builder/internals.go
daemon/container_unix.go
daemon/create.go
This reverts commit ff92f45be4, reversing
changes made to 80e31df3b6.
Reverting to make the next revert easier.
Signed-off-by: Tibor Vass <tibor@docker.com>
This fixes the case where directory is removed in
aufs and then the same layer is imported to a
different graphdriver.
Currently when you do `rm -rf /foo && mkdir /foo`
in a layer in aufs the files under `foo` would
only be be hidden on aufs.
The problems with this fix:
1) When a new diff is recreated from non-aufs driver
the `opq` files would not be there. This should not
mean layer differences for the user but still
different content in the tar (one would have one
`opq` file, the others would have `.wh.*` for every
file inside that folder). This difference also only
happens if the tar-split file isn’t stored for the
layer.
2) New files that have the filenames before `.wh..wh..opq`
when they are sorted do not get picked up by non-aufs
graphdrivers. Fixing this would require a bigger
refactoring that is planned in the future.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This test is failing once in a while on the CI, because the docker
attach command might be called after the container ends.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Fixes an issue where `VOLUME some_name:/foo` would be parsed as a named
volume, allowing access from the builder to any volume on the host.
This makes sure that named volumes must always be passed in as a bind.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Before this patch libcontainer badly errored out with `invalid
argument` or `numerical result out of range` while trying to write
to cpuset.cpus or cpuset.mems with an invalid value provided.
This patch adds validation to --cpuset-cpus and --cpuset-mems flag along with
validation based on system's available cpus/mems before starting a container.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Add a daemon flag to control this behaviour. Add a warning message when pulling
an image from a v1 registry. The default order of pull is slightly altered
with this changset.
Previously it was:
https v2, https v1, http v2, http v1
now it is:
https v2, http v2, https v1, http v1
Prevent login to v1 registries by explicitly setting the version before ping to
prevent fallback to v1.
Add unit tests for v2 only mode. Create a mock server that can register
handlers for various endpoints. Assert no v1 endpoints are hit with legacy
registries disabled for the following commands: pull, push, build, run and
login. Assert the opposite when legacy registries are not disabled.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Use `pkg/discovery` to provide nodes discovery between daemon instances.
The functionality is driven by two different command-line flags: the
experimental `--cluster-store` (previously `--kv-store`) and
`--cluster-advertise`. It can be used in two ways by interested
components:
1. Externally by calling the `/info` API and examining the cluster store
field. The `pkg/discovery` package can then be used to hit the same
endpoint and watch for appearing or disappearing nodes. That is the
method that will for example be used by Swarm.
2. Internally by using the `Daemon.discoveryWatcher` instance. That is
the method that will for example be used by libnetwork.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
Absorb Swarm's discovery package in order to provide a common node
discovery mechanism to be used by both Swarm and networking code.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
* Thanks to the Default gateway service in libnetwork, we dont have to add
containers explicitly to secondary public network. This is handled
automatically regardless of the primary network driver.
* Fixed the URL convention for kv-store to be aligned with the upcoming
changes to discovery URL
* Also, in order to bring consistency between external and internal network
drivers, we moved the driver configs via controller Init.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
- Libnetwork brings in :
* Default Gateway as a service for network drivers
* Persistence for local scoped networks using libkv
* BATS based Multi-host Integration-test infra and end-to-end tests
* libnetwork fixes for zookeeper and etcd backend
- Libkv upgrade brings in :
* boltdb support for local kv persistence
* other general bug fixes
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Avoid creating a global context object that will be used while the daemon is running.
Not only this object won't ever be garbage collected, but it won't ever be used for anything else than creating other contexts in each request. I think it's a bad practive to have something like this sprawling aroud the code.
This change removes that global object and initializes a context in the cases we don't have already one, like shutting down the server.
This also removes a bunch of context arguments from functions that did nothing with it.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Add 10 seconds timeout when running `sudo service docker stop`. This is
especially needed when running `sudo service docker restart`. Otherwise,
`restart` results in exitstatus 1, because `start` has nothing to do.
Signed-off-by: Ewa Czechowska <ewa@ai-traders.com>
The mount syscall does not handle string flags like "noatime",
we must use bitmasks like MS_NOATIME instead.
pkg/mount.Mount already handles this work.
Signed-off-by: Carl Henrik Lunde <chlunde@ping.uio.no>
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:
```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.
Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.
I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.
ping @icecrime @calavera @crosbymichael
Signed-off-by: Doug Davis <dug@us.ibm.com>
- Print the mount table as in /proc/self/mountinfo
- Do not exit prematurely when one of the ipc mounts doesn't exist.
- Do not exit prematurely when one of the ipc mounts cannot be unmounted.
- Add a unit test to see if the cleanup really works.
- Use syscall.MNT_DETACH to cleanup mounts after a crash.
- Unmount IPC mounts when the daemon unregisters an old running container.
Signed-off-by: David Calavera <david.calavera@gmail.com>
On boot, the init script is invoked from `/etc/rcN.d/S20docker` (where N
is the runlevel). Consequently, the init script tried to find the
defaults at `/etc/defaults/S20docker` and the binary at
`/usr/bin/S20docker`. This causes the docker daemon to fail at boot with
the error
/usr/bin/S20docker not present or not executable
Starting it manually works because it invokes `/etc/init.d/docker` which
has the correct basename.
Fix this by hardcoding "docker" as the base name.
Signed-off-by: Matthias Rampke <mr@soundcloud.com>
The shell builtin `compopt` is not available on the outdated bash
version 3.2.57 that ships with Mac OS.
It is used in Docker's bash completion to suppress trailing spaces
in advanced completions of hash map options, e.g. `--log-opt`.
If `compopt` is not available, the new behavior is to do nothing,
i.e. the user will have to delete the additional space.
Signed-off-by: Harald Albers <github@albersweb.de>
Once there have '\t' in the "CREATED BY" section, it will possibly
cause a messy output of docker history.
Signed-off-by: Hu Keping <hukeping@huawei.com>
The tests added cover the case when the Writer field returns and error
and, related to that, when the number of written bytes is less than 0.
Signed-off-by: Federico Gimenez <fgimenez@coit.es>
* Update format for 'Note:' to match other pages.
* Add link to Go's RFC3339Nano timestamp information.
Signed-off-by: Charles Chan <charleswhchan@users.noreply.github.com>
It defines global middlewares for every request.
This makes the server slightly more composable.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Pull in version e5fea92a6c8a5968bdb8005bf959c6e23113b689
Fixes libnetwork regressions that stopped drivers that set routes from working.
Pulls in libnetwork PRs #546#543
Signed-off-by: Tom Denham <tom@tomdee.co.uk>
- Add unit tests to make sure the functionality is correct.
- Add FilterByDriver to allow filtering volumes by driver, for future
`volume ls` filtering and whatnot.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This comment was wrongly referring to the old job mechanism and it
wasn't clear what it was trying to document.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
If an invalid logger address is provided on daemon start it will
silently fail. As syslog driver is doing, this check should be done on
daemon start and prevent it from starting even in other drivers.
This patch also adds integration tests for this behavior.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
If you don't have cgroup swap memory support, `dockerCmd`'s output in
these tests will be polluted by a warning from the daemon and will fail
the tests.
No need to have memory swap support for these tests to run as it will
be reset to -1 and everything will continue correctly.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
`docker rename foo ''` would result in:
```
usage: docker rename OLD_NAME NEW_NAME
```
which is the old engine's way of return errors - yes that's in the
daemon code. So I fixed that error msg to just be normal.
While doing that I noticed that using an empty string for the
source container name failed but didn't print any error message at all.
This is because we would generate a URL like: ../containers//rename/..
which would cause a 301 redirect to ../containers/rename/..
however the CLI code doesn't actually deal with 301's - it just ignores
them and returns back to the CLI code/caller.
Rather than changing the CLI to deal with 3xx error codes, which would
probably be a good thing to do in a follow-on PR, for this immediate
issue I just added a cli-side check for empty strings for both old and
new names. This way we catch it even before we hit the daemon.
API callers will get a 404, assuming they follow the 301, for the
case of the src being empty, and the new error msg when the destination
is empty - so we should be good now.
Add tests for both cases too.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Fixes#11957Fixes#12319
Also removes check for Darwin when the stdin reader is closed as it
doesn't appear to block any more.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This way provide both Time and TimeNano in the event. For the display of
the JSONMessage, use either, but prefer TimeNano Proving only TimeNano
would break Subscribers that are using the `Time` field, so both are set
for backwards compatibility.
The events logging uses nano formatting, but only provides a Unix()
time, therefor ordering may get lost in the output. Example:
```
2015-09-15T14:18:51.000000000-04:00 ee46febd64ac629f7de9cd8bf58582e6f263d97ff46896adc5b508db804682da: (from busybox) resize
2015-09-15T14:18:51.000000000-04:00 a78c9149b1c0474502a117efaa814541926c2ae6ec3c76607e1c931b84c3a44b: (from busybox) resize
```
By having a field just for Nano time, when set, the marshalling back to
`time.Unix(sec int64, nsec int64)` has zeros exactly where it needs to.
This does not break any existing use of jsonmessage.JSONMessage, but now
allows for use of `UnixNano()` and get event formatting that has
distinguishable order. Example:
```
2015-09-15T15:37:23.810295632-04:00 6adcf8ed9f5f5ec059a915466cd1cde86a18b4a085fc3af405e9cc9fecbbbbaf: (from busybox) resize
2015-09-15T15:37:23.810412202-04:00 6b7c5bfdc3f902096f5a91e628f21bd4b56e32590c5b4b97044aafc005ddcb0d: (from busybox) resize
```
Including tests for TimeNano and updated event API reference doc.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
GET /containers/json route used to reply with and empty array `[]` when no
containers where available. Daemon containers list refactor introduced
this bug by declaring an empty slice istead of initializing it as well
and it was now replying with `null`.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
When use `docker stats` to minitor a running container and then stop it,
there are some fields need to be reset to zero. Otherwise it will keep
displaying the data it received last time.
Signed-off-by: Hu Keping <hukeping@huawei.com>
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
cleaning up docker run -v documentation for man and web reference guide
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
- refactor to make it easier to split the api in the future
- addition to check the existing test case and make sure it contains
some expected output
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
To fix an issue with experimental multihost networking.
git hash: 00a92f066e628e4c6d50979c070df377575aad18
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Using @mavenugo's patch for enabling the libcontainer pre-start hook to
be used for network namespace initialization (correcting the conflict
with user namespaces); updated the boolean check to the more generic
SupportsHooks() name, and fixed the hook state function signature.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
- The build-time variables are passed as environment-context for command(s)
run as part of the RUN primitve. These variables are not persisted in environment of
intermediate and final images when passed as context for RUN. The build environment
is prepended to the intermediate continer's command string for aiding cache lookups.
It also helps with build traceability. But this also makes the feature less secure from
point of view of passing build time secrets.
- The build-time variables also get used to expand the symbols used in certain
Dockerfile primitves like ADD, COPY, USER etc, without an explicit prior definiton using a
ENV primitive. These variables get persisted in the intermediate and final images
whenever they are expanded.
- The build-time variables are only expanded or passed to the RUN primtive if they
are defined in Dockerfile using the ARG primitive or belong to list of built-in variables.
HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy, FTP_PROXY and NO_PROXY are built-in
variables that needn't be explicitly defined in Dockerfile to use this feature.
Signed-off-by: Madhav Puri <madhav.puri@gmail.com>
With go1.5's concurrency, the use of a goroutine in Log'ing events was
causing the resulting events to not be in order.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
For now docker stats will sum the rxbytes, txbytes, etc. of all
the interfaces.
It is OK for the output of CLI `docker stats` but not good for
the API response, especially when the container is in sereval
subnets.
It's better to leave these origianl data to user.
Signed-off-by: Hu Keping <hukeping@huawei.com>
Volumes are accounted when a container is created.
If the creation fails we should remove the reference from the counter.
Do not log ErrVolumeInUse as an error, having other volume references is
not an error.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This is the first step in converting out static strings into well-defined
error types. This shows just a few examples of it to get a feel for how things
will look. Once we agree on the basic outline we can then work on converting
the rest of the code over.
Signed-off-by: Doug Davis <dug@us.ibm.com>
@noxiouz points out that we don't need to check for a nil result from
C.CString(), since an out-of-memory condition causes a runtime panic
instead.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
Notary is capable of acting in offline mode, making use of cache TUF data.
When ping is not successful, notary should still be attempted without error.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Changes include :
* libnetwork support for userns
* driver api change to have 1 interface per endpoint
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Replace comma with column
Remove extra curly brace
Fix typo in REST API documentation
Replace comma with column in documentation.
Signed-off-by: Adrian Oprea <adrian@codesi.nz>
Remove extra curly brace
Remove extra curly brace from remote API docs.
Signed-off-by: Adrian Oprea <adrian@codesi.nz>
This is not a public interface and users are trying to use it.
We only use it to be able to generate the rpc protocol to talk with a plugin.
Signed-off-by: David Calavera <david.calavera@gmail.com>
If a logdriver doesn't register a callback function to validate log
options, it won't be usable. Fix the journald driver by adding a dummy
validator.
Teach the client and the daemon's "logs" logic that the server can also
supply "logs" data via the "journald" driver. Update documentation and
tests that depend on error messages.
Add support for reading log data from the systemd journal to the
journald log driver. The internal logic uses a goroutine to scan the
journal for matching entries after any specified cutoff time, formats
the messages from those entries as JSONLog messages, and stuffs the
results down a pipe whose reading end we hand back to the caller.
If we are missing any of the 'linux', 'cgo', or 'journald' build tags,
however, we don't implement a reader, so the 'logs' endpoint will still
return an error.
Make the necessary changes to the build setup to ensure that support for
reading container logs from the systemd journal is built.
Rename the Jmap member of the journald logdriver's struct to "vars" to
make it non-public, and to make it easier to tell that it's just there
to hold additional variable values that we want journald to record along
with log data that we're sending to it.
In the client, don't assume that we know which logdrivers the server
implements, and remove the check that looks at the server. It's
redundant because the server already knows, and the check also makes
using older clients with newer servers (which may have new logdrivers in
them) unnecessarily hard.
When we try to "logs" and have to report that the container's logdriver
doesn't support reading, send the error message through the
might-be-a-multiplexer so that clients which are expecting multiplexed
data will be able to properly display the error, instead of tripping
over the data and printing a less helpful "Unrecognized input header"
error.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
Noteworthy changes:
- Add Prestart/Poststop hook support
- Fix bug finding cgroup mount directory
- Add OomScoreAdj as a container configuration option
- Ensure the cleanup jobs in the deferrer are executed on error
- Don't make modifications to /dev when it is bind mounted
Other changes in runc:
https://github.com/opencontainers/runc/compare/v0.0.3...v0.0.4
Signed-off-by: David Calavera <david.calavera@gmail.com>
This PR makes a user visible behavior change with userland
proxy disabled by default and rely on hairpin NAT to be enabled
by default. This may not work in older (unsupported) kernels
where the user will be forced to enable userlandproxy if needed.
- Updated the Docs
- Changed the integration-cli to start with userlandproxy
desiabled by default.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
This changeset creates /dev/shm and /dev/mqueue mounts for each container under
/var/lib/containers/<id>/ and bind mounts them into the container. When --ipc:container<id/name>
is used, then the /dev/shm and /dev/mqueue of the ipc container are used instead of creating
new ones for the container.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
(cherry picked from commit d88fe447df)
Also, use the channel to determine if the broadcaster is closed,
removing the redundant isClosed variable.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The original purpose of this was to cancel downloads if pullV2Tag
returns an error, preventing an associated crash (see #15353). The
broadcaster now accomplishes the same thing that the pipe does, making
the pipe redundant. When pullV2Tag returns, all broadcasters are closed,
which means all further writes to those broadcasters will return errors.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Before, this only waited for the download to complete. There was no
guarantee that the layer had been registered in the graph and was ready
use. This is especially problematic with v2 pulls, which wait for all
downloads before extracting layers.
Change Broadcaster to allow an error value to be propagated from Close
to the waiters.
Make the wait stop when the extraction is finished, rather than just the
download.
This also fixes v2 layer downloads to prefix the pool key with "layer:"
instead of "img:". "img:" is the wrong prefix, because this is what v1
uses for entire images. A v1 pull waiting for one of these operations to
finish would only wait for that particular layer, not all its
dependencies.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Ensure that layers are not excluded from manifests based on previous pushes.
Continue skipping pushes on layers which were pushed by a previous tag.
Update push multiple tag tests.
Ensure that each tag pushed exists on the registry and is pullable.
Add output comparison on multiple tag push check.
fixes#15536
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Allow to set the signal to stop a container in `docker run`:
- Use `--stop-signal` with docker-run to set the default signal the container will use to exit.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Without this change, there was a narrow race condition that would allow
writers to finish when there was still data left to write. This is
likely to be what was causing some integration tests to fail with
truncated pull output.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Allows people to create out-of-process graphdrivers that can be used
with Docker.
Extensions must be started before Docker otherwise Docker will fail to
start.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This defines a 'context' object that is passed to each API handler.
Right now the context just has a unique 'requestID' for each API call.
The next steps would be:
- use this 'requestID' in our logging.
- determine the best way to format the logging to include this info.
In particular for log events that generate multiple entries in the log
we can use the requestID to help correlate the log entries.
Adding the requestID to the logging will be a challenge since it could mean
changing every single logrus.XXX() call to pass in the 'context' object.
But first step is to agree on a format, which we can discus in a subsequent
PR, but my initial thoughts are to add it right after the timestamp:
current format:
INFO[0039] POST /v1.21/build?buildargs=%7B%22foo%22%3A%22xxx%22%7D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=&ulimits=null
proposed format:
INFO[0039-83dea1222191] POST /v1.21/build?buildargs=%7B%22foo%22%3A%22xxx%22%7D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=&ulimits=null
Signed-off-by: Doug Davis <dug@us.ibm.com>
- utils_test.go and docker_utils_test.go
- Moved docker related function to docker_utils.go
- add a test for integration-cli/checker
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
String.Trim(string, " ") can only get ride of the blank, but actully we need
to remove all leading and trailing white space include "\t" for example.
Prior to this patch, one can login with "[Tab][Tab]username", and it will
not be consider as the same with "username".
Signed-off-by: Hu Keping <hukeping@huawei.com>
Underlying volume data may have been removed by some other tool.
Ignore and remove the reference in this case.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Closes#16146
While in there, modified the testing infrastructure for the help text
so that we can get commands with nested commands - like "volume".
Signed-off-by: Doug Davis <dug@us.ibm.com>
For both commands, volume is _not_ optional. Several volumes may
be specified.
Both commands now use the same name (VOLUME) for the command argument.
Signed-off-by: Harald Albers <github@albersweb.de>
+ Fix a couple of bugs introduced by previous vendoring:
- in bitseq which prevents to use experimental overlay networking
- in docker service ls cli o/p
+ Add missing http subrouter for newly introduced sandboxes
+ Fix fragmentation issue on vxlan header addition for overlay network driver
+ Remove libnetwork test code utilities from vendoring
Signed-off-by: Alessandro Boch <aboch@docker.com>
- Remove extraneous '-' in sudo note.
- Correct space formatting (minor).
- Simplify instructions for install EPEL and install with yum directly.
- Add `nodejs` to yum install list explicitly to match the comments -- more transparent for new users who might not be clear `npm` package also installs `nodejs`.
- Remove '--noprealloc` from MongoDB example as the option has been depcated since Mongo 2.6 and is now the default behaviour. See: http://docs.mongodb.org/manual/reference/program/mongod/
Signed-off-by: Charles Chan <charleswhchan@users.noreply.github.com>
The docker volume ls -f dangling=true filter was
inverted; the filtered results actually returned all
non-dangling volumes.
This fixes the filter and adds some integration tests
to test the correct behavior.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For now CLI `docker stats` will not block even if the container was
not running is because there is a 2s timeout setting when waiting for
the response.
I think why we hang there waiting for the container to run is because we
want to get the stats of container immediately when it starts running.
But it will block when use the API directly, for example
- curl
- Google Chrome plugin, Postman
- Firefox plugin, RESTClient
This patch keeps the feature that getting info immediately when container
starts running and in the meantime, it will not block when using the API
directrly.
Signed-off-by: Hu Keping <hukeping@huawei.com>
TL;DR: stop building static binary that may fail
Linker flag --unresolved-symbols=ignore-in-shared-libs was added
in commit 06d0843 two years ago for the static build case, presumably
to avoid dealing with problem of missing libraries.
For the record, this is what ld(1) man page says:
> --unresolved-symbols=method
> Determine how to handle unresolved symbols. There are four
> possible values for method:
> .........
> ignore-in-shared-libs
> Report unresolved symbols that come from regular object files,
> but ignore them if they come from shared libraries. This can
> be useful when creating a dynamic binary and it is known that
> all the shared libraries that it should be referencing are
> included on the linker's command line.
Here, the flag is not used for its purpose ("creating a dynamic binary")
and does more harm than good. Instead of complaining about missing symbols
as it should do if some libraries are missing from LIBS/LDFLAGS, it lets
ld create a binary with unresolved symbols, ike this:
$ readelf -s bundles/1.7.1/binary/docker-1.7.1 | grep -w UND
........
21029: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND dlopen
.........
Such binary is working just fine -- until code calls one of those
functions, then it crashes (for apparently no reason, i.e. it is
impossible to tell why from the diagnistics printed).
In other words, adding this flag allows to build a static binary
with missing libraries, hiding the problem from both a developer
(who forgot to add a library to #cgo: LDFLAGS -- I was one such
developer a few days ago when I was working on ploop graphdriver)
and from a user (who expects the binary to work without crashing,
and it does that until the code calls a function in one of those
libraries).
Removing the flag immediately unveils the problem (as it should):
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o):
In function `unixDlError':
(.text+0x20971): undefined reference to `dlerror'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o):
In function `unixDlClose':
(.text+0x8814): undefined reference to `dlclose'
The problem is, gosqlite package says:
#cgo LDFLAGS: -lsqlite3
which is enough for dynamic linking, as indirect dependencies (i.e.
libraries required by libsqlite3.so) are listed in .so file and will be
resolved dynamically by ldd upon executing the binary.
For static linking though, one has to list all the required libraries,
both direct and indirect. For libraries with pkgconfig support the
list of required libraries can be obtained with pkg-config:
$ pkg-config --libs sqlite3 # dynamic linking case
-lsqlite3
$ pkg-config --libs --static sqlite3 # static case
-lsqlite3 -ldl -lpthread
It seems that all one has to do is to fix gosqlite this way:
-#cgo LDFLAGS: -lsqlite3
+#cgo pkg-config: sqlite3
Unfortunately, cmd/go doesn't know that it needs to pass --static
flag to pkg-config in case of static linking
(see https://github.com/golang/go/issues/12058).
So, for one, one has to do one of these things:
1. Patch sqlite.go like this:
-#cgo LDFLAGS: -lsqlite3
+#cgo pkg-config: --static sqlite3
(this is exactly what I do in goploop, see
https://github.com/kolyshkin/goploop/commit/e9aa072f51)
2. Patch sqlite.go like this:
-#cgo LDFLAGS: -lsqlite3
+#cgo LDFLAGS: -lsqlite3 -ldl -lpthread
(I would submit this patch to gosqlite but it seems that
https://code.google.com/p/gosqlite/ is deserted and not maintained,
and patching it here is not right as it is "vendored")
3. Explicitly add -ldl for the static link case.
This is what this patch does.
4. Fork sqlite to github and maintain it there. Personally I am not
ready for that, as I'm neither a Go expert nor gosqlite user.
Now, #3 doesn't look like a clear solution, but nevertheless it makes
the build much better than it was before.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Calling runtime.Stack requires the buffer to be big enough to fit the
goroutines dump. If it's not big enough the dump will be truncated and
the value returned will be the same size as the buffer.
The code was changed to handle this situation and try again with a
bigger buffer. Each time the dump doesn't fit in the buffer its size is
doubled.
Signed-off-by: Cezar Sa Espinola <cezarsa@gmail.com>
Fixed issues related to network subcommand tests
- "network" in exempted list of short help check
- Condition for exact test modified to meet experimental commands
- Sorting of commands done in flags_experimental
Signed-off-by: Kunal Kushwaha <kunal.kushwaha@gmail.com>
The Docker Daemon should send actual actions client ask for to issue tokens,
not all the permissions that client is guaranteed.
Signed-off-by: xiekeyang <xiekeyang@huawei.com>
This adds a data struct in the aufs driver for including more
information about active mounts along with their reference count.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
I just tried the installation but docker didn't start automatically. So I had to execute the newly added command in order to get the hello-world verification running.
Signed-off-by: Jenny Gebske <jennifer@gebske.de>
2015-08-06 13:55:32 +02:00
1242 changed files with 109530 additions and 26498 deletions
cmd:=Cli.Subcmd("import",[]string{"file|URL|- [REPOSITORY[:TAG]]"},"Create an empty filesystem image and import the contents of the\ntarball (.tar, .tar.gz, .tgz, .bzip, .tar.xz, .txz) into it, then\noptionally tag it.",true)
cmd:=Cli.Subcmd("login",[]string{"[SERVER]"},"Register or login to a Docker registry server, if no server is\nspecified \""+registry.IndexServer+"\" is the default.",true)
cmd:=Cli.Subcmd("login",[]string{"[SERVER]"},Cli.DockerCommands["login"].Description+".\nIf no server isspecified \""+registry.IndexServer+"\" is the default.",true)
cmd:=Cli.Subcmd("logout",[]string{"[SERVER]"},"Logout from a Docker registry, if no server is\nspecified \""+registry.IndexServer+"\" is the default.",true)
cmd:=Cli.Subcmd("logout",[]string{"[SERVER]"},Cli.DockerCommands["logout"].Description+".\nIf no server isspecified \""+registry.IndexServer+"\" is the default.",true)
// Usage: docker port CONTAINER [PRIVATE_PORT[/PROTO]]
func(cli*DockerCli)CmdPort(args...string)error{
cmd:=Cli.Subcmd("port",[]string{"CONTAINER [PRIVATE_PORT[/PROTO]]"},"List port mappings for the CONTAINER, or lookup the public-facing port that\nis NAT-ed to the PRIVATE_PORT",true)
// A running container is stopped by first sending SIGTERM and then SIGKILL if the container fails to stop within a grace period (the default is 10 seconds).
cmd:=Cli.Subcmd("stop",[]string{"CONTAINER [CONTAINER...]"},"Stop a running container by sending SIGTERM and then SIGKILL after a\ngrace period",true)
cmd:=Cli.Subcmd("stop",[]string{"CONTAINER [CONTAINER...]"},Cli.DockerCommands["stop"].Description+".\nSending SIGTERM and then SIGKILL after agrace period",true)
nSeconds:=cmd.Int([]string{"t","-time"},10,"Seconds to wait for stop before killing it")
fmt.Fprintf(cli.err,"[DEPRECATED] The environment variable DOCKER_CONTENT_TRUST_OFFLINE_PASSPHRASE has been deprecated and will be removed in v1.10. Please use DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE\n")
fmt.Fprintf(cli.err,"[DEPRECATED] The environment variable DOCKER_CONTENT_TRUST_TAGGING_PASSPHRASE has been deprecated and will be removed in v1.10. Please use DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE\n")
logrus.Debugf("Warning: client and server don't have the same version (client: %s, server: %s)",userAgent[1],dockerVersion)
}
}
version:=version.Version(mux.Vars(r)["version"])
ifversion==""{
version=api.Version
}
ifcorsHeaders!=""{
writeCorsHeaders(w,r,corsHeaders)
}
ifversion.GreaterThan(api.Version){
http.Error(w,fmt.Errorf("client is newer than server (client API version: %s, server API version: %s)",version,api.Version).Error(),http.StatusBadRequest)
return
}
ifversion.LessThan(api.MinVersion){
http.Error(w,fmt.Errorf("client is too old, minimum supported API version is %s, please upgrade your client to a newer version",api.MinVersion).Error(),http.StatusBadRequest)
This package includes types for legacy API versions. The stable version of the API types live in `api/types/*.go`.
Consider moving a type here when you need to keep backwards compatibility in the API. This legacy types are organized by the latest API version they appear in. For instance, types in the `v1p19` package are valid for API versions below or equal `1.19`. Types in the `v1p20` package are valid for the API version `1.20`, since the versions below that will use the legacy types in `v1p19`.
### Package name conventions
The package name convention is to use `v` as a prefix for the version number and `p`(patch) as a separator. We use this nomenclature due to a few restrictions in the Go package name convention:
1. We cannot use `.` because it's interpreted by the language, think of `v1.20.CallFunction`.
2. We cannot use `_` because golint complains abount it. The code is actually valid, but it looks probably more weird: `v1_20.CallFunction`.
For instance, if you want to modify a type that was available in the version `1.21` of the API but it will have different fields in the version `1.22`, you want to create a new package under `api/types/versions/v1p21`.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
James Hunt