It has been asked on #docker-dev recently if it was possibleto attach a volume to a container after it was started.At first, I thought it would be difficult, because of howthe mnt namespace works. Then I thought better :-)
- Using nsenter, we will be able to run an arbitrary command within the context (technically: the namespaces) of our container, but without the associated security restrictions. Needless to say, this can be done only with root access on the Docker host. The simplest way to install nsenter and its associated docker-enter script is to run.
- By jboss. Updated 13 days ago. KIE Execution Server. Pulls 1M+ Overview Tags.
- # nsenter -t $(docker-pid web) -n ip addr add 10.12.0.117/21 dev web-int And set a new default route inside the container: # nsenter -t $(docker-pid web) -n ip route del default # nsenter -t $(docker-pid web) -n ip route add default via 10.12.7.254 dev web-int Again, we can verify from another host that the web server is available at 10.12.0.117.
Jan 13, 2015 Using nsenter, we will be able to run an arbitrary command within the context (technically: the namespaces) of our container, but without the associated security restrictions. Needless to say, this can be done only with root access on the Docker host. The simplest way to install nsenter and its associated docker-enter script is to run.
TL,DR
To attach a volume into a running container, we are going to:
- use nsenter to mount the whole filesystem containing thisvolume on a temporary mountpoint;
- create a bind mount from the specific directory that wewant to use as the volume, to the right location of this volume;
- umount the temporary mountpoint.
It’s that simple, really.
Preliminary warning
In the examples below, I deliberately included the $ signto indicate the shell prompt and help to make the differencebetween what you’re supposed to type, and what the machineis supposed to answer. There are some multi-line commands,with > continuation characters. I am aware that it makesthe examples really painful to copy-paste. If you want tocopy-paste code, look at the sample script at the end of thispost!
Step by step
In the following example, I assume that I started a simplecontainer named charlie, with the following command:
I also assume that I want to mount the host directory/home/jpetazzo/Work/DOCKER/docker to /src in my container.
Let’s do this!
nsenter
First, you will need nsenter, with the docker-enterhelper script. Why? Because we are going to mount filesystemsfrom within our container, and for security reasons, ourcontainer is not allowed to do that. Using nsenter, wewill be able to run an arbitrary command within the context(technically: the namespaces) of our container, but withoutthe associated security restrictions. Needless to say, thiscan be done only with root access on the Docker host.
The simplest way to install nsenter and its associateddocker-enter script is to run:
For more details, check the nsenter project page.
Find our filesystem
We want to mount the filesystem containing our host directory(/home/jpetazzo/Work/DOCKER/docker) in the container.
We have to find on which filesystem this directory is located.
First, we will canonicalize (or dereference) the file, justin case it is a symbolic link - or its path contains anysymbolic link:
A-ha, it is indeed a symlink! Let’s put that in an environmentvariable to make our life easier:
Then, we need to find which filesystem contains that path.We will use an unexpected tool for that, df:
Let’s use the -P flag (to force POSIX format, just incase you have an exotic df, or someone runs that on Solarisor BSD when those systems will get Docker too) and put theresult into a variable as well:
Find the device (and sub-root) of our filesystem
Now, in a world without bind mounts or BTRFS subvolumes,we would just have to look into /proc/mounts to find outthe device corresponding to the /home/jpetazzo filesystem,and we would be golden. But on my system, /home/jpetazzois a subvolume on a BTRFS pool. To get subvolume information(or bind mount information), we will check /proc/self/mountinfo.
If you had never heard about mountinfo, check proc.txtin the kernel docs, and be enlightened :-)
So, first, let’s retrieve the device of our filesystem:
Next, retrieve the sub-root (i.e. the path of the mountedfilesystem, within the global filesystem living in thisdevice):
Perfect. Now we know that we will need to mount /dev/sda2,and inside that filesystem, go to /jpetazzo, and from there,to the remaining path to our file (in our example,/go/src/github.com/docker/docker).
Let’s compute this remaining path, by the way:
Note: this works as long as there are no , in the path.If you have an idea to make that work regardless of thefunky characters that might be in the path, let me know!(I shall invoke the Shell Triad to the rescue: jessie,soulshake, tianon?)
Nsenter Docker Pid
The last thing that we need to do before diving into thecontainer, is to resolve the major and minor device numbersfor this block device. stat will do it for us:
Note that those numbers are in hexadecimal, and later, we willneed them in decimal. Here is a hackish way to convert themeasily:
Docker Nsenter 1
Putting it all together
There is one last subtle hack. For reasons that are beyond myunderstanding, some filesystems (including BTRFS) will updatethe device field in /proc/mounts when you mount them multipletimes. In other words, if we create a temporary block devicenamed /tmpblkdev in our container, and use that to mount ourfilesystem, then now our filesystem (in the host!) will appear as/tmpblkdev instead of e.g. /dev/sda2. This sounds like alittle detail, but in fact, it will screw up all future attemptsto resolve the filesystem block device.
Long story short: we have to make sure that the block device nodein the container is located at the same path than its counterparton the host.
Let’s do this:
Create a temporary mount point, and mount the filesystem:
Make sure that the volume mount point exists, and bind mountthe volume on it:
Cleanup after ourselves:
Docker Nsenter Example
(We don’t clean up the device node. We could be extra fancyand detect whether the device existed in the first place, butthis is already pretty complex as it is right now!)
Voilà!
Automating the hell out of it
This little snippet is almost copy-paste ready.
Status and limitations
This will not work on filesystems which are not based on block devices.
It will only work if /proc/mounts correctly lists the block devicenode (which, as we saw above, is not necessarily true).
Also, I only tested this on my local environment; I didn’t even tryon a cloud instance or anything like that, but I would love to knowif it works there or not!