2015-04-10

file sharing in docker

After the honeymoon phase with docker is over, you find some pitfalls with docker on osx. Most pitfalls only matter for local development, so everything is fine on the linux server (Why don't I have linux as my desktop os?).

The virtual box file sharing is really slow! Your django app doesn't starts anymore within the blink of an eye. It's takes like 5-10sec now.

I tried out multiple things for the file sharing problem.

samba

My first attempt was to use samba. The idea was to put the source code as close as possible to the container. I put my source code into the boot2docker vm and exported it with samba. But samba is not very unix aware, so symlinks don't work on server AND client, a deal breaker!

appletalk

Next on the list was the old appletalk. I wasn't very comfortable with the idea, but if it works I take it. I had some strange git problems with my mounted share, it doesn't worked good. I felt uncomfortable that my source code was hidden in an vm. Appletalk is also discontinued by apple itself.

nfs to the rescue

With nfs I finally found a almost perfect solution. I could leave my source code on my main host. Nfs has unix roots so symlinks are no problems. Nfs is built into os x so you only need to add one line to your /etc/exports file to enable it. In boot2docker everything is to there to enable it. When you mount your Users folder into boot2docker like the virtual box file sharing does it, it becomes a drop in replacement for the virtual box file sharing.

setup nfs

First export your User folder in your mac /etc/exports file

$ sudo vi /etc/exports

add this content:

#/etc/exports
/Users -alldirs -mapall=501:20 192.168.59.103

Set the uid (501), group (20) for your main user.

Restart nfsd

$ sudo nfsd restart

than add the file /var/lib/boot2docker/bootlocal.sh to your boot2docker vm:

$ boot2docker ssh # now in boot2docker
$ sudo vi /var/lib/boot2docker/bootlocal.sh

add this content

#!/bin/sh
#/var/lib/boot2docker/bootlocal.sh
sudo umount /Users
sudo /usr/local/etc/init.d/nfs-client start
sudo mount -t nfs -o noatime,bg,rw,vers=3,tcp,actimeo=1800,rsize=262144,wsize=262144 192.168.59.3:'/Users' /Users
192.168.50.1:/Users /Users nfs auto,rw,noatime,bg,vers=3,tcp,timeo=600,async,rsize=262144,wsize=262144 0 0
auto,rw,noatime,bg,vers=3,tcp,timeo=600,async,rsize=262144,wsize=262144

192.168.50.1:/Users /Users nfs auto,rw,noatime,bg,vers=3,tcp,timeo=600,async,rsize=262144,wsize=262144 0 0

Done! Easy! After your next boot2docker reboot, your Users folder is mounted with nfs.

I you use something different than boot2docker (virtual box) for running docker under osx, this instructions should be easy to adopt.

some benchmarks

Here are some simple basic benchmarks for testing the file performance. Instead of a benchmark tool I used some real test cases.

My test machine is a Macbook Pro 13" 2.6 GHz, 512 GB.

I mounted my mac file system with the noatime option, this gets me around 10% more disk performance.

I'm planning to switch to vmware fusion for running docker under osx, so I also tested vmware performence. The virtual box tests are run with boot2docker, the vmware fusion tests with ubuntu 14.04.

You can use this docker image for the tests or any other python image.

docker run -it --rm -v /Users/<myuser>/Downloads/:/downloads python:2 bash -l

I used the Django-1.8.tar.gz file for my tests.

I tested it on:

  • mac hfs - local filesystem with hfs (no docker)
  • vb sharing - virtual box file sharing, with virtual box extensions
  • vb nfs - virtual box sharing with nfs
  • fusion sharing - vmware fusion file sharing, with vmware extensions
  • fusion nfs - vmware fusion nfs sharing
  • fusion ext4 - vmware fusion local ext4

simple tar, cp

Some read and write tests.

The commands I run was:

cd /downloads
time tar xf Django-1.8.tar.gz --no-same-owner
time cp -Ra Django-1.8 Django-1.8-2

cat and compile

This should test the read performance and the python compile step simulates some python file operations.

The command I run was:

cd /downloads
time find Django-1.8 -type f | xargs -I{} cat {} > /dev/null
time python -m compileall Django-1.8 > /dev/null

conclusion

Virtual box sharing is really slow! I know that already before! With nfs virtualbox file performance is ok. Vmware fusion really makes everything faster and with nfs it even gets better.