2015-08-18

zfs as local vm storage for vmware fusion on osx

I first used boot2docker as docker platform for Mac OS X. With Boot2docker it's hard to customize stuff, because it's not build for that purpose.

Once I tried to put a desktop app into a docker container with 3D hardware support. The 3D performence was really bad...

VirtualBox isn't the best virtualization platform for this. When using boot2docker with Virtualbox I always thought this should/could perform a bit faster. See also my entry about file sharing in docker. VMware performs better on osx than VirtualBox, especially for 3D stuff.

So I was latent looking for other solutions. Another area I wasn't happy was my vm storage.

To safe space on my laptop SSD, I used btrfs with compression for my /var/lib/docker partition. First this was great but I had a few times some weird problems with btrfs. It was also a testbed to see how I like btrfs.

At the same time I tried zfs on my linux servers and it really blow me away. Cool features and super nice command line interface!

Just to say it's really great that there is boot2docker an VirtualBox. Free and open source and it offers an easy start into docker for osx.

But for more flexibility/performence I choice VMware and zfs.

zfs storage

Before openzfsonosx it was harder to get zfs running on osx. Just follow the install instructions on openzfsonosx.

As web developer you often have a laptop and not a desktop. Sadly there aren't any modern laptops with ECC ram, especially not a macbook. This could soon change with the new skylake xeons, fingers crossed!

WARNING! It's not recommended to use zfs without ECC for important data. Use it at your own risk. I'm not completely sure if zfs without ECC ram is more risky for your data, than any other file system without ECC.

See this links about this topic and make your mind.

I use my VMs and docker installation only for testing/development, and don't have any important data saved there, so this doesn't matter to much for me.

partitioning

I installed zfs on my macbook pro, normally all of your hard disk space is already partitioned.

Luckily the disk utility tool of osx can split a partition into two parts without destroying any data. Do this and keep the new created partition.

Check you target partition with diskutil.

$ diskutil list /dev/disk0

zpool creation

Then create a zpool on your target partition. Here is the zpool command I used. Replace /dev/disk0s4 with your target partition. I use compression to get more space out of my SSD.

$ sudo zpool create -f \
  -o ashift=12 \
  -O atime=off \
  -O normalization=formD \
  -O compression=lz4 \
  tank /dev/disk0s4

A short explanation of my chosen options:

  • atime: don't write the access time to disk, make reads a bit faster
  • normalization=formD: how to store/encode file names
  • ashift=12: block size of the zfs filesystem, 12 is my prefered size for size/performence
  • compression=lz4: fast/intelligent compression, only compresses files where it makes sense to compress them

Create a zfs filesystem for the root of your vms. And than for the vm.

sudo zfs create tank/vm
sudo zfs create tank/vm/docker

docker vm

Now just create any flavor of a ubuntu 14.04 vm as usual with vmware and save it in tank/vm/docker. You now can snapshot, backup the vm with the power of zfs.

setup docker vm

I just released vm4docker which does all the heavy lifting for you customizable docker vm setup.

With this ansible rules for osx and your docker vm you get:

  • vmware based docker setup
  • docker-compose
  • nfs export of you home folder to your docker vm
  • certificate setup of server / client
  • sublime plugin for propagation of save events to the docker vm, convenient for auto-reload of dev enviroments

enjoy your flexible docker setup!