BTRFS gotchas… (balance / scrub / snapshots / quota)

I’ve been using BTRFS for a few weeks now, and some bits are great (filesystem snapshots, dynamic resizing etc).

The “Good” and “Bad” things follow:

The bad things

Don’t use quota support and subvolumes. Once you start removing subvolumes, you’ll see kernel panics — at least up until (and including)l the 3.16 kernel (Debian Jessie). (see e.g. https://lists.debian.org/debian-kernel/2014/12/msg00310.html )

e.g.

WARNING: CPU: 2 PID: 19925 at /build/linux-LrLd2z/linux-3.16.5/fs/btrfs/qgroup.c:1347 btrfs_delayed_qgroup_accounting+0x46a/0xab0 [btrfs]()
....
Workqueue: btrfs-extent-refs btrfs_extent_refs_helper [btrfs]
....

You need a regular (weekly?) cron job to re-balance the filesystem. Else you’ll find yourself in the weird position of not being able to create files, yet ‘df’ shows there’s space left on the device.

Doing a “btrfs filesystem show /mount/point” will show it as having no space left —

For instance :
‘df’ output:

/dev/xvdf 41943040 26548956 14203460 66% /var/lib/lxc

while ‘btrfs’ itself says :

root@xxxxx:# btrfs filesystem show /var Label: 'VAR' uuid: 754f8e45-31b4-428f-a21f-2ad9b93b46f6 Total devices 1 FS bytes used 18.20GiB devid 1 size 40.00GiB used 40.00GiB path /dev/xvdf
Btrfs v3.17

To  fix this, you need ‘some’ free space for the balance to run …. so either nuke a few log files (if you’re lucky) or hope remounting the filesystem with ‘clear_cache’ is sufficient —

mount -o remount,clear_cache /var/

and then a :

btrfs balance start /var

I have a weekly cron like the following which is hopefully sufficient to control BTRFS —

for filesystem in $(mount -t btrfs | awk '{print $3}' )
do
    btrfs balance start $filesystem
done
for filesystem in $(mount -t btrfs | awk '{print $3}' )
do
    btrfs scrub start $filesystem
done

The good things

1. Read-only snapshots — great for backup jobs
2. Expandable / Resizeable … just add another disk / partition in…
3. No need to mess around with LVM / RAID