BTRFS gotchas… (balance / scrub / snapshots / quota)

I’ve been using BTRFS for a few weeks now, and some bits are great (filesystem snapshots, dynamic resizing etc).

The “Good” and “Bad” things follow:

The bad things

Don’t use quota support and subvolumes. Once you start removing subvolumes, you’ll see kernel panics — at least up until (and including)l the 3.16 kernel (Debian Jessie). (see e.g. https://lists.debian.org/debian-kernel/2014/12/msg00310.html )

e.g.

WARNING: CPU: 2 PID: 19925 at /build/linux-LrLd2z/linux-3.16.5/fs/btrfs/qgroup.c:1347 btrfs_delayed_qgroup_accounting+0x46a/0xab0 [btrfs]()
....
Workqueue: btrfs-extent-refs btrfs_extent_refs_helper [btrfs]
....

You need a regular (weekly?) cron job to re-balance the filesystem. Else you’ll find yourself in the weird position of not being able to create files, yet ‘df’ shows there’s space left on the device.

Doing a “btrfs filesystem show /mount/point” will show it as having no space left —

For instance :
‘df’ output:

/dev/xvdf 41943040 26548956 14203460 66% /var/lib/lxc

while ‘btrfs’ itself says :

root@xxxxx:# btrfs filesystem show /var Label: 'VAR' uuid: 754f8e45-31b4-428f-a21f-2ad9b93b46f6 Total devices 1 FS bytes used 18.20GiB devid 1 size 40.00GiB used 40.00GiB path /dev/xvdf
Btrfs v3.17

 

To  fix this, you need ‘some’ free space for the balance to run …. so either nuke a few log files (if you’re lucky) or hope remounting the filesystem with ‘clear_cache’ is sufficient —

mount -o remount,clear_cache /var/

and then a :

btrfs balance start /var

I have a weekly cron like the following which is hopefully sufficient to control BTRFS —

for filesystem in $(mount -t btrfs | awk '{print $3}' )
do
    btrfs balance start $filesystem
done
for filesystem in $(mount -t btrfs | awk '{print $3}' )
do
    btrfs scrub start $filesystem
done

The good things

  1. Read-only snapshots — great for backup jobs
  2. Expandable / Resizeable … just add another disk / partition in…
  3. No need to mess around with LVM / RAID

2 thoughts on “BTRFS gotchas… (balance / scrub / snapshots / quota)”

  1. So, not so sure this is a great thing… I have 2 8TB disks that are one btrfs filesystem. According to the manual, the balance command rewrites the entire filesystem (not so good).. I’ve not tried the scrub command yet, so not sure what it will do…

  2. So since I wrote the above post ….

    a. A Balance is only technically necessary if all the output from e.g. btrfs fi usage /path/to/filesystem shows it as being full (no unallocated space). If it’s a busy filesystem then you’ll need to do something probably weekly or monthly. If not, there may be no need.

    b. You can specify filters on a balance – so e.g. only rebalance (rewrite) the chunks that are e.g. 10% full – btrfs balance start -dusage=10 -musage=10 /path/to/filesystem – this will be a lot quicker and not need to do the entire disk.

Leave a Reply

Your email address will not be published. Required fields are marked *