Corrupted btrfs with a kernel panic at boot – What to do?

I used the 3.8 kernel on Ubuntu Server and after a crash I couldn’t boot anymore.
The error contained “remove_from_bitmap” and an OOPS, similar to this guys:

I tried mounting with ro,recovery, or with clear_cache, but nothing helped. I was getting:

I keep the drive in btrfs RAID1, but unfortunately the error was happening on both drives. Mounting degraded was no help either. I’ve upgraded to the latest kernel, 3.11, but nothing’s changed.

What finally helped was this.


Most of my files are now OK, unfortunately my InnoDB file from MySQL got corrupted and I have to recover from a backup.
The tree still had some problems after running btrfsck (it segfaults after some time), so I decided to redo the whole structure, recreate all the subvolumes and copy its contents. In order to degrade the RAID you need to mount it first (see the logic, wtf!). Fortunately it already mounts.

Not to mention, it doesn’t work. Apparently btrfs RAID1 must have minimum of two devices. You can still force it to degrade if you null the magicstring from the superblock with:

If you need, you can restore with:

Then I just recreated the filesystem manually, transfered the files from the broken FS to another one. Afterwards mkfs.btrfs on the broken one to clean it, and extend the first one to make use of it as RAID1 again.

I’d say btrfs is unstable and I don’t recommend using it in production, unless you keep current backups and are prepared for some downtime in case of a crash. Don’t ever store MySQL/MariaDB inside one. Until they fix it.