Although everything seems to work fine I get a lot of BTRFS log entries like these:
[ 99.376154] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 102.366885] BTRFS info (device nvme0n1p1): scrub: started on devid 1
[ 115.587706] BTRFS warning (device nvme0n1p1): checksum error at logical 1603822157824 on dev /dev/nvme0n1p1, physical 54848913408: metadata node (level 1) in tree 7761
[ 115.587711] BTRFS warning (device nvme0n1p1): checksum error at logical 1603822157824 on dev /dev/nvme0n1p1, physical 54848913408: metadata node (level 1) in tree 7761
[ 115.587714] BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 0, rd 0, flush 0, corrupt 5, gen 0
[ 115.587721] BTRFS error (device nvme0n1p1): unable to fixup (regular) error at logical 1603822157824 on dev /dev/nvme0n1p1
[ 122.823014] BTRFS info (device nvme0n1p1): scrub: finished on devid 1 with status: 0
[ 130.101416] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 160.897823] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 191.549999] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 222.292217] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 253.008953] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
The SSD is brand new but cloned (with Clonezilla) from my old SSD.
Is there anyway to find out what file causes the loggings?
The documentation for btrfs check with repair option is really scary, like “Don’t do it unless you really know what you are doing.”. I really don’t so I won’t!
As far as Btrfs is concerned, the first two things to regularly perform are:
“btrfs balance /”
“btrfs scrub /”
Each with the correct syntax and path – I’ve pointed to the “normal” small system path – the machine’s root directory but, that may well be not the case for your system – only you know which partitions on your system are using the Btrfs file system …
Alternatively, you can manually start the systemd “btrfs-balance.service” and “btrfs-scrub.service” services – first one – wait for it to complete and then, the next one …
If the “normal” Btrfs housekeeping doesn’t alleviate the issue, you’ll have to invoke the “Btrfs check” routine to find the bad block in the file system –
Be aware that, “btrfs check «device»” usually needs that file system to be unmounted –
Which, is bad news for a file system which happens to be being used as the system partition.
In an ideal world, if the file system with bad blocks is being used for the system partition, you’ll have to remove the disk and, check it with another system.
If you don’t have another system to hand, you can try to use “btrfs check --force «device»” on the one-and-only system but, please be aware that, nobody recommends this solution.
I have a fresh OpenSUSE USB that I can use for rescue mode and have run btrfs check on the unmounted partition. My problem is what to do about the errors that get reported.
The btrfs check documentation, btrfs-check(8) — BTRFS documentation, warns about using --repair but says very little about when to use it and when it might be save, if ever.
Since my system seems to work fine I am reluctant to use any dangerous commands that I don’t know will solve the problem. I was hoping that there is a way to determine which file is affected and maybe correct the problem by deleting or replacing the file.
I have tried to find if there is an unreadable file with
find /xxxx -type f -exec cp {} /dev/null ;
where xxxx is all directories in / except dev, proc and sys. No error was reported.
Apparently all files on the root partition can be accessed and fully read, still I get all those BTRFS errors logged.
I tried a btrfs balance which failed:
[ 7533.546968] BTRFS info (device nvme0n1p1): balance: start -d -m -s
[ 7533.547067] BTRFS info (device nvme0n1p1): relocating block group 1607378927616 flags data
[ 7534.580977] BTRFS info (device nvme0n1p1): found 177 extents, stage: move data extents
[ 7534.701963] BTRFS info (device nvme0n1p1): found 177 extents, stage: update data pointers
[ 7534.765461] BTRFS info (device nvme0n1p1): relocating block group 1607345373184 flags system
[ 7534.765569] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 7534.797513] BTRFS info (device nvme0n1p1): found 1 extents, stage: move data extents
[ 7534.825954] BTRFS info (device nvme0n1p1): relocating block group 1604895899648 flags data
[ 7535.217849] BTRFS info (device nvme0n1p1): found 4 extents, stage: move data extents
[ 7535.328831] BTRFS info (device nvme0n1p1): found 4 extents, stage: update data pointers
[ 7535.394330] BTRFS info (device nvme0n1p1): relocating block group 1603822157824 flags metadata
[ 7535.394405] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 7535.413117] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 7535.413188] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 7535.422930] BTRFS info (device nvme0n1p1): balance: ended with status: -5
You’ll need to check how often “btrfs trim” has been executed on that device –
If not often enough then, first “trim” to try and force the drive’s logic to correct the bad block.
If that doesn’t alleviate the issue, you’ll have to execute “check --repair”.
Assuming that, the original SSD is still available, if the Btrfs repair doesn’t help, you’ll have to clone again after initialising the new drive –
Tried fstrim. No errors but no apparent improvement.
A scrub after the fstrim gave this:
[10468.250360] BTRFS info (device nvme0n1p1): scrub: started on devid 1
[10481.618359] BTRFS warning (device nvme0n1p1): checksum error at logical 1603822157824 on dev /dev/nvme0n1p1, physical 54848913408: metadata node (level 1) in tree 7761
[10481.618366] BTRFS warning (device nvme0n1p1): checksum error at logical 1603822157824 on dev /dev/nvme0n1p1, physical 54848913408: metadata node (level 1) in tree 7761
[10481.618369] BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 0, rd 0, flush 0, corrupt 7, gen 0
[10481.618394] BTRFS error (device nvme0n1p1): unable to fixup (regular) error at logical 1603822157824 on dev /dev/nvme0n1p1
[10485.066091] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[10488.958008] BTRFS info (device nvme0n1p1): scrub: finished on devid 1 with status: 0
Maybe I’ll try “btrfs check --repair” if I can’t somehow find a safer option.
I have tried btrfs balance. Fails immediately with:
[ 787.870721] BTRFS info (device nvme0n1p1): balance: start -d -m -s
[ 787.871026] BTRFS info (device nvme0n1p1): relocating block group 1611774558208 flags system
[ 787.932191] BTRFS info (device nvme0n1p1): relocating block group 1610700816384 flags data
[ 788.342357] BTRFS info (device nvme0n1p1): found 6 extents, stage: move data extents
[ 788.422342] BTRFS info (device nvme0n1p1): found 6 extents, stage: update data pointers
[ 788.478359] BTRFS info (device nvme0n1p1): relocating block group 1603822157824 flags metadata
[ 788.478388] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 788.509005] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 788.509055] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
[ 788.517870] BTRFS info (device nvme0n1p1): balance: ended with status: -5
[ 793.642038] BTRFS error (device nvme0n1p1): bad tree block start, want 1603822157824 have 0
I have tried btrfs scrub. Fails with:
scrub done for 02e6a056-947f-41c5-a1f0-bf3a613864d4
Scrub started: Thu Apr 27 14:01:05 2023
Status: finished
Duration: 0:00:21
Total to scrub: 80.03GiB
Rate: 3.12GiB/s
Error summary: csum=1
Corrected: 0
Uncorrectable: 1
Unverified: 0
ERROR: there are uncorrectable errors
My old SSD, which I cloned with Clonezilla, has a healthy root partition. No errors reported from btrfs check/balance/scrub. I guess I could try “btrfs check --repair” and if it destroys my root partition I can clone it again.
Before I do that I’ll try the BTRFS mailing list to see if anyone knows if check/repair is the best option or if there is a safer way.
I will also try to find the best way to clone a BTRFS root partition since Clonezilla may have given me this bad clone.
I personally have an issue with cloning disks → the bad block table …
If the source disk isn’t so new and, it’s beginning to develop bad blocks then, any attempts to clone that disk to another (hopefully healthy) disk may well run into problems – especially if, the bad blocks are appearing in the file system’s data structures …
OK, OK, current disk cloning tools are clever enough to not try and simply clone block-for-block but,
I have some reservations about attempting to clone modern Copy on Write (CoW) file systems such as Btrfs –
Is there really a guarantee that, the file management being performed by file systems such as Btrfs is really being transferred to the new disk properly?
Thanks, I’ll look into that next time I need to clone a BTRFS partition.
For now, I’m up and running with a healty root partition. Gave up on trying to repair it and re-cloned it again from my old SSD, with Clonezilla. No problem this time.