Have I destroyed my root partition by issuing wrong btrfs commands?

moshpuke · May 7, 2024, 4:25am

Here’s the summary:

Found root partition almost full so I tried to delete the snapshot ID ‘1’ (likely during the OS installation two years ago).
Deleting it in yast2-snapper failed (with some illegal snapshot error popup), so I asked a chatbot and it told me to use sudo btrfs subvolume delete /.snapshots/1/snapshot. But it was unsuccessful due to errors like “warning, not deleting the default subvolume ID ***”. Then I was told to set a new default volume by issuing sudo btrfs subvolume set-default ****. After that command my OS was already dying as all kinds of errors started to pop up, audio turning off, application closing etc.
I just reboot and everything looked fine again until I noticed I couldn’t change anything in root partition as they were all labled ‘read-only filesystem’. zypper dup would fail when it started to modify the system.
I then boot to a snapshot from yesterday labeled ‘pre-zypp’ and did a snapper rollback in it. But now the machine seems to be in the same state, as everything in root partition is in ‘read-only filesystem’.

At the moment I can still browser internet, play games but root partition is in a frozen state.

Any hope for rescue?

arvidjaar · May 7, 2024, 4:32am

After snapper rollback you need to reboot into the new read-write default subvolume created by this command.

moshpuke · May 7, 2024, 5:31am

I forgot to add, I did reboot after using snapper rollback.

Now when I use zypper install or zypper rm I got

Continue? [y/n/v/…? shows all options] (y):
error: can’t create transaction lock on /usr/lib/sysimage/rpm/.rpm.lock (Read-only file system)
(1/5) Removing phonon-vlc-lang-0.12.0-2.2.noarch …[error]
Removal of (3576)phonon-vlc-lang-0.12.0-2.2.noarch(@System) failed:
Error: Subprocess failed. Error: RPM failed: Command exited with status 1.

I tested to create a file under /root/Downloads, it still works there though.

I think it freeze all files that had been changed before. I never touched the /root/Downloads folder before. I guess that’s why.

I don’t care about all the historical snapshots if I can just start from now on if there’s a way.

arvidjaar · May 7, 2024, 5:33am

Show

findmnt --real
snapper list
btrfs subvolume list /
btrfs subvoilume get-default /

And use preformatted text to make this readable.

moshpuke · May 7, 2024, 6:14am

snapper list
    # | Type   | Pre # | Date                     | User | Used Space | Cleanup | Description     | Userdata     
------+--------+-------+--------------------------+------+------------+---------+-----------------+--------------
   0  | single |       |                          | root |            |         | current         |              
1253+ | single |       | Mon Mar 18 22:31:47 2024 | root |  16.00 KiB |         | Hardware_Change |              
1425  | pre    |       | Mon May  6 17:04:03 2024 | root |  16.00 KiB | number  | zypp(zypper)    | important=yes
1426  | post   |  1425 | Mon May  6 17:18:26 2024 | root |   2.64 MiB | number  |                 | important=yes
1427- | pre    |       | Mon May  6 17:22:33 2024 | root |  16.00 KiB |         | zypp(zypper)    | important=yes
1428  | post   |  1427 | Mon May  6 17:22:39 2024 | root | 289.36 MiB | number  |                 | important=yes
1436  | pre    |       | Tue May  7 06:22:49 2024 | root |  16.00 KiB | number  | zypp(zypper)    | important=yes
1439  | pre    |       | Tue May  7 08:24:15 2024 | root |  16.00 KiB | number  | zypp(zypper)    | important=no 
1440  | pre    |       | Tue May  7 08:24:49 2024 | root |  16.00 KiB | number  | zypp(zypper)    | important=no 
1441  | pre    |       | Tue May  7 08:44:36 2024 | root |  16.00 KiB | number  | zypp(zypper)    | important=no 
1442  | pre    |       | Tue May  7 08:49:26 2024 | root |  16.00 KiB | number  | zypp(zypper)    | important=no 


btrfs subvolume list /
ID 256 gen 122 top level 5 path @
ID 257 gen 990489 top level 256 path @/var
ID 258 gen 990053 top level 256 path @/usr/local
ID 259 gen 989918 top level 256 path @/srv
ID 260 gen 990476 top level 256 path @/root
ID 261 gen 989918 top level 256 path @/opt
ID 262 gen 989918 top level 256 path @/boot/grub2/x86_64-efi
ID 263 gen 989918 top level 256 path @/boot/grub2/i386-pc
ID 264 gen 990485 top level 256 path @/.snapshots
ID 272 gen 990200 top level 256 path @/swap
ID 1576 gen 990448 top level 264 path @/.snapshots/1253/snapshot
ID 1702 gen 989917 top level 257 path @/var/lib/machines
ID 1801 gen 990204 top level 264 path @/.snapshots/1425/snapshot
ID 1802 gen 989135 top level 264 path @/.snapshots/1426/snapshot
ID 1803 gen 990464 top level 264 path @/.snapshots/1427/snapshot
ID 1804 gen 989147 top level 264 path @/.snapshots/1428/snapshot
ID 1812 gen 990204 top level 264 path @/.snapshots/1436/snapshot
ID 1817 gen 990448 top level 264 path @/.snapshots/1441/snapshot
ID 1818 gen 990464 top level 264 path @/.snapshots/1442/snapshot


btrfs subvolume get-default /
ID 1576 gen 990448 top level 264 path @/.snapshots/1253/snapshot

I tried rolling back to the March’s single snapshot too and found that filesystem still ‘read-only’ so I rolled to yesterday again.

arvidjaar · May 7, 2024, 6:16am

I asked for command output, not a picture. I cannot read it; it is too tiny. And please use

findmnt --real -u

so the columns are not truncated.

moshpuke · May 7, 2024, 6:30am

findmnt --real -u
TARGET              SOURCE                                FSTYPE                                OPTIONS
/                   /dev/mapper/cr_root[/@/.snapshots/1427/snapshot]
                                                          btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=1803,subvol=/@/.snapshots/1427/snapshot
├─/.snapshots       /dev/mapper/cr_root[/@/.snapshots]    btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=264,subvol=/@/.snapshots
├─/srv              /dev/mapper/cr_root[/@/srv]           btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=259,subvol=/@/srv
├─/boot/grub2/i386-pc
│                   /dev/mapper/cr_root[/@/boot/grub2/i386-pc]
│                                                         btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=263,subvol=/@/boot/grub2/i386-pc
├─/boot/grub2/x86_64-efi
│                   /dev/mapper/cr_root[/@/boot/grub2/x86_64-efi]
│                                                         btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=262,subvol=/@/boot/grub2/x86_64-efi
├─/opt              /dev/mapper/cr_root[/@/opt]           btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=261,subvol=/@/opt
├─/var              /dev/mapper/cr_root[/@/var]           btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=257,subvol=/@/var
├─/swap             /dev/mapper/cr_root[/@/swap]          btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=272,subvol=/@/swap
├─/usr/local        /dev/mapper/cr_root[/@/usr/local]     btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=258,subvol=/@/usr/local
├─/root             /dev/mapper/cr_root[/@/root]          btrfs                                 rw,relatime,ssd,space_cache=v2,subvolid=260,subvol=/@/root
├─/run/user/1000/doc
│                   portal                                fuse.portal                           rw,nosuid,nodev,relatime,user_id=1000,group_id=1000
├─/tmp/.mount_gearle93kd83
│                   gearlever_joplin_bafe5c.appimage      fuse.gearlever_joplin_bafe5c.appimage ro,nosuid,nodev,relatime,user_id=1000,group_id=1000
├─/boot/efi         /dev/nvme1n1p1                        vfat                                  rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro
└─/home             /dev/mapper/cr_home                   ext4                                  rw,relatime,data=ordered
  ├─/home/Doe/mnt/Games
  │                 /dev/nvme0n1p1                        fuseblk                               rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096
  ├─/home/Doe/mnt/Data
  │                 /dev/sda1                             fuseblk                               rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096
  ├─/home/Doe/dir1
  │                 asdflaksdf:sdfas                      fuse.rclone                           rw,nosuid,nodev,relatime,user_id=1000,group_id=1000
  └─/home/Doe/dir2
                    asdf:asdfdf                           fuse.rclone                           rw,nosuid,nodev,relatime,user_id=1000,group_id=1000

arvidjaar · May 7, 2024, 12:39pm

It is unclear what you did. Anyway - currently you are booted into snapshot 1427. So post complete recording (complete command line and output) of

snapper rollback
snapper list
btrfs subvolume get-default /

in this order. Do not reboot.

moshpuke · May 8, 2024, 1:49am

Thanks for helping. I did quite a lot during this time and decided to do a reinstall, which is long due as well.

All I need to figure out now is how snapshots ate up the root drive…
I just read that if you create a single snapshot manually without adding a custom clean-up algo, that snapshot would never be deleted. I guess that’s what happened to me since I created one in March?