Tumbleweed update - stalled boot

[idee]

Performed a zypper dup Tumbleweed update.
My root partition needs more space.
I rebooted into another partition with another installation LXQT and used the partition tool in YaST to expand the root.
I then used blkid to update and verify the fstab in the primary partition.
Rebooting to that install, it stalls with the Cylon red eye "working" status looking for the root partition.

OK ] Started Show Plymouth Boot Screen.
OK ] Started Forward Password Requests to Plymouth Directory Watch.
OK ] Reached target Paths.
OK ] Found device SAMSUNG_HD161GJ 2.
OK ] Reached target Initrd Root Device.
OK ] Finished dracut initqueue hook.
OK ] Reached target Remote File Systems (Pre).
OK ] Reached target Remote File Systems.
*** ] A start job is running for /dev/disk/by-uuid /9555609 ..... (10h 42min 52s / no limit)

the *** is the red Cylon searching eye and it never stops, stalls there.

Looks like it is not finding the target.
I assume there might be something in Grub or Boot that needs to be updated.

Any help is appreciated.
Thanks

[idee]

Any thoughts for help? Tks

[tsu2]

Looks to me that the filesystem is damaged and might be running a fsck.
What is the format? BTRFS?

You may need to just wait until the fsck completes, and presumably should repair any problems it encounters.
Yes... If your partition/filesystem is very big and your drive is slow, it can take a very, very long time.

Cross your fingers, or do anything else if you're superstitious... and do something else while you're waiting.
And hope that you won't still have problems when the task completes.

TSU

[idee]

Thanks,
I set it up to run overnight. Hopefully it will clean itself out soon.
The partition is ext4
Kernel is 5.8.14.1-default

Is there anything I can do to clean off space on the root drive?

Below are the two fstabs on my system.

My primary, that was updated and has the problem:

# /etc/fstab: static file system information.#
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
UUID=6dae7c25-d985-44e6-b386-23c7d3a7af64 / ext4 noatime,discard 0 1
UUID=7b77b22b-a19f-4f0e-ab29-ae35449f5f39 swap swap defaults 0 0
UUID=74d45143-adf6-41b3-be64-dbf00eec672e swap swap defaults 0 0
UUID=9e4b2cbe-4227-49f6-bc01-c133785b592d /home ext4 noatime,discard 0 2
UUID=48ABE0A128C0D54D /backups ntfs-3g uid=dad,gid=users,umask=0022 0 2
UUID=420EFA6B0EFA56FF /windows-D ntfs-3g uid=dad,gid=users,umask=0022 0 2
UUID=F84ECD3B4ECCF402 /windows-C ntfs noatime 0 2
UUID=c89bdf78-0c7b-4cb2-94af-7505756efe72 /linux-back9-LXQT ext4 noatime 0 2
tmpfs /tmp tmpfs noatime,mode=1777 0 0

My backup that does boot and has not been updated recently:

# /etc/fstab: static file system information.#
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
UUID=F84ECD3B4ECCF402 /windows-C ntfs noatime 0 2
UUID=6dae7c25-d985-44e6-b386-23c7d3a7af64 /linux-front9-Cinnamon ext4 noatime 0 2
UUID=7b77b22b-a19f-4f0e-ab29-ae35449f5f39 swap swap defaults 0 0
UUID=74d45143-adf6-41b3-be64-dbf00eec672e swap swap defaults 0 0
UUID=9e4b2cbe-4227-49f6-bc01-c133785b592d /home ext4 noatime,discard 0 2
UUID=48ABE0A128C0D54D /backups ntfs-3g uid=dad,gid=users,umask=0022 0 2
UUID=420EFA6B0EFA56FF /windows-D ntfs-3g uid=dad,gid=users,umask=0022 0 2
UUID=c89bdf78-0c7b-4cb2-94af-7505756efe72 / ext4 noatime,discard 0 1
tmpfs /tmp tmpfs noatime,mode=1777 0 0

[karlmistelberger]

Thanks,
I set it up to run overnight. Hopefully it will clean itself out soon.
The partition is ext4
Kernel is 5.8.14.1-default

Is there anything I can do to clean off space on the root drive?

Below are the two fstabs on my system.

My primary, that was updated and has the problem:



My backup that does boot and has not been updated recently:

Boot into the other partition, mount the broken system and show used disk space:

3400G:~ # mount /dev/sdb2 /mnt/
3400G:~ # df -h /mnt/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb2        30G  5.8G   23G  21% /mnt
3400G:~ # du -hd1 -t1m /mnt/
45M     /mnt/boot
288M    /mnt/var
20M     /mnt/etc
709M    /mnt/lib
9.8M    /mnt/sbin
11M     /mnt/lib64
11M     /mnt/root
4.7G    /mnt/usr
5.8G    /mnt/
3400G:~ #

You may access the journal by running

journalctl --directory /mnt/var/log/journal/..../ -b

Go to the end of the journal and report.

[mrmazda]

Is there anything I can do to clean off space on the root drive?

• If your journal is persistent, it could be gobbling a huge amount of space. If you are able to boot it, then use journalctl to manage it. If you have to boot your other installation, but it's mountable, then you can do it manually by deleting old files from the subdirectory located in /var/log/journal/.
• You can buy a little space until next dup by removing /var/log/updateTestcase*.
• Check to see if purge-kernels service has been running. Two installed kernels for most people is plenty.
• Inspect /tmp/ for old files, and delete if any are present.
• Most people need only the latest version of libLLVM. If you have more than one installed, consider uninstalling the older.

[karlmistelberger]

If your journal is persistent, it could be gobbling a huge amount of space. If you are able to boot it, then use journalctl to manage it. If you have to boot your other installation, but it's mountable, then you can do it manually by deleting old files from the subdirectory located in /var/log/journal/.

Manual deletion is tedious. Use

journalctl --directory /mnt/var... --disk-usage

to display size and

journalctl --directory /mnt/var... --vacuum-size ...

for freeing disk space.

Inspect /tmp/ for old files, and delete if any are present.

Remove /tmp form /etc/fstab. This moves /tmp to RAM. You will no longer need to worry about its growth.

[susejunky]

... I rebooted into another partition with another installation LXQT and used the partition tool in YaST to expand the root.

When a partition gets resized the file system on that partition needs to be resized as well and that might take time.

I guess YaST will run resize2fs for you but probably in the background. So chances are that the resizing was not finished yet when you left your LXQT installation and you ended up with a (very) unclean file system and now an fsck is running.

Regards

susejunky

[mrmazda]

Manual deletion is tedious.

I can have it done faster with MC than it takes to find out how using the journalctl man page or a web search for the missing examples typical of man pages, or to learn vacuum means recover wasted space. A dozen or fewer keystrokes to get there and do it is far less tedium than man page or web searching.

[karlmistelberger]

I can have it done faster with MC than it takes to find out how using the journalctl man page or a web search for the missing examples typical of man pages, or to learn vacuum means recover wasted space. A dozen or fewer keystrokes to get there and do it is far less tedium than man page or web searching.

I read in the man page once about --vacuum in 2014 and remember since then. When unsure I hit tab to list the options (1 keystroke):

i3-4130:~ # journalctl --
--after-cursor    --disk-usage      --force           --list-boots      --no-tail         --rotate          --update-catalog  --verify-key
--all             --dmesg           --full            --list-catalog    --output          --setup-keys      --user            --version
--boot            --dump-catalog    --grep            --local           --output-fields   --show-cursor     --user-unit       
--case-sensitive  --field           --header          --machine         --pager-end       --since           --utc             
--catalog         --fields          --help            --merge           --priority        --sync            --vacuum-files    
--cursor          --file            --identifier      --no-full         --quiet           --system          --vacuum-size     
--cursor-file     --flush           --interval        --no-hostname     --reverse         --unit            --vacuum-time     
--directory       --follow          --lines           --no-pager        --root            --until           --verify          
i3-4130:~ # journalctl --