Can't update or install

English Install/Boot/Login
1

I had a cascade of failures that I eventually thought was a BIOS problem. The BIOS would not even diagnose. However, I did get it to run tonight and it says I have a CD/DVD Drive failure. That kept me from doing a last ditch effort to update through a reinstall. I also got a warning that the usb wasn’t working. When it did boot to Plasma, the screen did not fully draw the desktop properly, leaving incomplete and non-responsive boxes onscreen until the system froze and I had to power off to reboot. I did get a marginally functioning desktop when I booted into icewm. I disconnected everything to connect up an old pc for awhile and now, having reconnected everything to the troubled pc, I have Plasma back. !?! I did not attempt to reinstall via usb since, for a couple of days, usb drives were not recognized either. But for some reason, they are tonight.

When I try to do zypper dup, it fails as shown:

Failed to provide Package kernel-firmware-20190409-1.1.noarch (openSUSE-Tumbleweed-Oss). Do you want to retry retrieval?

[repo-oss|http://download.opensuse.org/tumbleweed/repo/oss/] Can’t provide file ‘./noarch/kernel-firmware-20190409-1.1.noarch.rpm’ from repository ‘repo-oss’
History:

  • Can’t hardlink/copy /var/adm/mount/AP_0x7E65EB/noarch/kernel-firmware-20190409-1.1.noarch.rpm to /var/cache/zypp/packages/repo-oss
  • Can’t provide ./noarch/kernel-firmware-20190409-1.1.noarch.rpm

Abort, retry, ignore? [a/r/i] (a):

This is package 15 of 360. Zypper reports it as completely downloaded. Previously, I tried to ignore and move on, but I got many instances of the same error on other files. I ran zypper clean with no noticeable effect. I also got warnings that zypper could not write to disk because it was full. The disk is not full. I don’t think that even the partition is full. But, if I run df, I get this:

Filesystem 1K-blocks Used Available Use% Mounted ondevtmpfs 4010068 0 4010068 0% /dev
tmpfs 4020368 19008 4001360 1% /dev/shm
tmpfs 4020368 1564 4018804 1% /run
tmpfs 4020368 0 4020368 0% /sys/fs/cgroup
/dev/sda6 41945088 41045168 82208 100% /
/dev/sda6 41945088 41045168 82208 100% /var/lib/mariadb
/dev/sda6 41945088 41045168 82208 100% /var/lib/mysql
/dev/sda6 41945088 41045168 82208 100% /var/opt
/dev/sda6 41945088 41045168 82208 100% /opt
/dev/sda6 41945088 41045168 82208 100% /var/log
/dev/sda6 41945088 41045168 82208 100% /var/cache
/dev/sda6 41945088 41045168 82208 100% /var/lib/machines
/dev/sda6 41945088 41045168 82208 100% /var/crash
/dev/sda6 41945088 41045168 82208 100% /boot/grub2/x86_64-efi
/dev/sda6 41945088 41045168 82208 100% /var/tmp
/dev/sda6 41945088 41045168 82208 100% /var/lib/libvirt/images
/dev/sda6 41945088 41045168 82208 100% /var/lib/mailman
/dev/sda6 41945088 41045168 82208 100% /var/lib/pgsql
/dev/sda6 41945088 41045168 82208 100% /var/lib/named
/dev/sda6 41945088 41045168 82208 100% /tmp
/dev/sda6 41945088 41045168 82208 100% /boot/grub2/i386-pc
/dev/sda6 41945088 41045168 82208 100% /srv
/dev/sda6 41945088 41045168 82208 100% /var/spool
/dev/sda6 41945088 41045168 82208 100% /usr/local
/dev/sda7 161349660 14483020 146866640 9% /home
tmpfs 804072 16 804056 1% /run/user/1000
/dev/sdc1 2003072 706752 1296320 36% /run/media/bill/84D6-4404

I don’t understand why every folder shows the same status, 100% used, with the same file sizes. Maybe I’m looking in the wrong place or don’t understand the representation. For example, Dolphin shows /boot/grub2/i386-pc as 1.8 MB
I don’t know if the machine will boot from a usb drive. It should, but in past attempts at fixing this problem, the usb was not recognized just like the failed CD/DVD drive. If I can copy the Tumbleweed net install to a usb drive, I would gladly do a reinstall if that would let me update.

2

What is your free space:


erlangen:~ # btrfs filesystem usage /mnt/
Overall:
    Device size:                  40.00GiB
    Device allocated:              6.30GiB
    Device unallocated:           33.70GiB
    Device missing:                  0.00B
    Used:                          5.18GiB
    Free (estimated):             34.71GiB      (min: 34.71GiB)
    Data ratio:                       1.00
    Metadata ratio:                   1.00
    Global reserve:               16.42MiB      (used: 0.00B)

Data,single: Size:6.01GiB, Used:5.00GiB
   /dev/sdc3       6.01GiB

Metadata,single: Size:264.00MiB, Used:188.06MiB
   /dev/sdc3     264.00MiB

System,single: Size:32.00MiB, Used:16.00KiB
   /dev/sdc3      32.00MiB

Unallocated:
   /dev/sdc3      33.70GiB
erlangen:~ #

What happens upon running “zypper refresh --force && zypper dup”?

3

That’s “btrfs”. The file system is logically divided into subvolumes, but they share the same pool of free blocks.

In short, your file system is full, and that is probably the cause of most (or all) of your problems. You need to delete some older snapshots to free up space. Maybe

snapper list

to list the snapshots, and

snapper delete (snapshot number)

to delete.

man snapper

for other options.

You can use the Yast module. But maybe try to free up a bit of working disk space first.

4

Well, its obvious that I’m out of space. Here is what I got when I ran your suggestion

linux:/home/bill # btrfs filesystem usage /mnt/
Overall:
Device size: 40.00GiB
Device allocated: 40.00GiB
Device unallocated: 1.00MiB
Device missing: 0.00B
Used: 38.87GiB
Free (estimated): 298.34MiB (min: 298.34MiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 59.09MiB (used: 0.00B)

Data,single: Size:38.60GiB, Used:38.30GiB
/dev/sda6 38.60GiB

Metadata,DUP: Size:687.00MiB, Used:287.92MiB
/dev/sda6 1.34GiB

System,DUP: Size:32.00MiB, Used:16.00KiB
/dev/sda6 64.00MiB

Unallocated:
/dev/sda6 1.00MiB

Next, I ran the zyyper commands you suggest. I got past the kernel firmware that stopped me before but it failed at the kernel update:

Failed to provide Package kernel-default-5.0.8-1.1.x86_64 (openSUSE-Tumbleweed-Oss).
Do you want to retry retrieval?

[repo-oss|http://download.opensuse.org/tumbleweed/repo/oss/] Can’t provide file ‘./x8
6_64/kernel-default-5.0.8-1.1.x86_64.rpm’ from repository ‘repo-oss’
History:

  • Can’t hardlink/copy /var/adm/mount/AP_0xTrVwFd/x86_64/kernel-default-5.0.8-1.1.x86
    _64.rpm to /var/cache/zypp/packages/repo-oss
  • Can’t provide ./x86_64/kernel-default-5.0.8-1.1.x86_64.rpm

Next, I will try the snapper commands as suggested below by nrickert.
Thanks for showing me these commands.

5

Here is what I get running snapper list

snapper list
The config ‘root’ does not exist. Likely snapper is not configured.
See ‘man snapper’ for further instructions

Looking at man snapper, I think I should create snapshots, but If I’m already out of room, is that the right choice?

6

Try “btrfs subvolume list -s”.

7

No, don’t do that.

I think the default during install is to setup snapshots, but only if the file system has enough space. Apparently yours doesn’t.

You could maybe check if there are some large files in “/tmp” that you can delete. And, if needed, you could delete older logs

rm /var/log/*.xz

The files there with names ending in “.xz” are compressed archives of older logs.

Perhaps also look at “/var/lib/systemd/coredump”.

8

The subvolume list command gave no results. Tried using the -a switch with same result. I did remove the .xz files but that seemed to make no difference in available space. Tried removing files from /tmp but there seem to be hundreds and many are directories, so even using wildcards, rm didn’t delete enough to make a difference.

I don’t know how I got to this point. Look at my avatar. I’m old and my memory is imperfect. But I seem to remember that I read TW would run with 20GB, so I made the partition 40GB to give it room. I don’t have this computer loaded up with a lot of stuff, but I will delete things if that will help. I suppose I could enlarge the partition for sda6, but partitioning has always scared me. I welcome all suggestions.

9

What about “du -csh /mnt/*”:


linux-udd7:~ # du -csh /mnt/*
1.2M    /mnt/bin
84M     /mnt/boot
0       /mnt/dev
20M     /mnt/etc
0       /mnt/home
974M    /mnt/lib
11M     /mnt/lib64
0       /mnt/mnt
0       /mnt/opt
0       /mnt/proc
11M     /mnt/root
0       /mnt/run
11M     /mnt/sbin
0       /mnt/selinux
0       /mnt/srv
0       /mnt/sys
0       /mnt/tmp
5.1G    /mnt/usr
303M    /mnt/var
6.4G    total
linux-udd7:~ #

The above is after running a massive zypper dup, which claimed some 2 G and released them after completion:

linux-udd7:~ # btrfs filesystem usage /mnt/
Overall:
    Device size:                  40.00GiB
    Device allocated:              8.55GiB
    Device unallocated:           31.45GiB
    Device missing:                  0.00B
    Used:                          6.26GiB
    Free (estimated):             33.40GiB      (min: 33.40GiB)
    Data ratio:                       1.00
    Metadata ratio:                   1.00
    Global reserve:               19.00MiB      (used: 0.00B)

**Data,single: Size:8.01GiB, Used:6.06GiB**
   /dev/sdc3       8.01GiB

Metadata,single: Size:520.00MiB, Used:209.73MiB
   /dev/sdc3     520.00MiB

System,single: Size:32.00MiB, Used:16.00KiB
   /dev/sdc3      32.00MiB

Unallocated:
   /dev/sdc3      31.45GiB
linux-udd7:~ #
10

What is the output from:

du -s /tmp/.
11

Here are the results of the commands:

linux:/home/bill # btrfs filesystem usage /mnt/
Overall:
Device size: 40.00GiB
Device allocated: 40.00GiB
Device unallocated: 1.00MiB
Device missing: 0.00B
Used: 39.08GiB
Free (estimated): 85.00MiB (min: 85.00MiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 59.34MiB (used: 0.00B)

Data,single: Size:38.60GiB, Used:38.51GiB
/dev/sda6 38.60GiB

Metadata,DUP: Size:687.00MiB, Used:288.09MiB
/dev/sda6 1.34GiB

System,DUP: Size:32.00MiB, Used:16.00KiB
/dev/sda6 64.00MiB

Unallocated:
/dev/sda6 1.00MiB
linux:/home/bill # du -csh /mnt/
0 /mnt/
0 total
linux:/home/bill # du -s /tmp/
25477256 /tmp/

Also, I ran du -csh /mnt/* but that said no such file or directory. I guess I thought /tmp/ would be much bigger. There must be some larger files that I’m unaware of.

12

In future, please use CODE tags instead of QUOTE tags for displaying command output.

This part:


linux:/home/bill # du -s /tmp/
25477256        /tmp/

is showing 25G of space tied up in “/tmp”.

I don’t know if there is anything important there. Most of what goes into “/tmp” can be removed after a reboot. And you have done several reboots.

If there is nothing that needs saving, you could do (as root):


/bin/rm -rf /tmp/*

And then reboot and see whether things look better.

NOTE: That command will remove everything in “/tmp” – well, almost everything. And it won’t prompt for each file. So be careful that you get the “/tmp/*” part correct before you hit enter.

There will be a few hidden files (name begins with “.”) in “/tmp” that won’t be removed. Usually those don’t take up much space. If they turn out to also be a problem, we can deal with those later.

The reboot after removing, is because some running programs might be using those files.

I guess I thought /tmp/ would be much bigger. There must be some larger files that I’m unaware of.

The output of “du” is in kilobytes. So that’s 25G, not 25M. That’s more than half of your 40G partition.

13

That command to clear tmp all at once is what I was looking for! Here is the result of running it:

**linux:/home/bill #** btrfs filesystem usage /mnt/
Overall:
    Device size:                  40.00GiB
    Device allocated:             40.00GiB
    Device unallocated:            1.00MiB
    Device missing:                  0.00B
    Used:                         14.71GiB
    Free (estimated):             24.39GiB      (min: 24.39GiB)
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:               32.80MiB      (used: 0.00B)

Data,single: Size:38.60GiB, Used:14.20GiB
   /dev/sda6      38.60GiB

Metadata,DUP: Size:687.00MiB, Used:259.20MiB
   /dev/sda6       1.34GiB

System,DUP: Size:32.00MiB, Used:16.00KiB
   /dev/sda6      64.00MiB

Unallocated:
   /dev/sda6       1.00MiB

Isn’t there a setting somewhere to make clearing of tmp during each reboot? I’m reluctant to resizing the partition if all I need to do is clear tmp periodically.

The update that I attempted is 360 files, maybe more by now. I will try to get that done as soon I post this.

Thank you for your patience and your help.

14

That (not requoting) looks a lot better.

Isn’t there a setting somewhere to make clearing of tmp during each reboot? I’m reluctant to resizing the partition if all I need to do is clear tmp periodically.

Yes.

Copy “tmp.conf” from “/usr/lib/tmpfiles.d” to “/etc/tmpfiles.d” and then edit the copy in “/etc/tmpfiles.d”.

Comments in the file, together with the manpage, tell you what to change. Basically, you change the prefix letter to “D” for “/tmp” to force it to be cleaned on boot.

15

Thank you for the tips on tmp. I will make the changes.

I ran zypper dup and updated 366 files without incident. After the reboot, I still show over 24Gb available! I feel so much better!

16

There is a timer activating the service:

linux:~ # systemctl list-timers systemd-tmpfiles-clean.timer
NEXT                          LEFT     LAST                          PASSED UNIT                         ACTIVATES
Sat 2019-04-27 11:05:48 CEST  15h left Fri 2019-04-26 11:05:48 CEST  8h ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service

1 timers listed.
Pass --all to see loaded but inactive timers, too.
linux:~ # systemctl status systemd-tmpfiles-clean.timer
● systemd-tmpfiles-clean.timer - Daily Cleanup of Temporary Directories
   Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-clean.timer; static; vendor preset: disabled)
   Active: active (waiting) since Fri 2019-04-26 10:50:47 CEST; 9h ago
  Trigger: Sat 2019-04-27 11:05:48 CEST; 15h left
     Docs: man:tmpfiles.d(5)
           man:systemd-tmpfiles(8)

Apr 26 10:50:47 hofkirchen systemd[1]: Started Daily Cleanup of Temporary Directories.
linux:~ #  journalctl -b -u systemd-tmpfiles-clean.service 
-- Logs begin at Fri 2019-04-19 23:18:41 CEST, end at Fri 2019-04-26 20:00:06 CEST. --
Apr 26 11:05:48 linux systemd[1]: Starting Cleanup of Temporary Directories...
Apr 26 11:05:48 linux systemd-tmpfiles[3277]: [/usr/lib/tmpfiles.d/var.conf:21] Duplicate line for path "/var/lib", ignoring.
....
Apr 26 11:05:48 linux systemd-tmpfiles[3277]: [/usr/lib/tmpfiles.d/var.conf:23] Duplicate line for path "/var/spool", ignoring.
Apr 26 11:05:48 linux systemd[1]: systemd-tmpfiles-clean.service: Succeeded.
Apr 26 11:05:48 linux systemd[1]: Started Cleanup of Temporary Directories.
linux:~ #

Daily cleanup keeps the size of tmp directories down to several 100M.

17

Thank you. I have only one timer listed. It is the one to clean the tmp files!

18

Cleanup of directories has changed. While older installations additionally rely on cron newer ones rely on systemd-tmpfiles only which by default leaves /tmp and /var/tmp alone. To scrub these too you may copy /usr/lib/tmpfiles.d/tmp.conf to /etc/tmpfiles.d/tmp.conf and add the following two bold lines :

erlangen:~ # cat /etc/tmpfiles.d/tmp.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

# See tmpfiles.d(5) for details

# Clear tmp directories separately, to make them easier to override
# SUSE policy: we don't clean those directories
q /tmp 1777 root root -
q /var/tmp 1777 root root -
**R /tmp - - - 10d**
**R /var/tmp - - - 10d**

# Exclude namespace mountpoints created with PrivateTmp=yes
x /tmp/systemd-private-%b-*
X /tmp/systemd-private-%b-*/tmp
x /var/tmp/systemd-private-%b-*
X /var/tmp/systemd-private-%b-*/tmp

# Remove top-level private temporary directories on each boot
R! /tmp/systemd-private-*
R! /var/tmp/systemd-private-*
erlangen:~ #

For checking what your modifications actually do run:

SYSTEMD_LOG_TARGET=console SYSTEMD_LOG_LEVEL=debug /usr/bin/systemd-tmpfiles --clean 2>&1|less
19

Your arcane knowledge is amazing!

My version of the tmp.conf is slightly different. The lines above the lines you say to insert do not begin with q as in your example. Instead, they begin with D. The “original” version, the one to be copied to /etc, do begin with q. I just don’t remember when I modified this file or if it makes a difference.

20

There is nice command to find out what has been changed:


erlangen:~ # systemd-delta -t overridden 
[OVERRIDDEN] /etc/tmpfiles.d/tmp.conf → /usr/lib/tmpfiles.d/tmp.conf

--- /usr/lib/tmpfiles.d/tmp.conf        2019-04-12 16:12:51.000000000 +0200
+++ /etc/tmpfiles.d/tmp.conf    2019-04-28 15:38:55.268410416 +0200
@@ -11,6 +11,8 @@
 # SUSE policy: we don't clean those directories
 q /tmp 1777 root root -
 q /var/tmp 1777 root root -
+e /tmp - - - 10d
+e /var/tmp - - - 10d
 
 # Exclude namespace mountpoints created with PrivateTmp=yes
 x /tmp/systemd-private-%b-*

[OVERRIDDEN] /etc/modules-load.d/sg.conf → /usr/lib/modules-load.d/sg.conf

--- /usr/lib/modules-load.d/sg.conf     2019-03-11 11:44:16.000000000 +0100
+++ /etc/modules-load.d/sg.conf 2019-04-28 15:37:32.389778290 +0200
@@ -1 +1,3 @@
-sg
+# load sg module at boot time
+# the line below can be removed or commented out on non-SCSI systems
+#sg


2 overridden configuration files found.
erlangen:~ #