upgrade failure from Leap 15.1 with disk space warning on /boot

I want to upgrade from Leap 15.1 to 15.2 on a machine with the following device layout

> lsblk
NAME                                       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                                          8:0    0 149.1G  0 disk
├─sda1                                       8:1    0   156M  0 part  /boot
├─sda2                                       8:2    0    22G  0 part
│ └─cr_ata-WDC_WD1600BEVT-22ZCT0_WD-WXE708N10973-part2
│                                          254:0    0    22G  0 crypt
│   ├─system-root                          254:1    0  19.9G  0 lvm   /
│   └─system-swap                          254:2    0     2G  0 lvm   [SWAP]
└─sda4                                       8:4    0   127G  0 part
  └─cr_ata-WDC_WD1600BEVT-22ZCT0_WD-WXE708N10973-part4
                                           254:3    0   127G  0 crypt
    └─system-home                          254:4    0   127G  0 lvm   /home
sr0                                         11:0    1  1024M  0 rom

Disk space usage is reported as

> df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 2.0G     0  2.0G   0% /dev
tmpfs                    2.0G     0  2.0G   0% /dev/shm
tmpfs                    2.0G  1.5M  2.0G   1% /run
tmpfs                    2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/mapper/system-root   20G   13G  5.8G  69% /
/dev/sda1                148M   42M   96M  31% /boot
/dev/mapper/system-home  125G   49G   71G  41% /home
tmpfs                    393M   12K  393M   1% /run/user/1000

This layout has been used ever since openSuSE 13.2, where it was suggested by the installer. The /boot partition is not too large, but I always managed to update kernel versions, with keeping the running and one older kernel.
Now I use the USB installer downloaded from openSuSE, and it works fine until I receive the system probing screen, when it reports that I am out of disc space ob /boot (160% usage). Booting back into Leap 15.1, I deleted the older kernel version to gain some space, and repeated the attempt to upgrade. The same error turns up, now the usage of /boot is reported to be reduced to 123%, but still more then 100%. I can’t provide a screen shot during the upgrade, but here is the contents of the pop-up window shown:

Disk Space Warning - Yast2
Error: Out of disk space!
                           Free         Total
/boot    123%       -27.2 MiB   120.1 MiB
/          71%         5.0 GiB      17.6 GiB
/home  41%         69.1 GiB    117.5 GiB

However, device /dev/sda1 mounted on /boot still is at only 31% according to df -h. The installer probably adds up the memory already used and the additional one that will be taken up by the new version, for /boot the kernel still left from Leap 15.1 + the new one from the Leap 15.2 upgrade, but that should not add up to 123% if the new kernel is not much larger in size than the previous ones. Something seems to be inconsistent here.

Is there something wrong in the disk space calculation in the installer? Anyway, what can be done about this, except doing away with the upgrade and starting a totally new install of the machine with 15.2.

Thanks for any help.

A 15.2 kernel rpm is 152% of the size of a 13.2 kernel rpm. Kernels & initrds have been growing precipitously.

Just how big is/are your 15.1 initrd(s)? You may be able to reconfigure dracut to make them smaller. This one includes RAID support, but not LVM, nor plymouth:

# ll /boot/i*28.52*
-rw------- 1 root root 9189752 Jun 25 16:34 /boot/initrd-4.12.14-lp151.28.52-default

For minimizing size, my /etc/dracut.conf.d/dmodules.conf contains:

omit_drivers+="i18n uefi-lib encryptfs btrfs iscsi lvm lvm2 dmraid plymouth resume sata_sil usb_storage "

You could try a zypper online distribution upgrade with the kernel locked, so that the upgrade result would still be running a 15.1 kernel. Then you could reduce the installed kernels to one, if not already done, and a 15.2 kernel should fit via an ordinary zypper up after removing the kernel lock. I do all my OS (online, which is normal/common procedure here) upgrades with kernel locked, so I know it’s not a problem to do.

Long term, if not immediately, /boot needs to be larger. You should be able to shrink the sda2 lvm at its front, backup the content of sda1, delete and recreate sda1 at a larger size, then restore sda1 content, with plenty of room on /boot/ resulting.

A 15.1 install here with two kernels installed totals about 75 MB.
A 15.2 fresh install with two kernels istalled (the one from the DVD and one update) totals just in excess of 84 MB.
If your sda1 mounted on /boot is really 156MB two or even three 15.2 kernels should fit easily, unless you have an unusual configuration, e.g. very large initrd’s.
Apparently, as you write, something doesn’t add up.

Thanks for the replies so far.
While I agree with mrmazda that on the long run, extending the /boot partition would be desirable (and that might indeed be the way to go), I would be interested to find out what is going on with respect to the size counts here.

The single kernel installed on 15.1 in my present configuration uses a total of about 42 MB (according to df -h corresponding to 31% as mentioned in the original post). The size of the /boot contents is:

> ll /boot
total 33324
-rw-r--r-- 1 root root      512  4. Aug 2017  backup_mbr
lrwxrwxrwx 1 root root        1  4. Aug 2017  boot -> .
-rw-r--r-- 1 root root     1725 13. Jul 2019  boot.readme
-rw-r--r-- 1 root root   200382 11. Jun 09:16 config-4.12.14-lp151.28.52-default
drwxr-xr-x 2 root root     1024  4. Aug 2017  grub
drwxr-xr-x 6 root root     1024 19. Jul 20:37 grub2
lrwxrwxrwx 1 root root       34  1. Jul 16:59 initrd -> initrd-4.12.14-lp151.28.52-default
-rw------- 1 root root 14170268 17. Jul 20:12 initrd-4.12.14-lp151.28.52-default
drwx------ 2 root root    12288 12. Feb 2015  lost+found
-rw-r--r-- 1 root root   407933 11. Jun 09:25 symvers-4.12.14-lp151.28.52-default.gz
-rw-r--r-- 1 root root      484 11. Jun 09:25 sysctl.conf-4.12.14-lp151.28.52-default
-rw-r--r-- 1 root root  3646301 11. Jun 09:24 System.map-4.12.14-lp151.28.52-default
-rw-r--r-- 1 root root  8349426 11. Jun 09:28 vmlinux-4.12.14-lp151.28.52-default.gz
lrwxrwxrwx 1 root root       35  1. Jul 16:59 vmlinuz -> vmlinuz-4.12.14-lp151.28.52-default
-rw-r--r-- 1 root root  7327856 11. Jun 09:47 vmlinuz-4.12.14-lp151.28.52-default

I would expect that I should be able to fit the 15.2 kernel, if the initrd and kernel are not more than twice the size of the 15.1 ones. Still, the 15.2 installer reports the disk space warning, mentioned in the original post. In OrsoBruno’s case the increase seems to be more like 15%, which should fit into the available space easily. Any idea what is going on? Do we need extra space in /boot during the upgrade, which must be taken into consideration by the installer?

Now before asking for some advice on possible repartitioning, I’d be curious if we needto keep the gz versions of the symvers and vmlinuz of the 15.1 from which I start the upgrade, or could this be deleted.

Puzzled by this problem I did some tests in a VM.
First I moved everything but the booting kernel (vmlinuz-xxx), its initrd and their symlinks out of the /boot folder and, as expected, the system booted happily. Please note that you might need some of the other files to build additional modules etc. though.
Then I tried a fresh install with a /boot partition and the installer didn’t accept anything short of 110 MB and refused to go ahead otherwise; so I guess that you have to provide at least some 110 MB of free space.
By the way, you may try to reduce the size of the initrd by running the “dracut --hostonly --force” command that includes only modules needed to boot on the current system.
Interestingly, the installer also wanted an 8MB “BIOS boot partition” to proceed, apparently without means to create one other than by accepting the one proposed originally.

My take is that the new installer has (too many?) safeguards and maybe a bug or two, but maybe members with more insight will comment on that.

Update from further tests.
I was able to install 15.1 with a 110 MB /boot (formatted to EXT4, this seems to have a role). Then the 15.2 DVD installer let me proceed with an upgrade, but later installation of the 5.3.x kernel from 15.2 failed.
Ignoring that and going forward left me with a 15.2 system… happily booting with a 4.12 kernel from 15.1!
Trying to install from scratch 15.2 with a 110 MB /boot is stopped by a “not enough space on /boot…”.
Install with a 150 MB /boot (EXT4) succeeds with a warning of “only 20 MB remaining on /boot”, but then any kernel update aborts with “xxx MB needed on /boot…” errors.
Eventually I succeeded updating the kernel by deleting everything but the /boot/grub2 folder from /boot just before issuing “zypper up” but that way you are left with an unbootable system if anything goes wrong.

All in all, maybe you can find a workaround to upgrade to 15.2 with a 150 MB /boot, but then you are left with a system almost impossible to maintain.
(well, unless somebody comes up with some other piece of magic…)

I ran into this problem with openSUSE 12.3. I had been using “/boot” at 100M, which seemed to have oodles of space to spare back when I started doing that (maybe SuSE 10.2). But with 12.3, it was tight. There was space for two kernels, but not quite enough for three. So I got into the practice of manually deleting the oldest kernel before a kernel update.

When 13.1 came out, I did some repartitioning to enlarge “/boot” to at least 200M.

On my current desktop, where 12.3 was the first version installed, I went with 500M for “/boot”. And I’ll note that I only use a separate “/boot” because of the encryption.

Part of the problem is that when a kernel is installed, the size of the “initrd” is not known. So the space estimate is only an estimate, and perhaps they overestimate the amount of space needed.

IME, a fresh installation always generates a considerably larger initrd than those next generated after installation, whether with same kernel, or newer. IIRC, the first is more than double in size, possibly close to triple, of those generated later.

Thanks for the further comments and suggestions.

I will do some tests myself, e.g. decreasing the size of the initrd, moving out the gz files, but this will take me a day or two. I’ll report back the results.
Unfortunately, I did not take notes about the sizes of files in /boot in earlier installations, however, if I manage to get the update to 15.2. done, it will be interesting to see how big the kernel size is, and compare it to the 15.1 one on this machine.

In the meantime, one more question: is the old kernel (here 15.1) still needed during the upgrade? Is this only as a backup if something goes wrong? Then, it could be moved from /boot into some backup just before the upgrade is attempted (possibly using a live USB stick). But maybe there is some other reason that it has to stay.

Not all systems are equal, but here I see that the 15.2 vmlinuz is about 9 MB vs. 7.3 MB for 15.1, while the initrd’s are about the same at 13.5 MB or so. Apparently temporary files during the install make a material difference.
Also the .gz and system map files are several MB larger in 15.2

As for the 15.1 kernel, like my test previously reported shows, it is not needed if you do a DVD upgrade, and is not needed either after boot up even if you do a “zypper dup”, but then you are left with no backup in case of trouble.

Another option would be to create a new partition somewhere else, mount it as /boot and ignore or delete the old /boot partition. I have not tested that yet, but during an upgrade what really matters is the / (root) partition and you can reuse that without formatting.
Maybe you have to adjust or reinstall GRUB if you go that route, and anyway I would test the whole process in a VM beforehand.

Here are the results of two tests to decrease the size of the /boot contents:

  1. moved out the g-zipped symvers and vmlinux - the usage of /boot goes back to 24%; upgrade to 15.2 still fails with an estimated usage of /boot of 116%
  2. in addition: dracut --hostonly --force - this results in only a marginal decrease of initrd from 1.417 to 1.416 MB, so I didn’t even try to attempt the upgrade with that

So, in conclusion up to this point: there seem to be no easy way to solve this problem and increasing the size of the /boot partition on my system would be the path to go.

Thus, I would like to come back to mrmazda’s suggestion to do this as

You should be able to shrink the sda2 lvm at its front, backup the content of sda1, delete and recreate sda1 at a larger size, then restore sda1 content, with plenty of room on /boot/ resulting.

Can you give me some additional advice on that? Maybe some of the following questions are somehow obvious to the expert, but I would like to learn about disk modification in this process.

From what I read, I would probably have to do this from a live CD or USB stick to have the partitions on the hard disk unmounted for such a process. Right?
Which tool would you recommend? I read about gparted, but could the Yast partitioner be used as well?
How would I shrink sda2 at its front (which probably means at the border between sda1 and sda2)? I seem to find information how to shrink at the end, but then I would have to move the contents of the file system somehow to the end of the partition. Is that the idea?
I am not sure, why sda1 needs to be deleted/recreated and consequently its content needs to be “restored” - is there no way to enlarge the file system in sda1 once there is enough free space at the right place?
Once there is “free space” at the front of sda2, this could be used to enlarge the size of sda1 by “recreating”. If I would want to take the “free space” from sda3 rather than from sda2, would I need to do this precedure twice, i.a. shrink sda3, “move” sda2 and then enlarge sda1?

edit:

the numbers in item 2 of the following list

are incorrect (misplacement of decimal point), they should be 14.17 and 14.16 MB. This does of course not alter the conclusion given in my last post.

Right, if you are going to change the partition mounted at / (root). You may unmount and work on the /boot partition though, but any mistake might leave you with an unbootable system.

Which tool would you recommend? I read about gparted, but could the Yast partitioner be used as well?

GParted Live is a good option. The YaST partitioner seems to be safer in a sense, but that likely means that it won’t allow you to do what you are planning to do here…

How would I shrink sda2 at its front (which probably means at the border between sda1 and sda2)? I seem to find information how to shrink at the end, but then I would have to move the contents of the file system somehow to the end of the partition. Is that the idea?I am not sure, why sda1 needs to be deleted/recreated and consequently its content needs to be “restored” - is there no way to enlarge the file system in sda1 once there is enough free space at the right place?

Don’t assume that sda1 is just in front of sda2 or that sda3 follows sda2 in disk order: that depends on the history of that disk since its first formatting. To say something about that we need to see the result of:

gdisk -l /dev/sda

If partitions are really in disk order, likely you will be able to shrink sda2 at the front with GParted, then grow sda1 and finally grow the filesystem on sda1 to fill all available space.
I’m not sure if lvm has additional constraints, anyway with GParted you can plan the sequence of changes before actually committing to disk, so anything illegal should pop up as error.

Once there is “free space” at the front of sda2, this could be used to enlarge the size of sda1 by “recreating”. If I would want to take the “free space” from sda3 rather than from sda2, would I need to do this precedure twice, i.a. shrink sda3, “move” sda2 and then enlarge sda1?

Yes, I think so, but again GParted allows you to plan the sequence and then commit to disk once you are satisfied with your plan.

OK, good to hear about the tools. “sda3” is in fact “sda4” on this disk.

Here is the output of gdisk:

> sudo gdisk -l /dev/sda
[sudo] password for root:
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: MBR only
  BSD: not present
  APM: not present
  GPT: not present


***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format
in memory.
***************************************************************

Disk /dev/sda: 312581808 sectors, 149.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 5F983376-61E2-4F57-A4BE-511BB0A72517
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 312581774
Partitions will be aligned on 1-sector boundaries
Total free space is 9007 sectors (4.4 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          321535   156.0 MiB   8300  Linux filesystem
   2          321536        46313471   21.9 GiB    8E00  Linux LVM
   4        46315395       312576704   127.0 GiB   8E00  Linux LVM

Looks to me that the partitions are in order sda1, sda2, sda4, with sda1 and sda2 directly adjacent, while there is a gap between sda2 and sda4 with respect to sectors. Also, at the end, some sectors seem to be not in use.

Same result with respect to sector counts from fdisk -l:

Disk /dev/sda: 149.1 GiB, 160041885696 bytes, 312581808 sectors
Disk model: WDC WD1600BEVT-2
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000c12a9

Device     Boot    Start       End   Sectors  Size Id Type
/dev/sda1  *        2048    321535    319488  156M 83 Linux
/dev/sda2         321536  46313471  45991936   22G 8e Linux LVM
/dev/sda4       46315395 312576704 266261310  127G 8e Linux LVM




Disk /dev/mapper/cr_ata-WDC_WD1600BEVT-22ZCT0_WD-WXE708N10973-part2: 22 GiB, 23545774080 bytes, 45987840 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/system-root: 19.9 GiB, 21395144704 bytes, 41787392 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/system-swap: 2 GiB, 2147483648 bytes, 4194304 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/cr_ata-WDC_WD1600BEVT-22ZCT0_WD-WXE708N10973-part4: 127 GiB, 136323693568 bytes, 266257214 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/system-home: 127 GiB, 136319074304 bytes, 266248192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

So, would that be a “go” for trying out GParted Live and look how far I can get? Or do you see any particular problems to expect?

I would try GParted Live and plan on:

-shrinking sda4 and possibly moving to the end of the disk;
-moving sda2 towards the end of the disk;
-growing sda1 as needed and enlarge its filesystem to fill all available space;

and see if GParted has something to complain about.
BE CAREFUL: I would not do the following step (actually committing to disk) unless you are absolutely sure of what you are doing and have a known good backup of all important data just in case.
Also please note that I have no experience with moving/resizing logical volumes so don’t take my word for it.

It has been a while. When I last wanted to resize an LVM partition:

(1) backup the data;
(2) delete partition, then recreate to the new size;
(3) create the LVM on that partition, and structure into volumes as needed
(4) restore data.

Actually, I only backed up and restored for “/home”. I find it easier to do a clean install than to backup and restore for the root volume. And, of course, there isn’t much point in backup/restore on a swap volume.

sorry, it took some time to finish up repartitioning the hdd in order to have more space for /boot (now 666 MB). Indeed, the upgrade process has now been started successfully.
I used GParted for this purpose, as suggested. For those interested, what had to be done, see the following topic in the GParted forum: http://gparted-forum.surf4.info/viewtopic.php?pid=35638#p35638
Thanks for your help in the present forum as well. I guess, this is resolved.

For those interested, here is the space used in /boot after upgrade to Leap 15.2 and doing all available updates (so there are now two versions of the 5.3.18 kernel):


>df -H
total 83445
-rw-r--r-- 1 root root      512  4. Aug 2017  backup_mbr
lrwxrwxrwx 1 root root        1  4. Aug 2017  boot -> .
-rw-r--r-- 1 root root     1725  8. Jun 21:25 boot.readme
-rw-r--r-- 1 root root   228172 12. Jun 14:11 config-5.3.18-lp152.19-default
-rw-r--r-- 1 root root   228310 22. Jul 08:44 config-5.3.18-lp152.33-default
drwxr-xr-x 2 root root     1024  4. Aug 2017  grub
drwxr-xr-x 6 root root     1024 31. Jul 13:56 grub2
lrwxrwxrwx 1 root root       30 31. Jul 13:30 initrd -> initrd-5.3.18-lp152.33-default
-rw------- 1 root root 14534728 31. Jul 13:47 initrd-5.3.18-lp152.19-default
-rw------- 1 root root 14537856 31. Jul 13:48 initrd-5.3.18-lp152.33-default
drwx------ 2 root root    12288 12. Feb 2015  lost+found
-rw-r--r-- 1 root root   432065 12. Jun 14:57 symvers-5.3.18-lp152.19-default.gz
-rw-r--r-- 1 root root   432172 22. Jul 08:54 symvers-5.3.18-lp152.33-default.gz
-rw-r--r-- 1 root root      484 12. Jun 14:57 sysctl.conf-5.3.18-lp152.19-default
-rw-r--r-- 1 root root      484 22. Jul 08:54 sysctl.conf-5.3.18-lp152.33-default
-rw-r--r-- 1 root root  4599521 12. Jun 14:53 System.map-5.3.18-lp152.19-default
-rw-r--r-- 1 root root  4601182 22. Jul 08:53 System.map-5.3.18-lp152.33-default
-rw-r--r-- 1 root root 13879463 12. Jun 15:01 vmlinux-5.3.18-lp152.19-default.gz
-rw-r--r-- 1 root root 13879527 22. Jul 08:56 vmlinux-5.3.18-lp152.33-default.gz
lrwxrwxrwx 1 root root       31 31. Jul 13:30 vmlinuz -> vmlinuz-5.3.18-lp152.33-default
-rw-r--r-- 1 root root  9031920 12. Jun 15:36 vmlinuz-5.3.18-lp152.19-default
-rw-r--r-- 1 root root  9036016 22. Jul 09:19 vmlinuz-5.3.18-lp152.33-default

The previous values on Leap 15.1 were reported in post

https://forums.opensuse.org/showthread.php/541873-upgrade-failure-from-Leap-15-1-with-disk-space-warning-on-boot?p=2948029#post2948029

.
Except for vmlinux-5.3.18-lp152.19-default.gz the size increase is rather moderate, and the files should have fit nicely into my previous /boot partition. So, it is not really clear why the Installer of Leap 15.2 had been complaining.

But anyhow, the whole thing was an interesting experience.