How to resolve "A start job is running for dev-disk-by\..." problems? Raspberry Pi / openSUSE

In two different projects (getting openSUSE to boot from a USB drive on Raspberry Pi, and getting WiFi to work on RPi/openSUSE), I’ve run into a situation where after making a modification, upon reboot I get hung with the message “A start job is running for dev-disk-by\ …”. The “…” part varies, but it’s currently “x2did-mmc…”. There is “no limit” (it’s looking for root! as well as swap), so the boot just hangs.

I believe the problem is caused by systemd waiting for a device to come up that systemd wants to identify by its id. I can’t find where to set the boot system to have systemd to wait for the device to come up by device name ("/dev/sda3" or “/dev/mmcblk0p2”). Changing /etc/fstab to use UUID= or LABEL= to identify the partition hasn’t fixed it.

The result of all this, after multiple tries, is to go back and reinstall the OS from scratch and try again. And again. And again. But I’ve run out of ideas of how to fix it.

In the latest iteration, the failed-boot occurred after I did a “mkinitrd -f”, following a procedure I had used earlier (on an earlier version of Tumbleweed) to install the RPi-3 WiFi (brcm) – successfully in that case. This suggests that the problem is a result of some change in openSUSE dracut/mkinitrd code, but I haven’t been able to figure out how to fix it. I can’t see that any of the mkinitrd or dracut parameters can be used to change the systemd device search.

The problem also occurs when I do a grub2-mkconfig to try to change the boot drive to a USB drive (having done a chroot to that drive). When I subsequently try to boot from that USB drive, it gives the same message.

I’m out of ideas where to look or what to do next. Can anyone suggest where to look next? I’ve tried working on Leap 42.2 and Tumbleweed 1/31 and 2/3 version (and earlier versions) and can’t seem to get anywhere.

Unlikely has anything to do with systemd.
You need to describe in detail what you mean by “after making a modification”

TSU

I followed the instructions here: Re: [opensuse-arm] Still no wlan on Raspberry 3 - openSUSE ARM - openSUSE Mailing Lists
that describe how to get WiFi working on the RPi-3 under openSUSE (WiFi doesn’t work out-of-the box with openSUSE releases, and it doesn’t work with SLES until “zypper dup”). Briefly, the process here is install the firmware, if needed; edit /etc/dracut.conf.d/raspberrypi_modules.conf to deselect a module for install and include others, and then “mkinitrd -f”. Process worked in an earlier Tumbleweed (2007.01.13, I think it was), but not in later releases (this failure was on the 2017.01.28 or .31 release, but I’ve tried the .02.03 release as well).

It’s clear that someone is working on development in this area since the bootcode.bin file (from the Raspberry folks, I think) was changed from the USB-oblivious version (about 17KB) to the USB-aware version (about 50KB) between those versions; someone’s trying to get openSUSE positioned to support the USB booting, I think, and others have reported success. But USB booting is still not working, and that development seems to be having some unintended consequences.

In my first reply, I should have said that I’d done a good deal of searching for answers, and in a number of other threads, this message (“a start job … dev-disk-by…”) was identified to come from an error in /etc/fstab entries that prevented systemd from mounting the devices it was looking for. Those responses may have been incorrect, but they did lead the posters to find that there had been an error in /etc/fstab entries (usually the UUID pair for /swap), and correcting that cured the problem. In my case, I’ve tried the “/dev/sda3”, “LABEL=”, and “UUID=” formats in /etc/fstab entries, and they don’t correct this problem. It’s the “dev-disk-by\x2did-mmc…” part that I can’t seem to correct anywhere, and I can’t get systemd (if that’s the culprit) to drop its insistence on referencing by id.

This suggests that the problem could be kernel parameters. So what do you have in grub.cfg?

Hi
Is this after you make the changes to boot from USB?

Have you see this ML thread?
https://lists.opensuse.org/opensuse-arm/2017-01/msg00060.html

Hi, Malcolm,
No, I’ve bagged that project for now (but yes, I’d seen and followed that thread, unsuccessfully – thanks for the pointer!).

This was a cloned microSD that was booting normally, but without WiFi.  Followed the instructions to get WiFi going (which I'd done successfully on an earlier release of Tumbleweed), and got to this wait loop, "A start job ... dev-...", when I put it back into the microSD slot and tried to boot from it.  

 That's the same message I'd been stuck on when I tried to create a USB-bootable drive, but I got there by a different route.  :-)

I think the relevant part is the section that references the menuentry parameters:

### BEGIN /etc/grub.d/10_linux ###
menuentry 'openSUSE'  --class opensuse --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-795b86ba-b1c5-455b-8233-6fe18ba25aa4' {
    load_video
    set gfxpayload=keep
    insmod gzio
    insmod part_msdos
    insmod ext2
    set root='hd0,msdos2'
    if  x$feature_platform_search_hint = xy ]; then
      search --no-floppy --fs-uuid --set=root --hint='hd0,msdos2'  795b86ba-b1c5-455b-8233-6fe18ba25aa4
    else
      search --no-floppy --fs-uuid --set=root 795b86ba-b1c5-455b-8233-6fe18ba25aa4
    fi
    echo    'Loading Linux 4.9.6-1-default ...'
    linux    /boot/Image-4.9.6-1-default root=UUID=795b86ba-b1c5-455b-8233-6fe18ba25aa4  root=/dev/disk/by-id/mmc-SL32G_0x28b56c90-part2 disk=/dev/disk/by-id/mmc-SL32G_0x28b56c90 resume=/dev/disk/by-id/mmc-SL32G_0x28b56c90-part3 quiet splash=silent plymouth.enable=0  swiotlb=512,force cma=128M console=ttyS0,115200n8 console=tty quiet 
    echo    'Loading initial ramdisk ...'
    initrd    /boot/initrd-4.9.6-1-default
}

The start job is waiting for


dev-disk-by\x2did-mmc\x2dSL32G_0x28b56c90\x2dpart2.device

after having given up on the part3.device (swap) after 90 seconds. There’s a no-limit wait for the part2.device (root).

Now, the working microSD card (running SLES in this case), has the relevant grub.cfg section as:

### BEGIN /etc/grub.d/10_linux ###
menuentry 'SLES-12-SP2-ARM-X11-raspberrypi3_aarch64'  --class sles_12_sp2_arm_x11_raspberrypi3_aarch64 --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-0b2c007e-1581-42e2-a05c-4a66eb4c115d' {
    load_video
    set gfxpayload=keep
    insmod gzio
    insmod part_msdos
    insmod ext2
    set root='hd0,msdos2'
    if  x$feature_platform_search_hint = xy ]; then
      search --no-floppy --fs-uuid --set=root --hint='hd0,msdos2'  c6c03b10-12d7-43cf-9f66-d04fa8e179c1
    else
      search --no-floppy --fs-uuid --set=root c6c03b10-12d7-43cf-9f66-d04fa8e179c1
    fi
    echo    'Loading Linux 4.4.38-93-default ...'
    linux    /Image-4.4.38-93-default root=UUID=0b2c007e-1581-42e2-a05c-4a66eb4c115d  root=/dev/disk/by-id/mmc-SDU32_0x0033566b-part3 disk=/dev/disk/by-id/mmc-SDU32_0x0033566b resume=/dev/disk/by-id/mmc-SDU32_0x0033566b-part4 loglevel=3 plymouth.enable=0  console=ttyS0,115200n8 console=tty quiet ${extra_cmdline} 
    echo    'Loading initial ramdisk ...'
    initrd    /initrd-4.4.38-93-default
}

and I see that systemctl reports (for the swap/resume partition):

  dev-disk-by\x2did-mmc\x2dSDU32_0x0033566b\x2dpart4.swap                                                                 loaded active active    /dev/disk/by-id/mmc-SDU32_0x0033566b-part4

For the running (SLES) system, /dev/disk/by-id gives me:

 ls /dev/disk/by-id
mmc-SDU32_0x0033566b        usb-Generic_USB2.0_Card_Reader_12345678901234567890-0:0
mmc-SDU32_0x0033566b-part1  usb-Generic_USB2.0_Card_Reader_12345678901234567890-0:0-part1
mmc-SDU32_0x0033566b-part2  usb-Generic_USB2.0_Card_Reader_12345678901234567890-0:0-part2
mmc-SDU32_0x0033566b-part3  usb-Generic_USB2.0_Card_Reader_12345678901234567890-0:0-part3
mmc-SDU32_0x0033566b-part4

So, I think I need to edit my openSUSE grub.cfg file to use the “ID_SERIAL” (the 0x0033566b part of the id on the SLES microSD) of the microSD I’m using for openSUSE. But a) I can’t see how to get that (since the microSD is being accessed through the USB adapter – it won’t boot on its own – and the USB adapter reports ID_SERIAL=Generic_USB2.0_Card_Reader_12345678901234567890-0:0) and b) I’d have thought that grub2-mkconfig would have done that for me – I don’t know where its references to “mmc-SL32G_0x28b56c90” are coming from, though it’s possible they really match something in that openSUSE microSD card that I can’t see.

On Mon 13 Feb 2017 01:06:01 PM CST, hdtodd wrote:

arvidjaar;2812106 Wrote:
> This suggests that the problem could be kernel parameters. So what do
> you have in grub.cfg?

I think the relevant part is the section that references the menuentry
parameters:
<snip>

Hi
OK, I duplicated the issue, so when the mkinitrd -f is run something is
getting confused with the UUID’s


Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
openSUSE Leap 42.1|GNOME 3.16.2|4.1.36-44-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

Yes, though I couldn’t be sure if it was UUIDs or ID_SERIAL that’s getting confused. What’s most annoying is that there seems to be no way to say “don’t use UUID or disk ID, use device name” (/dev/sda1, etc.). And the process seems to insist on using /proc/cmdline as the authoritative source of kernel info rather than letting me override – seems to be no option for overrides.

Hi
Yup, edit… /boot/grub2/device.map

Mine says:

(hd0) /dev/disk/by-id/mmc-SL32G_0x28b56c90 

I tried changing it to “(hd0) /dev/sda”, and after grub2-mkconfig I got

linux    /boot/Image-4.9.6-1-default root=UUID=795b86ba-b1c5-455b-8233-6fe18ba25aa4  root=/dev/disk/by-id/mmc-SL32G_0x28b56c90-part2 disk=/dev/disk/by-id/mmc-SL32G_0x28b56c90 resume=/dev/disk/by-id/mmc-SL32G_0x28b56c90-part3 quiet splash=silent plymouth.enable=0  swiotlb=512,force cma=128M console=ttyS0,115200n8 console=tty quiet 

as the “linux” line for loading that openSUSE image. That is, same /dev/disk/by-id issue as before.

Now, to get grub2-mkconfig to work, I had to give it a /proc. I used the standard

cd /mnt/tgt #mount point for my target openSUSE microSD card
mount --bind /dev dev
mount --bind /sys sys
mount --bind /proc proc
chroot /mnt/tgt
grub2-mkconfig -o /boot/grub2/grub.cfg

I think grub2-mkconfig is picking up the running (SLES) /proc parameters and using them to create the new grub.cfg. Any way to avoid that? If I don’t feed it a viable /proc, it dies complaining that it can’t find /proc/self/mountinfo, but I don’t want it to be using the running-system mount info.

I’ve tried a number of changes, in succession:

# cat device.map
(hd0) /dev/sda

# cat device.map
(hd0) /dev/mmcblk0
# cat bootpart.cfg
search --set=root=LABEL=ROOT
set prefix=($root)/boot/grub2

None of these made a difference … still stuck in the boot-up “A start job” loop.

Used fdisk to change the disk id to that listed in the openSUSE menuentry in grub2. Same hang.

Manually edit grub.cfg to remove UUID references and replace dev\disk\by-id references with LABEL= references, while at the same time changing fstab to refer to devices by label. The result was that it’s now hanging with a “by-label” hang on SWAP and ROOT. But it still is looking for that long name with the “dev-disk-by …” – I can’t get it (systemd, I presume) to be looking for /dev/sda3 or LABEL=ROOT, it’s always prefaced by that long string.

If this can be resolved with something easy because I’ve been overlooking a simple switch, please tell me. But otherwise please don’t waste time trying to figure it out. Since the WiFi setup procedure worked at one time and no longer does, and since WiFi doesn’t work upon initial install, I presume it’ll be fixed eventually. While openSUSE has a richer set of features that I prefer, I’ll make do with SLES for now and hope that the issues get resolved over time.

Thanks for trying to help.
David

These root= and resume= lines are added by you or YaST manually. grub2-mkconfig itself only puts UUID there. So start with removing these kernel parameters from /etc/default/grub and rerun grub2-mkconfig. You can post output of blkid here to verify that UUID is indeed correct.

device.map is absolutely irrelevant for grub2. The only case when grub2 is referencing it is when you use grub device notation with grub-install command line (like grub-install (hd0)). And even then it only determines device where grub-install writes its bootblock - not the content of grub.cfg (grub-mkconfig does not even read device.map)

The root= and resume= phrases (it’s all one line) were added by grub2-mkconfig. I didn’t that’s a component of YaST, and I don’t invoke YaST, directly at least or indirectly as far as I know, at any point. And I didn’t add them myself.

I had long ago edited grub.cfg to remove those references, or to change them to /dev/sda3 and /dev/sda2, respectively, with no change in end result. I tried again yesterday, and again: same boot loop.

I know you’re convinced it’s a problem in grub. There may be a problem there (I see a bunch of things I don’t like), but I think that’s not causing my boot-loop problem: I still think it’s caused by something set for use by systemd, perhaps during boot. For example, if you go to /usr/lib/systemd/system and “grep -r dev- *”, you’ll see, among others, the following entries:

lvm2-pvscan@.service:BindsTo=dev-block-%i.device
serial-getty@.service:BindsTo=dev-%i.device
serial-getty@.service:After=dev-%i.device systemd-user-sessions.service plymouth-quit-wait.service

which have the same form as the boot wait-loop I’m hitting. I’ve seen the “dev-block-” prefix during some iteration of trying various configuration edits, but generally it’s a “dev-disk-by.…” wait. Still, I think this problem is most immediately related to systemd. It might, indeed, be caused by a problem in grub.cfg itself (though I’ve edited those parameter lines enough times to be pretty convinced it’s not). But I think it’s more likely related to something else generated for systemd, either during the mkconfig process or during the kernel startup.
[/QUOTE]

device.map is absolutely irrelevant for grub2. The only case when grub2 is referencing it is when you use grub device notation with grub-install command line (like grub-install (hd0)). And even then it only determines device where grub-install writes its bootblock - not the content of grub.cfg (grub-mkconfig does not even read device.map)

I can confirm that changing device.map to reference /dev/sda3 and LABEL=ROOT did not resolve my boot-loop problem.

These strings come from configuration file that is edited either by user or by YaST. grub-mkconfig is just a messenger here.

I had long ago edited grub.cfg to remove those references, or to change them to /dev/sda3 and /dev/sda2, respectively, with no change in end result.

So show what happens when you remove these strings completely.

I know you’re convinced it’s a problem in grub.

Where did I say that? The problem has nothing to do with grub; but it is necessary to find where reference to these non-existent devices come from.

Oh, and BTW it would be rather helpful to show what devices you actually have (ls -lR /dev/disk/by-*).

If I remove those strings completely (editing just the grub.cfg file), it boots to hang in the same place (“A start job … dev-disk-by\x2did…”).

Where did I say that? The problem has nothing to do with grub; but it is necessary to find where reference to these non-existent devices come from.

Sorry, I misunderstood your focus. But so far, any changes I’ve made to grub.cfg, including the changes you suggested here, have had no impact upon the boot-loop hang at “A start job … dev-disk-by\x2did…”. See below.

Oh, and BTW it would be rather helpful to show what devices you actually have (ls -lR /dev/disk/by-*).

Booting from the microSD to run SLES 12.2, with openSUSE 4.9.6 kernel on a microSD card mounted in a USB adapter (that is, mmcblk0 has SLES, sda has openSUSE):


Pi-6w:~ # ls -lR /dev/disk/by-*
/dev/disk/by-id:
total 0
lrwxrwxrwx 1 root root 13 Dec 31  1969 mmc-SDU32_0x0033566b -> ../../mmcblk0
lrwxrwxrwx 1 root root 15 Dec 31  1969 mmc-SDU32_0x0033566b-part1 -> ../../mmcblk0p1
lrwxrwxrwx 1 root root 15 Dec 31  1969 mmc-SDU32_0x0033566b-part2 -> ../../mmcblk0p2
lrwxrwxrwx 1 root root 15 Dec 31  1969 mmc-SDU32_0x0033566b-part3 -> ../../mmcblk0p3
lrwxrwxrwx 1 root root 15 Dec 31  1969 mmc-SDU32_0x0033566b-part4 -> ../../mmcblk0p4
lrwxrwxrwx 1 root root  9 Dec 31  1969 usb-Generic_USB2.0_Card_Reader_12345678901234567890-0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 Dec 31  1969 usb-Generic_USB2.0_Card_Reader_12345678901234567890-0:0-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Dec 31  1969 usb-Generic_USB2.0_Card_Reader_12345678901234567890-0:0-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 Dec 31  1969 usb-Generic_USB2.0_Card_Reader_12345678901234567890-0:0-part3 -> ../../sda3

/dev/disk/by-label:
total 0
lrwxrwxrwx 1 root root 15 Dec 31  1969 BOOT -> ../../mmcblk0p2
lrwxrwxrwx 1 root root 10 Dec 31  1969 EFI -> ../../sda1
lrwxrwxrwx 1 root root 10 Dec 31  1969 ROOT -> ../../sda2
lrwxrwxrwx 1 root root 10 Dec 31  1969 SWAP -> ../../sda3

/dev/disk/by-path:
total 0
lrwxrwxrwx 1 root root 13 Dec 31  1969 platform-3f202000.sdhost -> ../../mmcblk0
lrwxrwxrwx 1 root root 15 Dec 31  1969 platform-3f202000.sdhost-part1 -> ../../mmcblk0p1
lrwxrwxrwx 1 root root 15 Dec 31  1969 platform-3f202000.sdhost-part2 -> ../../mmcblk0p2
lrwxrwxrwx 1 root root 15 Dec 31  1969 platform-3f202000.sdhost-part3 -> ../../mmcblk0p3
lrwxrwxrwx 1 root root 15 Dec 31  1969 platform-3f202000.sdhost-part4 -> ../../mmcblk0p4
lrwxrwxrwx 1 root root  9 Dec 31  1969 platform-3f980000.usb-usb-0:1.2:1.0-scsi-0:0:0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 Dec 31  1969 platform-3f980000.usb-usb-0:1.2:1.0-scsi-0:0:0:0-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Dec 31  1969 platform-3f980000.usb-usb-0:1.2:1.0-scsi-0:0:0:0-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 Dec 31  1969 platform-3f980000.usb-usb-0:1.2:1.0-scsi-0:0:0:0-part3 -> ../../sda3

/dev/disk/by-uuid:
total 0
lrwxrwxrwx 1 root root 15 Dec 31  1969 0b2c007e-1581-42e2-a05c-4a66eb4c115d -> ../../mmcblk0p3
lrwxrwxrwx 1 root root 15 Dec 31  1969 4479-5615 -> ../../mmcblk0p1
lrwxrwxrwx 1 root root 15 Dec 31  1969 59ad7379-fc8b-4e66-a451-9f43bf7ebd5b -> ../../mmcblk0p4
lrwxrwxrwx 1 root root 10 Dec 31  1969 5F76-974D -> ../../sda1
lrwxrwxrwx 1 root root 10 Dec 31  1969 795b86ba-b1c5-455b-8233-6fe18ba25aa4 -> ../../sda2
lrwxrwxrwx 1 root root 15 Dec 31  1969 c6c03b10-12d7-43cf-9f66-d04fa8e179c1 -> ../../mmcblk0p2
lrwxrwxrwx 1 root root 10 Dec 31  1969 fa126ee2-3aa3-464b-8bec-56e2dc077ba2 -> ../../sda3

I thought it might be helpful to strip away some of the parameters that seem to be complicating the analysis, recreate grub.cfg again, and try another boot. Here are the files I think are used in that configuration-generation process, stripped to the minimum as best I can tell:

Pi-6w:/mnt/tgt # cat etc/default/grub
GRUB_DISTRIBUTOR=openSUSE
GRUB_DEFAULT=0
GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10
GRUB_CMDLINE_LINUX=" root=/dev/mmcblk0p2 splash=silent plymouth.enable=0  swiotlb=512,force cma=128M console=ttyS0,115200n8 console=tty quiet"
GRUB_TERMINAL=gfxterm
GRUB_GFXMODE=800x600
GRUB_GFXPAYLOAD_LINUX=keep
GRUB_THEME="/boot/grub2/themes/openSUSE/theme.txt"
GRUB_BACKGROUND="/boot/grub2/themes/openSUSE/background.png"
Pi-6w:/mnt/tgt # cat etc/default/grub_installdevice
/dev/mmcblk0
Pi-6w:/mnt/tgt # cat boot/grub2/device.map
(hd0) /dev/mmcblk0

I’ll note that I also edited boot/grub2/bootpart.cfg, to point to mmcblk0p2, but it got changed by grub2-mkconfig to point to sda2 again (which is where this was mounted at the time grub2-mkconfig was run).

I did the usual

mount /dev/sda2 /mnt/tgt
cd /mnt/tgt
mount --bind /dev dev
mount --bind /sys sys
mount --bind /proc proc
chroot /mnt/tgt
<edited files to create the ones listed above>
grub2-mkconfig -o /boot/grub2/grub.cfg
<edit that grub.cfg to remove all the "other OS" references in section 30, since they reference SLES>
exit
poweroff
<move the openSUSE microSD card to the slot, reboot>

and the result this time is

A start job is running for dev-mmcblk0p2.device (... / no limit)

Again, something is sticking that “dev-” prefix on and not looking for /dev/mmcblk0p2. Looking in /usr/lib/systemd/system with grep -r dev- *", the only text file that references “dev-%i.device” is serial-getty (though a number of binary files include “dev-”); moving that serial-getty file to /root (to get it out of the way) and removing tty references from the GRUB_CMDLINE_LINUX string and rebooting, I ended up with the same wait-loop message.

Well, previously you stated that system waited for

dev-disk-by\x2did-mmc\x2dSL32G_0x28b56c90\x2dpart2.device

which does not exist according to your last /dev/disk listing. This device was later shown by you to be explicitly listed on kernel command line both as root and as resume device. Now you state that system waits for

A start job is running for dev-mmcblk0p2.device (... / no limit)

which obviously is very different from what you observed before (or at least from what you told us before). So your statement about “no impact” is wrong - change to kernel command line did have very visible impact.

Now, if your system waits for something as simple as /dev/mmcblk0, I suspect that your initrd does not include necessary drivers. Which also may explain your previous problems because initrd also includes “stored command line” which it may attempt to use when you do not supply kernel parameters. So try to chroot and run mkinitrd to recreate it for current root device. It would be helpful if you could capture output (e.g. by running it under “script” command) and show here. Try to reboot without any extra kernel parameters. What happens?

Yep, you’re right. Didn’t get rid of the “dev-” part, but did change the device name.

Now, if your system waits for something as simple as /dev/mmcblk0, I suspect that your initrd does not include necessary drivers. Which also may explain your previous problems because initrd also includes “stored command line” which it may attempt to use when you do not supply kernel parameters. So try to chroot and run mkinitrd to recreate it for current root device. It would be helpful if you could capture output (e.g. by running it under “script” command) and show here. Try to reboot without any extra kernel parameters. What happens?

It hangs with the same “A start job … dev-mmcblk0p2.device” wait loop.

Here’s the dracut modules file, modified per the instructions on getting the WAN service working:

Pi-6w:/mnt # cat tgt/etc/dracut.conf.d/raspberrypi_modules.conf
add_drivers+=" bcm2835-sdhost bcm2835_dma mmc_block dwc2 "
# Workaround for Wifi
omit_drivers+=" sdhci-iproc"

Here’s the log of mounting the sda partitions and doing the mkinitrd. More interesting info after this rather lengthy log. I think you’ve found the problem, but I don’t know how to fix it.

Pi-6w:~ # lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    1  29.7G  0 disk 
├─sda1        8:1    1   200M  0 part 
├─sda2        8:2    1    29G  0 part 
└─sda3        8:3    1 493.7M  0 part 
mmcblk0     179:0    0  30.2G  0 disk 
├─mmcblk0p1 179:1    0   200M  0 part /boot/efi
├─mmcblk0p2 179:2    0   266M  0 part /boot
├─mmcblk0p3 179:3    0  29.2G  0 part /
└─mmcblk0p4 179:4    0 493.6M  0 part [SWAP]
Pi-6w:~ # mount /dev/sda2 /mnt/tgt
Pi-6w:~ # mount /dev/sda1 /mnt/tgt/boot/efi
Pi-6w:~ # cd /mnt/tgt
Pi-6w:/mnt/tgt # mount --bind /dev dev
Pi-6w:/mnt/tgt # mount --bind /sys sys
Pi-6w:/mnt/tgt # mount --bind /proc proc
Pi-6w:/mnt/tgt # chroot /mnt/tgt
Pi-6w:/ # mkinitrd --verbose
Creating initrd: /boot/initrd-4.9.6-1-default
dracut: Executing: /usr/bin/dracut -v --logfile /var/log/YaST2/mkinitrd.log --force /boot/initrd-4.9.6-1-default 4.9.6-1-default
modinfo: ERROR: Module usb_common not found.
modinfo: ERROR: Module virt_dma not found.
modinfo: ERROR: Module bcm2835_sdhost not found.
modinfo: ERROR: Module bcm2835_thermal not found.
modinfo: ERROR: Module bcm2835_cpufreq not found.
dracut: dracut module 'systemd-bootchart' will not be installed, because command '/usr/lib/systemd/systemd-bootchart' could not be found!
dracut: *** Including module: bash ***
dracut: *** Including module: systemd ***
dracut: *** Including module: warpclock ***
dracut: *** Including module: systemd-initrd ***
dracut: *** Including module: i18n ***
dracut: *** Including module: drm ***
dracut: Possible missing firmware "a420_pfp.fw" for kernel module "msm.ko"
dracut: Possible missing firmware "a420_pm4.fw" for kernel module "msm.ko"
dracut: Possible missing firmware "a330_pfp.fw" for kernel module "msm.ko"
dracut: Possible missing firmware "a330_pm4.fw" for kernel module "msm.ko"
dracut: Possible missing firmware "a300_pfp.fw" for kernel module "msm.ko"
dracut: Possible missing firmware "a300_pm4.fw" for kernel module "msm.ko"
dracut: *** Including module: plymouth ***
dracut: *** Including module: kernel-modules ***
dracut: Possible missing firmware "aic94xx-seq.fw" for kernel module "aic94xx.ko"
dracut: Possible missing firmware "wd719x-risc.bin" for kernel module "wd719x.ko"
dracut: Possible missing firmware "wd719x-wcs.bin" for kernel module "wd719x.ko"
dracut: Possible missing firmware "sd8688.bin" for kernel module "libertas_sdio.ko"
dracut: Possible missing firmware "sd8688_helper.bin" for kernel module "libertas_sdio.ko"
dracut: Possible missing firmware "sd8686.bin" for kernel module "libertas_sdio.ko"
dracut: Possible missing firmware "sd8686_helper.bin" for kernel module "libertas_sdio.ko"
dracut: Possible missing firmware "sd8385.bin" for kernel module "libertas_sdio.ko"
dracut: Possible missing firmware "sd8385_helper.bin" for kernel module "libertas_sdio.ko"
dracut: Omitting driver sdhci_iproc
dracut: *** Including module: rootfs-block ***
dracut: *** Including module: terminfo ***
dracut: *** Including module: udev-rules ***
dracut: Skipping udev rule: 40-redhat.rules
dracut: Skipping udev rule: 50-firmware.rules
dracut: Skipping udev rule: 50-udev.rules
dracut: Skipping udev rule: 91-permissions.rules
dracut: Skipping udev rule: 80-drivers-modprobe.rules
dracut: *** Including module: dracut-systemd ***
dracut: *** Including module: haveged ***
dracut: *** Including module: usrmount ***
dracut: *** Including module: base ***
dracut: *** Including module: fs-lib ***
dracut: *** Including module: shutdown ***
dracut: *** Including module: suse ***
dracut: *** Including modules done ***
dracut: *** Installing kernel module dependencies and firmware ***
dracut: *** Installing kernel module dependencies and firmware done ***
dracut: *** Resolving executable dependencies ***
dracut: *** Resolving executable dependencies done***
dracut: *** Hardlinking files ***
dracut: *** Hardlinking files done ***
dracut: *** Stripping files ***
dracut: *** Stripping files done ***
dracut: *** Store current command line parameters ***
dracut: Stored kernel commandline:
dracut:  root=UUID=795b86ba-b1c5-455b-8233-6fe18ba25aa4 rootfstype=ext4 rootflags=rw,relatime,data=ordered
dracut: *** Creating image file '/boot/initrd-4.9.6-1-default' ***
dracut: Image: /var/tmp/dracut.ISSKT3/initramfs.img: 20M
dracut: ========================================================================
dracut: Version: dracut-044-20.1
dracut: 
dracut: Arguments: -v --logfile --force
dracut: 
dracut: dracut modules:
dracut: bash
dracut: systemd
dracut: warpclock
dracut: systemd-initrd
dracut: i18n
dracut: drm
dracut: plymouth
dracut: kernel-modules
dracut: rootfs-block
dracut: terminfo
dracut: udev-rules
dracut: dracut-systemd
dracut: haveged
dracut: usrmount
dracut: base
dracut: fs-lib
dracut: shutdown
dracut: suse
dracut: ========================================================================
dracut: drwxr-xr-x  16 root     root            0 Feb 15 14:33 .
dracut: crw-r--r--   1 root     root       5,   1 Feb 15 14:32 dev/console
<deleted to reduce posting size>
dracut: drwxr-xr-x   5 root     root            0 Feb 15 14:33 lib/modules/4.9.6-1-default/kernel/drivers/mmc
dracut: drwxr-xr-x   2 root     root            0 Feb 15 14:33 lib/modules/4.9.6-1-default/kernel/drivers/mmc/card
dracut: -rw-r--r--   1 root     root        68994 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/card/mmc_block.ko
dracut: drwxr-xr-x   2 root     root            0 Feb 15 14:33 lib/modules/4.9.6-1-default/kernel/drivers/mmc/core
dracut: -rw-r--r--   1 root     root       311074 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/core/mmc_core.ko
dracut: drwxr-xr-x   2 root     root            0 Feb 15 14:33 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host
dracut: -rw-r--r--   1 root     root        30450 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/cb710-mmc.ko
dracut: -rw-r--r--   1 root     root        69626 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/dw_mmc.ko
dracut: -rw-r--r--   1 root     root        42674 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/mmci.ko
dracut: -rw-r--r--   1 root     root        47898 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/mtk-sd.ko
dracut: -rw-r--r--   1 root     root        44266 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/rtsx_pci_sdmmc.ko
dracut: -rw-r--r--   1 root     root        49954 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/rtsx_usb_sdmmc.ko
dracut: -rw-r--r--   1 root     root        12426 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sdhci-brcmstb.ko
dracut: -rw-r--r--   1 root     root        14578 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sdhci_f_sdh30.ko
dracut: -rw-r--r--   1 root     root        89034 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sdhci.ko
dracut: -rw-r--r--   1 root     root        24210 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sdhci-msm.ko
dracut: -rw-r--r--   1 root     root        23386 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sdhci-of-arasan.ko
dracut: -rw-r--r--   1 root     root        22962 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sdhci-of-esdhc.ko
dracut: -rw-r--r--   1 root     root        53346 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sdhci-pci.ko
dracut: -rw-r--r--   1 root     root        16698 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sdhci-pltfm.ko
dracut: -rw-r--r--   1 root     root        18930 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sdhci-tegra.ko
dracut: -rw-r--r--   1 root     root        36202 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/sunxi-mmc.ko
dracut: -rw-r--r--   1 root     root        31346 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/tifm_sd.ko
dracut: -rw-r--r--   1 root     root        26946 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/toshsd.ko
dracut: -rw-r--r--   1 root     root        52666 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/usdhi6rol0.ko
dracut: -rw-r--r--   1 root     root        15618 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/ushc.ko
dracut: -rw-r--r--   1 root     root        33754 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/via-sdmmc.ko
dracut: -rw-r--r--   1 root     root        54162 Jan 29 07:57 lib/modules/4.9.6-1-default/kernel/drivers/mmc/host/vub300.ko
...
dracut: drwxr-xr-x   2 root     root            0 Feb 15 14:32 var/tmp
dracut: ========================================================================
dracut: *** Creating initramfs image file '/boot/initrd-4.9.6-1-default' done ***
Pi-6w:/ # lsblk -o name,uuid
NAME        UUID
sda         
├─sda1      5F76-974D
├─sda2      795b86ba-b1c5-455b-8233-6fe18ba25aa4
└─sda3      fa126ee2-3aa3-464b-8bec-56e2dc077ba2
mmcblk0     
├─mmcblk0p1 4479-5615
├─mmcblk0p2 c6c03b10-12d7-43cf-9f66-d04fa8e179c1
├─mmcblk0p3 0b2c007e-1581-42e2-a05c-4a66eb4c115d
└─mmcblk0p4 59ad7379-fc8b-4e66-a451-9f43bf7ebd5b
Pi-6w:/ # 

So after checking the missing modules logged above, I did this on the openSUSE install:

Pi-6w:/mnt # find . -name bcm2835* -print
./tgt/lib/modules/4.9.6-1-default/kernel/drivers/dma/bcm2835-dma.ko
./tgt/lib/modules/4.9.6-1-default/kernel/drivers/watchdog/bcm2835_wdt.ko
./tgt/lib/modules/4.9.6-1-default/kernel/drivers/char/hw_random/bcm2835-rng.ko

… and this on the SLES install that was running (and on which the openSUSE microSD was mounted as sda2):

Pi-6w:/mnt # find /lib/modules -name bcm2835* -print
/lib/modules/4.4.21-69-default/kernel/drivers/char/hw_random/bcm2835-rng.ko
/lib/modules/4.4.21-69-default/kernel/drivers/cpufreq/bcm2835-cpufreq.ko
/lib/modules/4.4.21-69-default/kernel/drivers/dma/bcm2835-dma.ko
/lib/modules/4.4.21-69-default/kernel/drivers/mmc/host/bcm2835-sdhost.ko
/lib/modules/4.4.21-69-default/kernel/drivers/thermal/bcm2835-thermal.ko
/lib/modules/4.4.21-69-default/kernel/drivers/watchdog/bcm2835_wdt.ko
/lib/modules/4.4.38-93-default/kernel/drivers/char/hw_random/bcm2835-rng.ko
/lib/modules/4.4.38-93-default/kernel/drivers/cpufreq/bcm2835-cpufreq.ko
/lib/modules/4.4.38-93-default/kernel/drivers/dma/bcm2835-dma.ko
/lib/modules/4.4.38-93-default/kernel/drivers/mmc/host/bcm2835-sdhost.ko
/lib/modules/4.4.38-93-default/kernel/drivers/thermal/bcm2835-thermal.ko
/lib/modules/4.4.38-93-default/kernel/drivers/watchdog/bcm2835_wdt.ko

So if I’m reading this right, there are some critical modules missing when I go to do the mkinitrd on openSUSE .

BUT … that was a running openSUSE before I followed the procedure to get the WAN running, Re: [opensuse-arm] Still no wlan on Raspberry 3 - openSUSE ARM - openSUSE Mailing Lists, which I had followed before, successfully, with an earlier distribution of openSUSE (2017.01.03, I think it was).

I can’t imagine that running mkinitrd would have deleted just those modules from /lib/modules/…, and they must have been there if they’re needed, because it booted successfully.

So … am I misinterpreting? I’m inclined to wipe this SD with a fresh install of openSUSE, verify that it boots successfully, run through the WAN install process (link above), and see if I get the same result.

But before I blow away this install: am I interpreting this correctly, and is there another way to fix this install? (I don’t have anything on this that I need to preserve.)

I’m not really familiar with this hardware, but quick look suggests that BCM2835 is supported by upstream kernel module sdhci-iproc (which you explicitly blacklist) since 4.6. I do not see any bcm2835-sdhost in current upstream Linux sources (of course it may be added as out of tree module by SUSE).

config MMC_SDHCI_IPROC
        tristate "SDHCI support for the BCM2835 & iProc SD/MMC Controller"