dup to 20210708 results in unwanted device mapper

karlmistelberger · July 10, 2021, 6:11am

I never asked for device mapper and it never got invoked on earlier boots:

**erlangen:~ #** journalctl -g mapper -o short-monotonic -b -2 
-- Journal begins at Fri 2021-07-09 12:59:35 CEST, ends at Sat 2021-07-10 05:56:52 CEST. -- 
    0.862845] erlangen kernel: device-mapper: uevent: version 1.0.3 
    0.862881] erlangen kernel: device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com 
    3.514058] erlangen systemd[1]: Listening on Device-mapper event daemon FIFOs. 
**erlangen:~ #**

Behavior changed when upgrading to 20210708:

**erlangen:~ #** journalctl -g mapper -o short-monotonic -b 
-- Journal begins at Fri 2021-07-09 12:59:35 CEST, ends at Sat 2021-07-10 06:00:22 CEST. -- 
    0.875164] localhost kernel: device-mapper: uevent: version 1.0.3 
    0.875204] localhost kernel: device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com 
    3.041850] localhost systemd[1]: Condition check resulted in Device-Mapper Multipath Default Configuration being skipped. 
    3.042404] localhost systemd[1]: Starting Device-Mapper Multipath Device Controller... 
    3.046197] localhost systemd[1]: Started Device-Mapper Multipath Device Controller. 
    3.072589] localhost kernel: device-mapper: multipath service-time: version 0.3.0 loaded 
    3.666437] localhost systemd[1]: Stopping Device-Mapper Multipath Device Controller... 
    3.683309] localhost systemd[1]: Stopped Device-Mapper Multipath Device Controller. 
    4.299749] erlangen systemd[1]: Listening on Device-mapper event daemon FIFOs. 
**erlangen:~ #**

Rolling back to pre dup snapshot fixes the issue, but that is not what I want. Any idea how to undo change of device mapper behavior only?

marel · July 10, 2021, 9:59am

Is there really something different apart from more (debug) output?

Things are started on then stopped again and at the end it seems like the the device mapper is listening in both cases.

I must say I do currently not have an idea what the device-mapper does but that is for more things in the boot process.

marel · July 10, 2021, 10:29am

I just did update Tumbleweed from 20210701 to 20210708 and do have the same 3 lines of output you got in earlier boots

Did read a bit about the device mapper:

$ sudo dmsetup status
No devices found

karlmistelberger · July 11, 2021, 6:37am

M Fredericks:

I just did update Tumbleweed from 20210701 to 20210708 and do have the same 3 lines of output you got in earlier boots

Did read a bit about the device mapper:
$ sudo dmsetup status
No devices found

I had some 20 devices in /dev/mapper. As a last resort I rolled back to Tumbleweed 20210704. With /dev/mapper being empty I duped again and arrived at 20210708 with the folder still being empty:

**erlangen:~ #** ll /dev/mapper/                                
total 0 
crw------- 1 root root 10, 236 Jul 11 05:56 **control**
**erlangen:~ #** dmsetup status 
No devices found 
**erlangen:~ #**

**erlangen:~ #** journalctl -o short-monotonic -g device-mapper -b 
-- Journal begins at Fri 2021-07-09 12:59:35 CEST, ends at Sun 2021-07-11 06:30:55 CEST. -- 
    0.849178] erlangen kernel: device-mapper: uevent: version 1.0.3 
    0.849215] erlangen kernel: device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com 
    3.520091] erlangen systemd[1]: Listening on Device-mapper event daemon FIFOs. 
**erlangen:~ #**

marel · July 11, 2021, 11:54am

Interesting, any known devices or just garbage?

I duped again and arrived at 20210708 with the folder still being empty

So problem solved I guess

karlmistelberger · July 11, 2021, 7:46pm

The existing devices where eclipsed by device mapper:

**erlangen:~ #** journalctl -b -7 -o short-monotonic --dmesg|grep dm- 
    0.889306] localhost kernel: device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: **dm-**devel@redhat.com 
    3.158065] localhost kernel: BTRFS: device fsid 9e9fb019-007e-497d-9ff0-bda6eb2b131e devid 1 transid 490 /dev/**dm-**12 scanned by systemd-udevd (557) 
    3.164310] localhost kernel: BTRFS: device label Leap-15.3 devid 1 transid 1139 /dev/**dm-**11 scanned by systemd-udevd (517) 
    3.200015] localhost kernel: BTRFS: device label tumbleweed-nvme0n1p3 devid 1 transid 168921 /dev/**dm-**20 scanned by systemd-udevd (519) 
    3.398440] localhost kernel: BTRFS info (device **dm-**20): disk space caching is enabled 
    3.398443] localhost kernel: BTRFS info (device **dm-**20): has skinny extents 
    3.405710] localhost kernel: BTRFS info (device **dm-**20): enabling ssd optimizations 
    4.064960] erlangen kernel: BTRFS info (device **dm-**20): disk space caching is enabled 
    4.102587] erlangen kernel: EXT4-fs (**dm-**21): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none. 
    4.112273] erlangen kernel: FAT-fs (**dm-**18): Volume was not properly unmounted. Some data may be corrupt. Please run fsck. 
    4.114222] erlangen kernel: EXT4-fs (**dm-**2): mounted filesystem with ordered data mode. Opts: data=ordered. Quota mode: none. 
    4.637599] erlangen kernel: BTRFS info (device **dm-**20): devid 1 device path /dev/mapper/eui.0025385661b057b5-part3 changed to /dev/**dm-**20 scanned by systemd-udevd (996) 
    4.647457] erlangen kernel: BTRFS info (device **dm-**20): devid 1 device path /dev/**dm-**20 changed to /dev/mapper/eui.0025385661b057b5-part3 scanned by systemd-udevd (996) 
    4.773531] erlangen kernel: BTRFS info (device **dm-**20): devid 1 device path /dev/mapper/eui.0025385661b057b5-part3 changed to /dev/**dm-**20 scanned by systemd-udevd (1022) 
    4.787066] erlangen kernel: BTRFS info (device **dm-**20): devid 1 device path /dev/**dm-**20 changed to /dev/mapper/eui.0025385661b057b5-part3 scanned by systemd-udevd (1022) 
   74.615008] erlangen kernel: BTRFS warning (device **dm-**20): qgroup rescan init failed, qgroup is not enabled 
**erlangen:~ #**

So problem solved I guess
Yes. There were some nasty side effects, such as stale entries in the window list of the KDE panel, search in the application starter failed to work correctly and some more. However the system is back to normal. Upgrade to snapshot 20210709 worked correctly. dm-devices didn’t show up again so far.

karlmistelberger · October 14, 2021, 3:23pm

The same happened with another machine:

**i3-4130:~ #** journalctl -b -4 -g mapper --no-full  
-- Journal begins at Thu 2021-09-30 18:53:22 CEST, ends at Thu 2021-10-14 15:07:09 CEST. -- 
Oct 14 13:44:17 i3-4130 kernel: device-mapper: uevent: version 1.0.3 
Oct 14 13:44:17 i3-4130 kernel: device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com 
Oct 14 13:44:18 i3-4130 dracut-cmdline[214]: Using kernel command line parameters:  rd.driver.pre=btrfs rd.driver.pre=scsi_dh_alua rd.driver.pre=scsi_dh_emc rd.driver.pre=scsi_dh_rdac rd.driver.pre=dm_multipath root=/dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 rootfst… 
Oct 14 13:44:19 i3-4130 systemd[1]: Condition check resulted in Device-Mapper Multipath Default Configuration being skipped. 
Oct 14 13:44:19 i3-4130 systemd[1]: Starting Device-Mapper Multipath Device Controller... 
Oct 14 13:44:19 i3-4130 systemd[1]: Started Device-Mapper Multipath Device Controller. 
Oct 14 13:44:19 i3-4130 kernel: device-mapper: multipath service-time: version 0.3.0 loaded 
Oct 14 13:44:19 i3-4130 systemd[1]: Found device /dev/mapper/CT250MX500SSD1_2021E2A43EB9-part1. 
Oct 14 13:44:20 i3-4130 systemd[1]: Stopping Device-Mapper Multipath Device Controller... 
Oct 14 13:44:20 i3-4130 systemd[1]: Stopped Device-Mapper Multipath Device Controller. 
Oct 14 13:44:20 i3-4130 systemd[1]: Listening on Device-mapper event daemon FIFOs. 
Oct 14 13:44:21 i3-4130 kernel: BTRFS info (device dm-2): devid 1 device path /dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 changed to /dev/dm-2 scanned by systemd-udevd (714) 
Oct 14 13:44:21 i3-4130 kernel: BTRFS info (device dm-2): devid 1 device path /dev/dm-2 changed to /dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 scanned by systemd-udevd (714) 
Oct 14 13:44:21 i3-4130 kernel: BTRFS info (device dm-2): devid 1 device path /dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 changed to /dev/dm-2 scanned by systemd-udevd (714) 
Oct 14 13:44:21 i3-4130 kernel: BTRFS info (device dm-2): devid 1 device path /dev/dm-2 changed to /dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 scanned by systemd-udevd (714) 
Oct 14 13:46:16 i3-4130 kernel: BTRFS info (device dm-2): devid 1 device path /dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 changed to /dev/dm-2 scanned by systemd-udevd (4178) 
Oct 14 13:46:16 i3-4130 kernel: BTRFS info (device dm-2): devid 1 device path /dev/dm-2 changed to /dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 scanned by systemd-udevd (4178) 
Oct 14 13:46:16 i3-4130 kernel: BTRFS info (device dm-2): devid 1 device path /dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 changed to /dev/dm-2 scanned by systemd-udevd (4178) 
Oct 14 13:46:16 i3-4130 kernel: BTRFS info (device dm-2): devid 1 device path /dev/dm-2 changed to /dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 scanned by systemd-udevd (4178) 
**i3-4130:~ #**

‘snapper rollback’ and duping again fixed the problem. Problem is presumably caused by bogus kernel command line:

**i3-4130:~ #** journalctl -b -4 _PID=214 --no-pager  
-- Journal begins at Thu 2021-09-30 18:53:22 CEST, ends at Thu 2021-10-14 15:07:09 CEST. -- 
Oct 14 13:44:18 i3-4130 dracut-cmdline[214]: dracut-dracut-055+suse.119.g6c4187af-1.1 
Oct 14 13:44:18 i3-4130 dracut-cmdline[214]: Using kernel command line parameters:  rd.driver.pre=btrfs rd.driver.pre=scsi_dh_alua rd.driver.pre=scsi_dh_emc rd.driver.pre=scsi_dh_rdac rd.driver.pre=dm_multipath root=/dev/mapper/CT250MX500SSD1_2021E2A43EB9-part2 rootfsty pe=btrfs rootflags=rw,relatime,ssd,space_cache,subvolid=836,subvol=/@/.snapshots/517/snapshot,subvol=@/.snapshots/517/snapshot   BOOT_IMAGE=/boot/vmlinuz-5.14.9-1-default root=UUID=227128c2-8703-4859-a006-30dccf5b299c quiet plymouth.enable=0 net.ifnames=0 mitigations=auto 
**i3-4130:~ #**

Intended parameters are:

**i3-4130:~ #** journalctl -b _PID=217 --no-pager   
-- Journal begins at Thu 2021-09-30 18:53:22 CEST, ends at Thu 2021-10-14 15:07:09 CEST. -- 
Oct 14 14:28:53 i3-4130 dracut-cmdline[217]: dracut-dracut-055+suse.119.g6c4187af-1.1 
Oct 14 14:28:53 i3-4130 dracut-cmdline[217]: Using kernel command line parameters:  rd.driver.pre=btrfs root=UUID=227128c2-8703-4859-a006-30dccf5b299c rootfstype=btrfsrootflags=rw,relatime,ssd,space_cache,subvolid=984,subvol=/@/.snapshots/645/snapshot,subvol=@/.snapshots/645/snapshot   BOOT_IMAGE=/boot/vmlinuz-5.14.9-1-default root=UUID=227128c2-8703-4859-a006-30dccf5b299c quiet plymouth.enable=0 net.ifnames=0 mitigations=auto 
**i3-4130:~ #**