Page 1 of 2 12 LastLast
Results 1 to 10 of 16

Thread: virt-manager NAT networks inactive on system boot

  1. #1

    Default virt-manager NAT networks inactive on system boot

    Hello,
    I am using libvirt-kvm and virt-manager on my opensuse tumbleweed desktops and I noticed an issue.

    All virtual networks created via virt-manager and configured to be active at boot, do not actually come up when I boot the computers. Those remain defined in virt-manager but are not active, unless i manually go and turn them to active. This is on my 2 desktop system, both on KDE plasma and both using networkmanager network service.
    If I use the command "sudo virsh net-list" on boot, list turns out empty. If I manually start the NAT networks from virt-manager and re run "sudo virsh net-list", the networks show up as active persistent and autostart.

    Interesting enough, if I do the same on a 3rd openSUSE tumbleweed box, that is a headless server, no gui and using wicked network service, NAT networks do come up on system boot as expected.

    Is there anyone that can help me troubleshoot the issue and fix it on the desktops that run KDE plasma and networkmanager?

  2. #2
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,935
    Blog Entries
    2

    Default Re: virt-manager NAT networks inactive on system boot

    Ok,
    Took me awhile to work my way through the issues...

    First,
    I suspect you may have installed your Hypervisor and Tools during your initial install...
    On a whim I did this (which I have always heavily criticized) to review what happens...
    Which produced the problem described...
    libvirtd fails to start on boot.
    libvirtd can be started manually, but on close inspection is only partially functional.
    KVM kernel modules built and working which can be viewed by running dmesg or journalctl -b.

    The first issue I ran into is that there is a libvirtd issue where it will fail connecting to libnetcontrol if you're running Network Manager.
    Since Network Manager is enabled by default in 15.1,
    So, that's the first issue... You can't run libvirt using Network Manager, you'll have to switch to Wicked... This can be seen and verified running
    Code:
    systemctl status libvirtd.service
    That's important to know, and an interesting issue I didn't run into before... but seems to be a documented issue at least since LEAP 15.0.

    But,
    At least for my situation where I installed Hypervisor and Tools during the initial install, that isn't enough, you'll still see something else (I've forgotten what that is for the moment and don't want to roll everything back just to identify it) won't start, and force-re-installing the kvm-server and kvm-tools and their patterns didn't fix the problem. I completely removed Kvm-server and kvm-tools and libvirt packages to get a working system.
    Code:
    zypper rm kvm-server kvm-tools patterns-kvm_server kvm-tools patterns-kvm_tools 'libvirt*'
    Followed by using YaST to install Hypervisor and Tools.

    After completing the above,
    Finally libvirtd started up on boot and everything looks normal.

    It's interesting that when incorrectly installed,
    Libvirtd fails to start on boot silently without throwing any events marked as errors, but starting the service manually revealed the failures that had to be overcome...

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  3. #3

    Default Re: virt-manager NAT networks inactive on system boot

    Hello,

    Thanks for your reply. I had a look at the services and did not notice anything failed. Very interesting indeed.
    I am sure that on my desktop+networkmanager based machines I installed KVM from Yast after I installed and updated tumbleweed.
    On the headless server+wicked, I don't recall if I installed KVM during installation or later on. That never had any issues so I did not pay too much attention to it.

    I also tried disabling apparmor on my desktop to see if it was any help, but not that I can say.

    Going to run the action you recommended and follow up.

    Thank you!
    Mauro

  4. #4

    Default Re: virt-manager NAT networks inactive on system boot

    I tried remove all kvm components, reboot, reinstall, reboot, no change.
    I changed apparmor settings on all profiles to complain, reboot and no change.

    After the changes above, situation is the same:
    If I reboot my laptop/deskop, open virt-manager, go to QEMU/KVM I see my virtual networks defied there, marked to start on boot. but currently inactive.
    If I run sudo virsh net-list, the list is empty
    If I check service status by running sudo systemctl status libvirtd.service, I see it active (running) no issue that I can see.

    So what I tried next:
    1. went to check the logs and found a lot of errors regarding libvirtd on startup, as Tsu2 said. Attaching error logs below.
    2. without rebooting, I issued a sudo systemctl stop libvirtd.service
    3. without rebooting, I issued a sudo systemctl start libvirtd.service

    libvirtd came back, and from both virt-manager and sudo virsh net-list, I can see my virtual networks active.

    As a temporary workaround I guess I can make a small login script that restarts libvirtd.service.
    As for a permanent solution, unless you guys have other ideas, I will search openSUSE bugzilla if there are open bugs. If not, I will file one.

    PS
    I have a couple of centOS7 installs, using networkmanager and it seems there is no problem with virtual networks after reboot.

    ilvipero@MGLT-AERO14:~> sudo journalctl -b | grep libvirt
    Jul 08 14:22:13 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-sock → /run/libvirt/virtlockd-sock; please update the unit file accordingly.
    Jul 08 14:22:13 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-admin-sock → /run/libvirt/virtlockd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:13 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-sock → /run/libvirt/virtlogd-sock; please update the unit file accordingly.
    Jul 08 14:22:13 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-admin-sock → /run/libvirt/virtlogd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:18 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-sock → /run/libvirt/virtlogd-sock; please update the unit file accordingly.
    Jul 08 14:22:18 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-admin-sock → /run/libvirt/virtlogd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:18 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-sock → /run/libvirt/virtlockd-sock; please update the unit file accordingly.
    Jul 08 14:22:18 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-admin-sock → /run/libvirt/virtlockd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:19 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-sock → /run/libvirt/virtlogd-sock; please update the unit file accordingly.
    Jul 08 14:22:19 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-admin-sock → /run/libvirt/virtlogd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:19 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-sock → /run/libvirt/virtlockd-sock; please update the unit file accordingly.
    Jul 08 14:22:19 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-admin-sock → /run/libvirt/virtlockd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:19 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-sock → /run/libvirt/virtlogd-sock; please update the unit file accordingly.
    Jul 08 14:22:19 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-admin-sock → /run/libvirt/virtlogd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:19 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-sock → /run/libvirt/virtlockd-sock; please update the unit file accordingly.
    Jul 08 14:22:19 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-admin-sock → /run/libvirt/virtlockd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:20 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-sock → /run/libvirt/virtlogd-sock; please update the unit file accordingly.
    Jul 08 14:22:20 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlogd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-admin-sock → /run/libvirt/virtlogd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:20 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-sock → /run/libvirt/virtlockd-sock; please update the unit file accordingly.
    Jul 08 14:22:20 MGLT-AERO14 systemd[1]: /usr/lib/systemd/system/virtlockd-admin.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-admin-sock → /run/libvirt/virtlockd-admin-sock; please update the unit file accordingly.
    Jul 08 14:22:21 MGLT-AERO14 libvirtd[1757]: 2019-07-08 06:22:21.560+0000: 1757: info : libvirt version: 5.4.0
    Jul 08 14:22:21 MGLT-AERO14 libvirtd[1757]: 2019-07-08 06:22:21.560+0000: 1757: info : hostname: MGLT-AERO14
    Jul 08 14:22:21 MGLT-AERO14 libvirtd[1757]: 2019-07-08 06:22:21.560+0000: 1757: warning : virGetHostnameImpl:540 : getaddrinfo failed for 'MGLT-AERO14': Name or service not known
    Jul 08 14:22:21 MGLT-AERO14 libvirtd[1757]: libvirt version: 5.4.0
    Jul 08 14:22:21 MGLT-AERO14 libvirtd[1757]: hostname: MGLT-AERO14
    Jul 08 14:22:21 MGLT-AERO14 libvirtd[1757]: Failed to intialize libnetcontrol. Management of interface devices is disabled
    Jul 08 14:22:23 MGLT-AERO14 libvirtd[1757]: internal error: Child process (VIR_BRIDGE_NAME=virbr1 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/nat.conf --leasefile-ro --dhcp-script=/usr/lib64/libvirt/libvirt_leaseshelper) unexpected exit status 5:
    Jul 08 14:22:24 MGLT-AERO14 libvirtd[1757]: internal error: Child process (VIR_BRIDGE_NAME=virbr0 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib64/libvirt/libvirt_leaseshelper) unexpected exit status 5:
    Jul 08 14:26:52 MGLT-AERO14 libvirtd[1757]: this function is not supported by the connection driver: virConnectNumOfInterfaces
    ilvipero@MGLT-AERO14:~>

  5. #5

    Default Re: virt-manager NAT networks inactive on system boot

    Quick update for the record.I added this line to my crontab:
    @reboot root ( sleep 30 ; systemctl restart libvirtd.service )

    Dirty hack but works for me, I hope it helps others down the road.

  6. #6
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,935
    Blog Entries
    2

    Default Re: virt-manager NAT networks inactive on system boot

    Quote Originally Posted by ilvipero View Post
    Quick update for the record.I added this line to my crontab:
    @reboot root ( sleep 30 ; systemctl restart libvirtd.service )

    Dirty hack but works for me, I hope it helps others down the road.
    On the other hand...
    As I described, you'd likely determine the cause of your failure by running
    Code:
    systemctl start libvirtd && systemctl status libvirtd
    And,
    If the problem you see is as I described, then you can fix it as I described (properly re-install).

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  7. #7

    Default Re: virt-manager NAT networks inactive on system boot

    Hi,

    Actually I did but there are no errors when I restart the service manually. Also re-installing per your recommendation was tried but did not help in my case.

    But yes your help was of great value because I had another better look at my logs during boot up and found the errors with libvirtd .you can see in the logs i posted above.


    Thanks!
    Mauro

  8. #8
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,935
    Blog Entries
    2

    Default Re: virt-manager NAT networks inactive on system boot

    Post the entire result of
    Code:
    systemctl start libvirtd.service
    Or, if it's already running, then restart instead of start...

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  9. #9

    Default Re: virt-manager NAT networks inactive on system boot

    Tsu2, thank you for your help.

    libvirtd.service - Virtualization daemon
    Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: disabled)
    Active: active (running) since Tue 2019-07-09 17:19:55 CST; 16s ago
    Docs: man:libvirtd(8)
    https://libvirt.org
    Main PID: 11568 (libvirtd)
    Tasks: 19 (limit: 32768)
    Memory: 15.9M
    CGroup: /system.slice/libvirtd.service
    ├─ 2729 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib6>
    ├─ 2730 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib6>
    └─11568 /usr/sbin/libvirtd --listen

  10. #10
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,935
    Blog Entries
    2

    Default Re: virt-manager NAT networks inactive on system boot

    Quote Originally Posted by ilvipero View Post
    Tsu2, thank you for your help.

    libvirtd.service - Virtualization daemon
    Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: disabled)
    Active: active (running) since Tue 2019-07-09 17:19:55 CST; 16s ago
    Docs: man:libvirtd(8)
    https://libvirt.org
    Main PID: 11568 (libvirtd)
    Tasks: 19 (limit: 32768)
    Memory: 15.9M
    CGroup: /system.slice/libvirtd.service
    ├─ 2729 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib6>
    ├─ 2730 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib6>
    └─11568 /usr/sbin/libvirtd --listen
    OK, and it's customary to enclose console output (and input) in CODE tagas (the hash button in the text editor).
    According to what you posted, your system is working.
    The object was to try to capture output when it sin't working.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •