How can Network Manager and Virtual Machine Manager networks coexist?

[joba1]

I have openSuse 11.4 on my notebook.
I use Network Manager due to using lots of WLANs.
For testing and development I use Virtual Machine Manager and virsh to setup and start/stop kvm images.
Those images usually have an interface for NAT internet access and one or more interfaces to communicate with each other.

Network Manager is running fine. Switching connectivity works well.
The Virtual Machine Manager networks also run fine. VMs can access the internet and talk to eachother.

But as soon as the Network Manager changes something (e.g other WLAN, or reconnect due to WLAN was out of reach) the VMs loose (at least) their internet connection.

I see that the hosts firewall rules for them are lost. Not sure if this is all.

If I restart the virtual interfaces and add them to the bridge manually, all is fine again.

How can I prevent the Network Manager from destroying the firewall rules and bridges related to the VMs?


P.S.: will provide the related commands and output soon...

[joba1]

Ok, here is the host commands showing the current network config, switching wlan and then restoring network connectivity for the VM.

Question is: how can I avoid loss of network connectivity in the VM?
Or if unavoidable, how can I automate restoring it in a clean way?

To clarify: virbr1/vnet0/192.168.200.0 is the public NAT network of the VMs

root@job2 ~
# ip addr show dev wlan0
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:26:c7:3c:e8:1c brd ff:ff:ff:ff:ff:ff
    inet 192.168.11.101/24 brd 192.168.11.255 scope global wlan0

root@job2 ~
# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.000000000000       yes
virbr1          8000.fe54004530b0       yes             vnet0
virbr2          8000.fe5400845bbc       yes             vnet1

root@job2 ~
# ip addr show dev virbr1
23: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fe:54:00:45:30:b0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.200.1/24 brd 192.168.200.255 scope global virbr1

root@job2 ~
# iptables -L -t nat | grep 192.168
MASQUERADE  tcp  --  192.168.200.0/24    !192.168.200.0/24    masq ports: 1024-65535 
MASQUERADE  udp  --  192.168.200.0/24    !192.168.200.0/24    masq ports: 1024-65535 
MASQUERADE  all  --  192.168.200.0/24    !192.168.200.0/24    

root@job2 ~
# iptables -L | grep 192.168
ACCEPT     all  --  anywhere             192.168.200.0/24    state RELATED,ESTABLISHED 
ACCEPT     all  --  192.168.200.0/24     anywhere            

root@job2 ~
# ssh jobv1 ping -c 2 www.suse.de
Password: 
PING turing.suse.de (195.135.220.3) 56(84) bytes of data.
64 bytes from turing.suse.de (195.135.220.3): icmp_req=1 ttl=48 time=86.9 ms
64 bytes from turing.suse.de (195.135.220.3): icmp_req=2 ttl=48 time=64.9 ms

--- turing.suse.de ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 64.966/75.964/86.963/11.001 ms

root@job2 ~
# grep jobv1 /etc/hosts
192.168.200.11  jobv1

root@job2 ~
# nmcli con up uuid 'ad9289e1-e87b-48e1-86e5-6c7d13a282c3'
Active connection state: activating
Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/15
state: activated
Connection activated

root@job2 ~
# nmcli con up uuid '53049bf2-15c8-4c4f-a921-8af4a03facf8'
Active connection state: activating
Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/16
state: activated
Connection activated

root@job2 ~
# ssh jobv1 ping -c 2 www.suse.de
Password: 
PING turing.suse.de (195.135.220.3) 56(84) bytes of data.

--- turing.suse.de ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1000ms


root@job2 ~
# iptables -L | grep 192.168

root@job2 ~
# iptables -L -t nat | grep 192.168

root@job2 ~
# ip addr show dev virbr1
23: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fe:54:00:45:30:b0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.200.1/24 brd 192.168.200.255 scope global virbr1

root@job2 ~
# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.000000000000       yes
virbr1          8000.fe54004530b0       yes             vnet0
virbr2          8000.fe5400845bbc       yes             vnet1

root@job2 ~
# virsh net-destroy public
Network public destroyed


root@job2 ~
# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.000000000000       yes
virbr2          8000.fe5400845bbc       yes             vnet1

root@job2 ~
# virsh net-start public
Network public started


root@job2 ~
# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.000000000000       yes
virbr1          8000.000000000000       yes
virbr2          8000.fe5400845bbc       yes             vnet1

root@job2 ~
# iptables -L -t nat | grep 192.168
MASQUERADE  tcp  --  192.168.200.0/24    !192.168.200.0/24    masq ports: 1024-65535 
MASQUERADE  udp  --  192.168.200.0/24    !192.168.200.0/24    masq ports: 1024-65535 
MASQUERADE  all  --  192.168.200.0/24    !192.168.200.0/24    

root@job2 ~
# iptables -L | grep 192.168
ACCEPT     all  --  anywhere             192.168.200.0/24    state RELATED,ESTABLISHED 
ACCEPT     all  --  192.168.200.0/24     anywhere            

root@job2 ~
# brctl addif virbr1 vnet0

root@job2 ~
# ssh jobv1 ping -c 2 www.suse.de
Password: 
PING turing.suse.de (195.135.220.3) 56(84) bytes of data.
64 bytes from turing.suse.de (195.135.220.3): icmp_req=1 ttl=48 time=66.9 ms
64 bytes from turing.suse.de (195.135.220.3): icmp_req=2 ttl=48 time=92.9 ms

--- turing.suse.de ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 66.935/79.924/92.913/12.989 ms

[joba1]

Just in case someone has the same problem: This is the script I wrote to restore the network connectivity.

It would already help me a bit if I could find a Network Manager hook where I can run this script. Ideas?

#!/bin/bash

# restore VM network connectivity after Network Manager destroyed it

# list active virtual networks
vnets=`virsh net-list | grep active | sed 's/ *active *[yn][eo]s* *//'`
[ -n "$vnets" ] || exit

# determine bridges associated those networks
vbridges=`for n in $vnets; do virsh net-info "$n" | awk -v n="$n" '/Bridge:/ { print $2" "n }'; done`

# determine interfaces associated with those bridges
vbpattern=`echo "$vbridges" | awk 'BEGIN { d="" } { printf "%s%s", d, $1; d="|" }'`
binterfaces=`brctl show | awk 'NR>1 && $2 != "" { b=$1 } NR>1 && $4 != "" { print $4" "b } NR>1 && $2 == "" { print $1" "b }'`
vinterfaces=`echo "$binterfaces" | egrep "$vbpattern"`

# restart the virtual networks (destroys bridges association to interfaces)
for n in $vnets; do
        virsh net-destroy "$n"
done
sleep 1
for n in $vnets; do
        virsh net-start "$n"
done

# reassociate interfaces with the bridges
[ -n "$vinterfaces" ] || exit
echo "$vinterfaces" | while read i b; do
        echo "add interface '$i' to bridge '$b'"
        brctl addif $b $i
done

[drab]

joba1 wrote:

>
> Just in case someone has the same problem: This is the script I wrote to
> restore the network connectivity.
>
> It would already help me a bit if I could find a Network Manager hook
> where I can run this script. Ideas?

When an interface is brought up/down by Network Manager, the scripts in

/etc/NetworkManager/dispatcher.d/

are called with two arguments:

- interface (wlan0, eth0, etc)
- action (up, down, vpn-down, etc)

If you put your script there it should be run automatically. I had to do
this approach to make Network Manager keep a fixed ip address.

Hope that helps.

--
Don

[joba1]

Thanks, that would have helped.

In the meantime I found out that I can just enable masquerading on the host to have the same effect.
The firewall rules set by libvirt are still destroyed, but the more general rules for masquerading do the same job, so no harm done.

Now I can let libvirtd autostart my bridges for the virtual machines (until now I had to manually start them after the normal network was active) and let network manager switch my external connection as needed. My vm's just keep their connction: Great!