Results 1 to 6 of 6

Thread: firewalld and libvirt zone

  1. #1
    Join Date
    Aug 2008
    Location
    Belgium
    Posts
    234

    Default firewalld and libvirt zone

    I have a problem with a XEN VM. But I think that the problem is related to the dom0 and firewalld rules.

    In 2016 I created a VM (fully virtualized) on my server with this network typology:
    Code:
     ┌────────────────────────┐                        ┌───────┐
     │ Tumbleweed Server with │ eno1 (192.168.1.120)   │ CABLE │
     │ DHCP + DNS + firewalld ├────────────────────────│ Modem ├─── Internet
     │                        │enslaved in br0 (for VM)│       │
     │                        │                        └───────┘
     └────────────────────────┘                    Gateway (192.168.1.1)
    the bridge br0 was created when installing the xen tools. From the VM I could connect to internet and to the dom0.

    Some month ago I added a new internet connection via ppoe and some vlan and I needed to set firewalld.
    Thereafter I could not more start the VM . Not sure if problem is related to the new connections.
    Now I try to recreate the VM on my server.
    The server has 3 ethernet ports configured (eno1 for VM, eno2 for vlans, eno3 for ppoe)

    My new network topology look like this
    Code:
     ┌────────────────────────┐                      ┌───────┐
     │ Tumbleweed Server with │ eno3 (no IP)         │ CABLE │
     │ DHCP + DNS + firewalld ├───────────ppp0───────│ Modem ├─── Internet
     │                        │                      │       │
     │ do intervlan routing   │                      └────┬──┘
     └───┬────────────────┬───┘                    Gateway (192.168.1.1)
     eno2 (No IP)     eno1 (192.168.1.120)                
         │                │enslaved in br0 (for VM)       
         │                │                                    
    trunk│ port           │                               
     ┌───┴────────────────┴───────────────────────────────┴───┐
     │         TL─SG3216         Swithch Level 2              │
     │                                                        │
     │                         VLAN  ID                       │
     │       2                  3                    4        │
     │(192.168.20.0/24) (192.168.30.0/24)   (192.168.4.0/24)  │
     └───────┬──────────────────┬─────────────────────┬───────┘
             │                  │                     │
            PCs                 PCs                  Printer
      192.168.20.100─     192.168.30.100─      192.168.4.50
      192.168.20.199      192.168.30.199 
    when I create a VM with virt-manager or Yast the network cannot be found but the creation is succesfully using a local iso.
    I configured the network connection in the VM setting a static IP address. (192.168.90.2) and updated the local DNS for this range of address

    After this I'm able to connect to the dom0 but not to the internet from the VM.

    Before starting the VM the network configuration of the dom0 is like this
    Code:
    hpprol2:~ # ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
    2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
        link/ether 9c:8e:99:5b:48:12 brd ff:ff:ff:ff:ff:ff
        altname enp2s0f0
    3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
        link/ether 9c:8e:99:5b:48:13 brd ff:ff:ff:ff:ff:ff
        altname enp2s0f1
    4: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
        link/ether 9c:8e:99:5b:48:14 brd ff:ff:ff:ff:ff:ff
        altname enp2s0f2
    5: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
        link/ether 9c:8e:99:5b:48:15 brd ff:ff:ff:ff:ff:ff
        altname enp2s0f3
    6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
        link/ether ae:02:30:64:2e:32 brd ff:ff:ff:ff:ff:ff
        inet 192.168.1.100/24 brd 192.168.1.255 scope global br0:100
           valid_lft forever preferred_lft forever
        inet 192.168.1.101/24 brd 192.168.1.255 scope global secondary br0:101
           valid_lft forever preferred_lft forever
        inet 192.168.1.120/24 brd 192.168.1.255 scope global secondary br0
           valid_lft forever preferred_lft forever
    7: vlan3@eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
        link/ether 9c:8e:99:5b:48:13 brd ff:ff:ff:ff:ff:ff
        inet 192.168.30.1/24 brd 192.168.30.255 scope global vlan3
           valid_lft forever preferred_lft forever
    8: vlan4@eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
        link/ether 9c:8e:99:5b:48:13 brd ff:ff:ff:ff:ff:ff
        inet 192.168.4.1/24 brd 192.168.4.255 scope global vlan4
           valid_lft forever preferred_lft forever
    9: vlan2@eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
        link/ether 9c:8e:99:5b:48:13 brd ff:ff:ff:ff:ff:ff
        inet 192.168.20.1/24 brd 192.168.20.255 scope global vlan2
           valid_lft forever preferred_lft forever
    10: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc pfifo_fast state UNKNOWN group default qlen 3
        link/ppp 
        inet xx.xx.xx.xx peer xx.xx.xx.xx/32 scope global ppp0
           valid_lft forever preferred_lft forever
    11: virbr1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
        link/ether 52:54:00:3c:cc:26 brd ff:ff:ff:ff:ff:ff
        inet 192.168.90.1/24 brd 192.168.90.255 scope global virbr1
           valid_lft forever preferred_lft forever
    hpprol2:~ # ip route show
    default dev ppp0 scope link 
    10.24.97.36 dev ppp0 proto kernel scope link src 81.241.102.90 
    192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.100 
    192.168.4.0/24 dev vlan4 proto kernel scope link src 192.168.4.1 
    192.168.20.0/24 dev vlan2 proto kernel scope link src 192.168.20.1 
    192.168.30.0/24 dev vlan3 proto kernel scope link src 192.168.30.1 
    192.168.90.0/24 dev virbr1 proto kernel scope link src 192.168.90.1 linkdown 
    The virbr1 was created during creation of the VM.
    After starting the VM the network the network configuration in dom0 is (change only)
    Code:
    hpprol2:~ # ip a
    ...
    11: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
        link/ether 52:54:00:3c:cc:26 brd ff:ff:ff:ff:ff:ff
        inet 192.168.90.1/24 brd 192.168.90.255 scope global virbr1
           valid_lft forever preferred_lft forever
    13: vif2.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master virbr1 state UP group default qlen 32
        link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
    


    firewalld configuration:
    In firewalld I have a new zone libvirt where I activated services dhcp, dns, http, https, ftp, tftp
    I can reach the dom0 using ftp client
    I cannot ping an internet address
    When I start the browser in the VM I receive "Connection failed"
    Using wiresharlk in dom0 I see the next lines
    Code:
    No.    Time    Source    Destination    Protocol    Length    Info
    68    27.184922774    192.168.90.2    192.168.1.120    DNS    79    Standard query 0x5ec0 A search.opensuse.org
    69    27.185321284    192.168.1.120    192.168.90.2    DNS    139    Standard query response 0x5ec0 A search.opensuse.org CNAME proxy.opensuse.org CNAME proxy-nue.opensuse.org A 195.135.221.140
    70    27.185670715    192.168.90.2    195.135.221.140    TCP    74    59392 → 443 [SYN] Seq=0 Win=65340 Len=0 MSS=1452 SACK_PERM=1 TSval=689059451 TSecr=0 WS=128
    71    27.185713311    192.168.90.1    192.168.90.2    ICMP    102    Destination unreachable (Port unreachable)

    In firewalld port 443 is allowed in zone libvirt.

    I have some direct rules in firewalld in the default zone that are needed to reach internet from the vlan.
    Code:
    # cat /etc/firewalld/direct.xml
    <?xml version="1.0" encoding="utf-8"?>
    <direct>
      <rule ipv="ipv4" table="filter" chain="FORWARD" priority="0">-i ppp0 -o vlan2 -m state --state RELATED,ESTABLISHED -j ACCEPT</rule>
      <rule ipv="ipv4" table="filter" chain="FORWARD" priority="0">-i ppp0 -o vlan3 -m state --state RELATED,ESTABLISHED -j ACCEPT</rule>
      <rule ipv="ipv4" table="filter" chain="FORWARD" priority="1">-i vlan2 -o vlan3 -j REJECT</rule>
      <rule ipv="ipv4" table="filter" chain="FORWARD" priority="1">-i vlan3 -o vlan2 -j REJECT</rule>
      <rule ipv="ipv4" table="filter" chain="FORWARD" priority="0">-i ppp0 -o br0 -m state --state RELATED,ESTABLISHED -j ACCEPT</rule>
      <rule ipv="ipv4" table="nat" chain="POSTROUTING" priority="0">-o ppp0 -j MASQUERADE</rule>
      <passthrough ipv="ipv4">-I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu</passthrough>
    </direct>
    So I added two rules for the virbr1
    Code:
      <rule ipv="ipv4" table="filter" chain="FORWARD" priority="0">-i virbr1 -o ppp0 -j ACCEPT</rule>
      <rule ipv="ipv4" table="filter" chain="FORWARD" priority="0">-i ppp0 -o virbr1 -m state --state RELATED,ESTABLISHED -j ACCEPT</rule>
    but still no internet connection.

    I think that I missed something in the firewalld configuration but I'm unable to find a solution to have vm connection with the internet

    Any idea?
    Many thanks in advance
    Philippe
    Tumbleweed (x86_64) Kernel 5.10.5-1 with KDE plasma

  2. #2
    Join Date
    Sep 2010
    Location
    Poland
    Posts
    1,959

    Default Re: firewalld and libvirt zone

    Really nice ASCII drawings
    Can you add a rule to log dropped packets? Maybe it will give you some hints on what is failing.
    There was an option to enable that in the old firewall there must be a way to add it with firewalld since I think it's using iptables under the hood.
    Best regards,
    Greg

  3. #3
    Join Date
    Sep 2012
    Posts
    6,315

    Default Re: firewalld and libvirt zone

    Quote Originally Posted by phil524 View Post
    Code:
    # cat /etc/firewalld/direct.xml
    In Tumbleweed firewalld is based on nftables by default; it is impossible to allow anything using direct rules that is not allowed by main firewalld configuration.

    You may consider exploring firewalld policies which offer support to configure forwarding between zones or switching backend back to iptables which should enable you to use direct rules to overrule firewalld (pun intended).

  4. #4
    Join Date
    Aug 2008
    Location
    Belgium
    Posts
    234

    Default Re: firewalld and libvirt zone

    Hello,

    Quote Originally Posted by arvidjaar View Post
    In Tumbleweed firewalld is based on nftables by default; it is impossible to allow anything using direct rules that is not allowed by main firewalld configuration.

    You may consider exploring firewalld policies which offer support to configure forwarding between zones or switching backend back to iptables which should enable you to use direct rules to overrule firewalld (pun intended).
    Thanks for your remark about the limits of direct rules.
    I reread the xen network installation (see https://doc.opensuse.org/documentati...bvirt-networks) and I recreated the xen VM and choosing a network definition using an existing bridge (https://doc.opensuse.org/documentati...8.4.7.6.7.6.11).
    This solved the problem and I have now internet connection in the VM.

    This avoid also the creation of a virbr1 bridge and the activation of a libvirt zone in firewalld. The xen VM is using the default zone where br0 is present.
    I removed the direct rules that I created in firewalld for the virbr1 device.

    As far as I understand the xen network using libvirt connection is a Layer 3 forwarding while using directly the bridge is a Layer 2
    See https://doc.opensuse.org/documentati...tworks-bridged where they explain:
    Code:
    The network bridge configuration provides a Layer 2 switch for VM Guests, switching Layer 2 Ethernet packets between ports on the bridge based on MAC addresses associated with the ports. This gives the VM Guest Layer 2 access to the VM Host Server's network. This configuration is analogous to connecting the VM Guest's virtual Ethernet cable into a hub that is shared with the host and other VM Guests running on the host. The configuration is often referred to as shared physical device.
    Many thanks
    Philippe
    Tumbleweed (x86_64) Kernel 5.10.5-1 with KDE plasma

  5. #5
    Join Date
    Sep 2010
    Location
    Poland
    Posts
    1,959

    Default Re: firewalld and libvirt zone

    Quote Originally Posted by arvidjaar View Post
    In Tumbleweed firewalld is based on nftables by default; it is impossible to allow anything using direct rules that is not allowed by main firewalld configuration.

    You may consider exploring firewalld policies which offer support to configure forwarding between zones or switching backend back to iptables which should enable you to use direct rules to overrule firewalld (pun intended).
    Oh I didn't know this. I thought it was iptables. Guess I learned something new today. Thanks
    Best regards,
    Greg

  6. #6
    Join Date
    Sep 2010
    Location
    Poland
    Posts
    1,959

    Default Re: firewalld and libvirt zone

    Quote Originally Posted by phil524 View Post
    Hello,



    Thanks for your remark about the limits of direct rules.
    I reread the xen network installation (see https://doc.opensuse.org/documentati...bvirt-networks) and I recreated the xen VM and choosing a network definition using an existing bridge (https://doc.opensuse.org/documentati...8.4.7.6.7.6.11).
    This solved the problem and I have now internet connection in the VM.

    This avoid also the creation of a virbr1 bridge and the activation of a libvirt zone in firewalld. The xen VM is using the default zone where br0 is present.
    I removed the direct rules that I created in firewalld for the virbr1 device.

    As far as I understand the xen network using libvirt connection is a Layer 3 forwarding while using directly the bridge is a Layer 2
    See https://doc.opensuse.org/documentati...tworks-bridged where they explain:
    Code:
    The network bridge configuration provides a Layer 2 switch for VM Guests, switching Layer 2 Ethernet packets between ports on the bridge based on MAC addresses associated with the ports. This gives the VM Guest Layer 2 access to the VM Host Server's network. This configuration is analogous to connecting the VM Guest's virtual Ethernet cable into a hub that is shared with the host and other VM Guests running on the host. The configuration is often referred to as shared physical device.
    Many thanks
    Philippe
    Thank you for posting the solution. I'm really glad it works now.
    Best regards,
    Greg

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •