openvswitch loses configuration on reboot

I hope this isn’t a double post. I can’t seem to find my original.

I followed these instructions with minor modifications:

https://en.opensuse.org/Portal:Wicked/OpenvSwitch

Here are my steps:

Created my bridge and my ports:
ovs-vsctl add-br ovsbr0
for i in {0…24}
do
ovs-vsctl add-port ovsbr0 port${i} – set Interface port${i} type=internal
done

No errors and ovs-vsctl show showed everything as fine.

Created my ifcfg* scripts:
echo STARTMODE=‘auto’ >ifcfg-ovsbr0
echo BOOTPROTO=static >>ifcfg-ovsbr0
echo IPADDR=10.0.0.254 >>ifcfg-ovsbr0
echo NETMASK=255.255.255.0 >>ifcfg-ovsbr0
echo LINK_REQUIRED=‘no’ >>ifcfg-ovsbr0
echo OVS_BRIDGE=‘yes’ >>ifcfg-ovsbr0
echo OVS_BRIDGE_PORT_DEVICE=‘port0’ >>ifcfg-ovsbr0

for i in {0…24}
do
echo STARTMODE=‘auto’ > ifcfg-port${i}
echo BOOTPROTO=‘none’ >>ifcfg-port${i}
done

Performed ifup all connected my KVM VM systems to the ports and everything worked (e.g. ssh, ping, etc.).

When I reboot the bridge and ports are not there. Running this again re-creates them fine:
ovs-vsctl add-br ovsbr0
for i in {0…24}
do
ovs-vsctl add-port ovsbr0 port${i} – set Interface port${i} type=internal
done
ifup all

Am I missing a step to preserve the database? I could not find anything in the openvswitch documentation.

Any help will be great appreciated.

Although I haven’t done this myself personally,
To my eye,
the following is possibly faulty because although you assigned to a bridge device, you didn’t specify an interface like what is described in the referenced openvswitch portal (which should probably be renamed as a SDB, portals I usually see don’t get into the technical stuff)

ovs-vsctl add-br ovsbr0
for i in {0..24}
do
ovs-vsctl add-port ovsbr0 port${i} -- set Interface port${i} type=internal
done

Although it’s cool to set your ports programmaticly, it might be more clear for “only” 25 ports to just use repetitive statements for sureness (You can also modify to use programming after you’re sure your config works)

TSU

As for your reboot problem,
Did you make the required configuration in the openvswitch service?

# /usr/lib/systemd/system/openvswitch.service
 [Unit]
 Description=Open vSwitch
 Before=network.service
 ...

TSU

In Leap 42.3 that file is already configured correctly:


kvm:/var/lib # more /usr/lib/systemd/system/openvswitch.service
[Unit]
Description=Open vSwitch
Before=network.target network.service
After=network-pre.target ovsdb-server.service ovs-vswitchd.service
PartOf=network.target
BindsTo=ovsdb-server.service
BindsTo=ovs-vswitchd.service

[Service]
Type=oneshot
ExecStart=/bin/true
ExecStartPost=/usr/share/openvswitch/scripts/ovs-dpdk-migrate-2.6.sh
ExecReload=/bin/true
ExecStop=/bin/true
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
kvm:/var/lib # 


Make I am making a big assumption, but here goes.

openvswitch has a backend database. Since my configuration works with no errors or complaints when setting it up and when querying the database I assume the schema is correct. Yet the database retains no data after reboot.

You can probably test your assumption that the settings aren’t in your database by restarting the wicked service (which is described as a working alternative to rebooting).
I’m assuming that the openvswich configuration doesn’t somehow exist in memory during your User session, so restarting the wicked service should read the configuration from the openvswitch database and in so doing verify your database contain your settings.

After a reboot,
You can also try invoking openvswitch without re-creating your configuration(eg systemctl restart openvswitch) which can further indicate whether your database contains your settings.

If in the end you can’t find a solution, creating network settings and objects like this is fairly commonly done by invoking with lines in /etc/sysctl.conf. If you end up doing this, I recommend you read related blogs and MAN pages, I also used this method for modifying TCP/IP stack settings in an article I wrote long ago

https://sites.google.com/site/4techsecrets/optimize-and-fix-your-network-connection/tcp-buffer-re-sizing/persisting-tcp-buffer-and-congestion-control-algorithm-changes

TSU

Curious,
If your openvswitch is not working, is the service running?
Especially if the service is not running, you may want to view the journal snippet in

systemctl status openvswitch.service

And, although I’d presume that openvswitch.service cannot start without the following perhaps they should be checked as well

systemctl status ovsdb-server.service
systemctl status ovs-vswitchd.service

TSU

Thanks, those are great troubleshooting suggestions. The /etc/sysctl.conf is certainly an option and easily done. I just hate when things don’t work as designed :wink:

Yes all services are running. My apologies for leaving it out of the post. My original contained that information, but somehow I messed up in posting it.

Thank you very much for your help so far. I definitely have a timing issue between the openvswitch.service and the network.service.

If I set the bridge and ports up as I originally did and do not reboot they function as they should.

If I go into yast and disable wicked, reboot, login and check my openvswitch database everything has been saved. Restarting wicked in yast brings the system back online and openvswitch works as I have configured it.

Not disabling wicked before a reboot clears/empties/corrupts the openvswitch database.

I would much rather somehow place a pause between the two services and have a longer boot process than have to disable and re-enable wicked. Otherwise, when the system requires a hard reset I am again having to reconfigure openvswitch.

Any suggestions? I am surprised no one else has run across this issue. i would think docker would utilize this although possibly they are moving the openvswitch part into userspace with DPDK and do not have the same issues.

In the systemd Unit file,
You can try substituting “Requires=” in place of “After=”

Documentation for “Requires=”
https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Requires=

TSU

Openswitch database is automatically preserved, this is specific issue of interaction with wicked. Wicked creates OVS bridges dynamically during ifup and destroys them during ifdown. On sytsem shutdown during network stop ifdown for all interfaces is performed which removes bridge.

Wicked can handle only very limited subset of OVS features - it can only setup bridges on top of existing physical interfaces, it cannot create virtual ports.

Your only workaround is to set STARTMODE=nfsroot for bridge device. Note that this will completely disable ifdown for this interface, even manually. Apparently current wicked also supports --no-delete option on interface which may help in this case although I am not sure whether it is possible to mark bridge as “no delete” in configuration.

Another big hammer is to simply disable ExecStop action of wicked.service thus leaving network running. This means someone may still accidentally run “ifdown ovsbridge” removing it from OVS database.

Requires is orthogonal to After; one cannot “substitute” another. You apparently did not even attempt to read your link yourself.

Thank you very much for this response. It at least confirms that I am not crazy. Since I am actually working in a development, and not a production environment I am beginning to think the easier route is to just script the build of the bridge and internal ports each time before spinning up the virtual environment. I will admit that my background is in large enterprise architecture and I started this thread to determine if I was seeing a bug since I couldn’t find anything on this in the documentation or through searches. Your answer helps me accept that this is at least an understandable occurrence, and why I am seeing it. I seem to be good at finding eclectic issues.

So this works!


kvm:~ # cat /etc/sysconfig/network/ifcfg-ovsbr0 
STARTMODE=nfsroot
BOOTPROTO=static
IPADDR=10.0.0.254
NETMASK=255.255.255.0
LINK_REQUIRED=no
OVS_BRIDGE=yes
OVS_BRIDGE_PORT_DEVICE=port0
kvm:~ # 

And an example of all my port files since they are all the same:


kvm:~ # cat /etc/sysconfig/network/ifcfg-port0
STARTMODE=nfsroot
BOOTPROTO=none
kvm:~ # 

I guess the final question is should I accept this as a solution or should I file it in a bug report?

As I answered in another discussion, IMHO without OVS_BRIDGE=yes wicked should not destroy bridge on ifdown. In my testing it still did.

Second topic for bug report is documentation - the behavior is absolutely unclear after reading manual page. A better explanation what OVS_BRIDGE actually does would certainly be helpful.

https://bugzilla.opensuse.org/show_bug.cgi?id=1091786