MicroOS: Configure static network IP via combustion?

docbobo · March 21, 2022, 9:06am

I’ve been trying to set up a static IP address for MicroOS (Tumbleweed) via either combustion or ignition. Documentation for Wifi setup (https://en.opensuse.org/Portal:MicroOS/WifiSetup) kind of suggested, that this could be done via ifcfg-* files but after two days of trying all kinds of different things, I came to realize that this isn’t working; at least, it isn’t working with Tumbleweed.

Right now, I ended up with creating an appropriate *-nmconnection file under /etc/NetworkManager/system-connections. This gets me the desired result iff the system is rebooted at the end of the combustion run. Before that, I still get the IP address assigned via DHCP (I am using combustion: network because I am installing some packages). Unfortunately, things like systemctl reboot do not seem to work as part of the combustion run either.

Is this the right way to assign a static IP address to the interface in an automated way? If not, what’s the right way to do this?
If this is correct, how do I get the system to pick up the changes from my nmconnection file at the end of the combustion run?

arvidjaar · March 21, 2022, 10:37am

Wicked is being deprecated and NeworkManager should be default on Tumbleweed now. Your description agrees with this information.

If this is correct, how do I get the system to pick up the changes from my nmconnection file at the end of the combustion run?

Did you try to simply restart NetworkManager?

docbobo · March 21, 2022, 12:03pm

Yes, I tried that as well. There are a few problems with that, though.

Even manually running that command does not yield the correct result. Why? Because there is now an additional nmconnection file under /run/NetworkManager/system-connections which is being preferred. What works is removing that file, then running nmcli con reload
Running systemctl restart NetworkManager from combustion results in Running in chroot, ignoring request.
Deleting anything from /run/NetworkManager/system-connections [FONT=arial]or [/FONT]/var/run/NetworkManager/system-connections [FONT=arial]will somehow break the boot process, complaining about a corrupt filesysttem[/FONT]
Running nmcli from combustion also doesn’t work and produces errors as well: [FONT=courier new]Could not create NMClient object: Could not connect: No such file or directory.[/FONT]

docbobo · March 21, 2022, 1:14pm

Okay. So when using “combustion: network”, it seems the best way to configure the static IP of the machine would be via kernel parameters. A combination work ignition and combustion seems to produce the correct result:


{
  "ignition": {
    "version": "3.3.0"
  },
  "kernelArguments": {
    "shouldExist": 
      "ip=10.0.0.111::10.0.0.1:255.255.255.0:k3s-server-temp:enp6s18:on:10.0.0.1"
    ]
  }
}

This ignition file will re-configure the kernel to provide an additional cmdline parameter, then restart the system. Upon restart, it will have the correct IP address and will run the combustion script…

malcolmlewis · March 21, 2022, 1:23pm

Hi
A question, this is a baremetal install of MicroOS? I do use MicroOS/TW and k3s clusters but with vagrant/libvirt.

docbobo · March 21, 2022, 1:33pm

This is on Proxmox, so basically libvirt, I guess.

malcolmlewis · March 21, 2022, 1:41pm

Hi
Hmmm, so with vagrant I set the ip address, either private or physical, you can’t do that with Proxmox? FWIW, my MicroOS systems are all running wicked, only things I configure are ssh keys/access, a k3s server token and some sysctl config.

docbobo · March 21, 2022, 1:48pm

Theoretically, I could provide the IP address via my DHCP servers. I suspect vagrant is doing something similar. To be honest, I didn’t want to do that because I’d like the cluster to start up properly independent of the existence of a DHCP server. But looking at how complicated something so simple turned out to be, I am tempted to change my mind on that one.

malcolmlewis · March 21, 2022, 2:00pm

Hi
All mine are static ip’s.


# Use libvirt for the provider
ENV'VAGRANT_DEFAULT_PROVIDER'] = 'libvirt'


# Configure the nodes
cluster_nodes = 
   { :host => "node-4",
     :type => "master",
     :box => "opensuse/MicroOS.x86_64",
     :ip => "192.168.10.180",
     :cpu => 2,
     :ram => 2048
   },
#   { :host => "node-5",
#     :type => "node",
#     :box => "opensuse/MicroOS.x86_64",
#     :ip => "192.168.10.181",
#     :cpu => 2,
#     :ram => 2048
#   },
#   { :host => "node-6",
#     :type => "node",
#     :box => "opensuse/MicroOS.x86_64",
#     :ip => "192.168.10.182",
#     :cpu => 2,
#     :ram => 2048
#   },
   { :host => "worker-4",
     :type => "worker",
     :box => "opensuse/MicroOS.x86_64",
     :ip => "192.168.10.183",
     :cpu => 1,
     :ram => 1024
   },
   { :host => "worker-5",
     :type => "worker",
     :box => "opensuse/MicroOS.x86_64",
     :ip => "192.168.10.184",
     :cpu => 1,
     :ram => 1024
   },
#   { :host => "worker-6",
#     :type => "worker",
#     :box => "opensuse/MicroOS.x86_64",
#     :ip => "192.168.10.185",
#     :cpu => 1,
#     :ram => 1024
#   },
]


# Configure the domain to use
varDomain = "homelinux.org"


# Bring up the node loop
Vagrant.configure("2") do |config|


   # The 'config' loop for all nodes in the cluster
   config.vm.provision "file", source: "/data/vagrant/MicroOS1/K3s_hosts", destination: "/tmp/k3s_hosts"
   config.vm.synced_folder ".", "/vagrant", disabled: true
   # Setup the timezone
   config.vm.provision "shell", inline: "timedatectl set-timezone America/Chicago"
   # The main cluster node loop
   cluster_nodes.each do |node|


      # The per node 'config' loop
      config.vm.define node:host] do |node_config|


         # Setup the node distribution and network
         node_config.vm.box = node:box]
         node_config.vm.network "public_network", :dev => "eth3", ip: node:ip], :netmask => "255.255.255.0"
         node_config.vm.hostname = "#{node:host]}.#{varDomain}"
         # The node specific configuration loop based on type
         if node:type] == "master"
            node_config.vm.provision "shell", inline: <<-SHELL
            echo "Configure Master..."
            cat /tmp/k3s_hosts >> /etc/hosts
            #curl https://dl.min.io/client/mc/release/linux-amd64/mc -o /usr/local/bin/mc
            #chmod 0755 /usr/local/bin/mc
            NODE_NAME=`hostnamectl --transient`
            NODE_IP=$(grep `hostnamectl --transient` /etc/hosts | awk '{print $1}' | tail -n1)
            curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --cluster-init \
            --node-ip $NODE_IP --node-external-ip $NODE_IP" \
            K3S_TOKEN_FILE=/etc/rancher/k3s/node-token-seed \
            K3S_NODE_NAME=$NODE_NAME sh -s -
            SHELL
         end
         if node:type] == "node"
            node_config.vm.provision "shell", inline: <<-SHELL
            echo "Configure Node..."
            cat /tmp/k3s_hosts >> /etc/hosts
            curl https://dl.min.io/client/mc/release/linux-amd64/mc -o /usr/local/bin/mc
            chmod 0755 /usr/local/bin/mc
            NODE_NAME=`hostnamectl --transient`
            NODE_IP=$(grep `hostnamectl --transient` /etc/hosts | awk '{print $1}' | tail -n1)
            curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --server \
            https://node-4:6443 --node-ip $NODE_IP --node-external-ip $NODE_IP" \
            K3S_TOKEN_FILE=/etc/rancher/k3s/node-token-seed K3S_NODE_NAME=$NODE_NAME sh -
            SHELL
         end
         if node:type] == "worker"
            node_config.vm.provision "shell", inline: <<-SHELL
            echo "Configure Worker..."
            cat /tmp/k3s_hosts >> /etc/hosts
            NODE_NAME=`hostnamectl --transient`
            NODE_IP=$(grep `hostnamectl --transient` /etc/hosts | awk '{print $1}' | tail -n1)
            curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--node-ip $NODE_IP --node-external-ip $NODE_IP" \
            K3S_URL="https://node-4:6443" \
            K3S_TOKEN_FILE=/etc/rancher/k3s/node-token-seed K3S_NODE_NAME=$NODE_NAME sh -
            SHELL
         end


         # The node libvirt provisioning loop
         node_config.vm.provider :libvirt do |libvirt|
            libvirt.cpus = "#{node:cpu]}"
            libvirt.memory = "#{node:ram]}"
            # Add ignition iso to set root password, ssh key, ssh options and node-token-seed
            libvirt.storage :file, :device => :cdrom, :path => '/data/vagrant/MicroOS1/ignition.iso'
         end


      end


   end


end

docbobo · March 22, 2022, 6:29pm

Well, I don’t believe that vagrant is using either ignition or combustion. My question is really about: how to use the supposed mechanisms to provision the instances with a static ip address instead of DHCP…

But thank you anyway!

mrmazda · March 22, 2022, 7:15pm

Static IP can be simple enough, on TW & Leap at least:

# cat /etc/systemd/network/eth0.network
[Match]
Name=eth0

[Network]
Address=192.168.###.###/24
DNS=192.168.###.##X 1.1.1.1 1.0.0.1
Gateway=192.168.###.##X
IPv6AcceptRA=no
LinkLocalAddressing=no
# systemctl status wickedd
● wickedd.service
     Loaded: masked (Reason: Unit wickedd.service is masked.)
     Active: inactive (dead)
# rpm -qa | grep workManag
# systemctl list-unit-files | egrep -i 'solv|ager|net'
dbus-org.freedesktop.network1.service  alias           -
display-manager.service                enabled         enabled
network.service                        alias           -
systemd-network-generator.service      disabled        disabled
systemd-networkd-wait-online.service   masked          disabled
systemd-networkd.service               enabled         disabled
systemd-resolved.service               disabled        disabled
systemd-networkd.socket                enabled         disabled
network-online.target                  static          -
network-pre.target                     static          -
network.target                         static          -
#

malcolmlewis · March 22, 2022, 8:59pm

Hi
No, but vagrant allocates the required ip address for the system and k3s to use, so Proxmox cannot do the same?

I see on one of the nodes;


worker-4:~ # cat /etc/sysconfig/network/ifcfg-eth0
BOOTPROTO='dhcp'
STARTMODE='auto'

worker-4:~ # cat /etc/sysconfig/network/ifcfg-eth1
#VAGRANT-BEGIN
# The contents below are automatically generated by Vagrant. Do not modify.
BOOTPROTO='static'
IPADDR='192.168.10.183'
NETMASK='255.255.255.0'
DEVICE='eth1'
PEERDNS='no'
STARTMODE='auto'
USERCONTROL='no'
#VAGRANT-END

eth0 the vagrant management interface, eth1 for k3s.

malcolmlewis · March 22, 2022, 9:21pm

Hi
So I’m guessing Proxmox is more like SUSE & Rancher Harvester? So there you can use vlans in my brief run of Harvester testing/playing. I just used vlan1 and untagged so it picked up the dhcp allocation for a k3s cluster.

I suggest heading over to join the SUSE & Rancher Community as there are a number of folks using Proxmox, they may have further information? https://community.suse.com