Greetings,
I am not an experienced Suse or Linux user but I jump in to this trying to achieve a couple of servers running a Host > Docker and some containers with software like Nextcloud, Traefik, a firewall and a web monitoring called Matomo.
Most of these have Docker images readily available and I have spent a few hours a week for 3-4 months trying to get this to work.
My hardware are two identical Proliant DL 380 G7 servers with 16 2’5" drives each, all installed getting up to about 7TB minus a system disk raid 1… so here is what happened:
Due to the NIC being a Broadcom chip, I was unable to make it work with Debian as a host. Yes, I tried the NonFreeSoftware iso, but it still wouldnt work, I had the drivers, the HP discs, and I was still unable to get it to work.
Trying ESXi, VMWare have deprecated the CPU’s and despite using the command option AllowLegacyCPU at the setup, it still wouldn’t work.
Testing out Fedora Server 32 the installation went fine and currently running that as host with Webmin as GUI, but there is no updated Docker image for that version of Fedora
I could write more about what I have tried and failed, despite reading and asking in several different places, but those above made me start over, well i haven’t replaced Fedora yet, but that is why I am here.
So I consider alternatives and recalled Suse from back in the days when I was first testing out Linux and always ended up using a OpenSuse laptop for a while, messing around a bit, nothing serious. Up until now I have always used Windows but for this setup I just don’t have the correct licensing.
My questions are:
Do SUSE support Broadcom NIC BCM5709C out of the box or will I have similar issues as I had with Debian?
I see that OpenSuse has updated Docker images but I still have to select what “flavour” of OpenSuse would be adequate as a server Host for Docker. Advice? Should I perhaps gor for Suse Enterprise Server, is that even possible without getting a support contract?
Hi,
This forum is as good as any for your first questions…
Regarding Broadcom network support…
Yeah, Broadcom has a weird semi-proprietary licensing policy which means it’s not pure open source and publicly licensed… Which in turn means problems getting anything accepted by the main Linux kernel. It’ll also mean that you’ll likely find official Broadcom support rare in any distro, but most everyone will have some kind of unofficial or voluntary support.
We do have Users who are Broadcom experts and have built special drivers for openSUSE users.
Looking up your NIC, I don’t see that anyone has asked about your specific model before in the Wireless forum, but I also see that your NIC isn’t something extremely recent so there is a fairly good chance that it’s supported.
Read the Broadcom “sticky” at the top of the Forum posts and do what you can. If you run into problems, post in that forum.
As for a recommended Docker platform,
If you want reliability and least problems, LEAP is recommended (current version 15.2) because updates are incremental and relatively few compared to Tumbleweed.
If reliability isn’t a concern as much as you might want to be using the leading and bleeding edge latest of everything, then Tumbleweed can be considered. Just remember though that the more changes that happen regularly, the greater the chance a change may affect your use. Typically Tumbleweed is armed with plenty of ways to address nearly any problem from snapshot rollbacks to its OpenQA testing to lessen the occurrence of problems, but you will just need to be more prepared and be ready to do whatever is necessary.
Regarding other topics in your post…
As would likely be expected, Docker will be almost if not exactly identical to how you have already learned to setup, manage and maintain. If you have any questions about that, you can post in the Virtualization forum.
I don’t know how you’re setting up VMware ESXi, that is a HostOS product. If you’re talking about ESX, Workstation or Player, those should all install or run with openSUSE without any problem. If you’re talking about running openSUSE as Guests, I don’t know of any problem on any VMware product. At the moment, there is a VMware issue with kernel 5.8 which would be an issue installing on Tumbleweed (no problem on LEAP), but that’s expected to be addressed any day now.
As for SLES (SUSE Enterprise Server), that’s a licensed product. It’s not ree unless you’re running a trial copy.
openSUSE is not exactly the same as SLES and setups are often different, but you can accomplish a great many things… Even highly advanced and leading technologically on openSUSE.
You might find useful my slidedeck on openSUSE… It describes pre-installation choices, the installation process (including some options you can change) and some recommended first steps after install. You also find a similar guide where you download your DVD.
According to the HP specs, these servers have SUSE LINUX Enterprise Server 11 and SUSE LINUX Enterprise Server 10 support.
The current SLES version is 15 – meaning, these servers aren’t exactly new …
Back then, SLES may well have had BCM5709C Ethernet drivers – searching the Broadcom site doesn’t reveal any information on these Ethernet (cable) devices …
No idea – install a scratch drive and try to install Leap 15.2 – if the installation supports that Ethernet (cable) device then, all is well …
Hi
The QLogic BCM5706/5708/5709/5716 device are supported with the bnx2 module and associated firmware files, really need to see the PCI ID’s of the device;
/sbin/lspci -nnk | egrep -A3 "Network|Ethernet"
The other option is to boot from a Live USB device and check.
Whenever you install on “enterprise” hardware and the dl380 can be considered such,
One of the first things that should be done is to investigate how to install software and drivers from the manufacturer.
HP/Compaq has changed its means of distributing its “setup package” over the years and I haven’t had to look this up for several years.
But, I think the following is what you need to download and install. In any case, you should get on the HP support forums and ask for exactly what you need (In previous years, there was something called a Softpaq, and there were downloadable DVDs).
When you install the above, hopefully that tool should be able to scan your machine for its components and then automatically create a list of what you need to download and install. I don’t know if this machine is still supported well by HP, but you should install anything you can.
An alternative is to do what HP consumer products do, or browse through the business sections of HP’s website… enter your machine’s serial number and if it’s recognized, you’ll be taken to a page that lists software and drivers to download.
Keep in mind, if necessary yo can install a $10 NIC temporarily to get network access if needed, I usually keep a couple of old NICs that’ll slide into PCI slots for these kinds of situations.
I thank you all for your replies, valuable advice all of it and I am impressed by the level of answers and commitment. :good:
Will start by answering that thing about Brodacom Malcolm posted - and thanks for the command, it helped A LOT, :
This is the card details. 4 ports.
03:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries NetXtreme II BCM5709 Gigabit Ethernet [14e4:1639] (rev 20) Subsystem: Hewlett-Packard Company NC382i Integrated Multi-port PCI Express Gigabit Server Adapter [103c:7055]
Kernel driver in use: bnx2
Kernel modules: bnx2
03:00.1 Ethernet controller [0200]: Broadcom Inc. and subsidiaries NetXtreme II BCM5709 Gigabit Ethernet [14e4:1639] (rev 20)
Subsystem: Hewlett-Packard Company NC382i Integrated Multi-port PCI Express Gigabit Server Adapter [103c:7055]
Kernel driver in use: bnx2
Kernel modules: bnx2
04:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries NetXtreme II BCM5709 Gigabit Ethernet [14e4:1639] (rev 20)
Subsystem: Hewlett-Packard Company NC382i Integrated Multi-port PCI Express Gigabit Server Adapter [103c:7055]
Kernel driver in use: bnx2
Kernel modules: bnx2
04:00.1 Ethernet controller [0200]: Broadcom Inc. and subsidiaries NetXtreme II BCM5709 Gigabit Ethernet [14e4:1639] (rev 20)
Subsystem: Hewlett-Packard Company NC382i Integrated Multi-port PCI Express Gigabit Server Adapter [103c:7055]
Kernel driver in use: bnx2
Kernel modules: bnx2
Now, looking at other posts…
@tsu2@dcurtisfra
Yes, indeed those are Enterprise, actually got them from work as they virtualized them. I have everything HPE related covered, got the ISO’s and have updated all the FW to latest available, so that is, to me, a no brainer, if you get some hardware you better have a clue how to handle it and update it, even over Windows… . The machines are up to 10 years old after all but still run very well, probably thanks to SAS drives, which will be replaced eventually for SSD’s. The CPU’s can also be upgraded from current 2,6GHz to 3,4GHz for about 100€ per server. Having them in my garage at home raises some awareness about power consumption, but so far the average is about 250w per day per server.
As you can see in pictures below, these servers are at the bottom, the 4U in the middle is an old 2xOpteron that will eventually be decommissioned (I run my 10TB file share on it) and the top box with the Corsair logo is a recent Xeon build I intended to use for FreeNAS but replaced that with ESXi instead, currently running ESXi > Ubuntu > Docker > Ubiquiti UNMS for my network on that machine.
There is room for more once I clean out some odd items…
As for OpenSuse I am already running Leap on a laptop to get acquainted. I think I remember when YAST was the update package manager, now a lot of apps are called YAST-something… confusing, but I know what to look for. I will go for Leap on the servers if the Enterprise versions updates are locked behind licensing or support agreements.
I would do the installation and setup the same way I run Fedora, install the OS on the servers and use Webmin for access. That circumvents any Remote Connection woes, I have learned to appreciate RDP in Windows, but don’t trust VNC. With Webmin I have TOTP/TFA authentication on one account so I can access it from anywhere, so I hope Webmin will run smoothly on Leap. This would also make me skip ESXi, which is the Host platform that is free for usage, unlike VCenter. Webmin has turned out to be a wet dream come true, so far. Now I look for 2FA for everything. If it isn’t available I am likely to skip that application.
As for Docker I can only hope it will run better and smoother. I have a list of applications I want to run there, Traefik and OPNSense among others, but I need to sort out some cabling before messing with Firewall and such, since those conns are not correctly setup yet.
Again, I appreciate the answers, please ask and answer more and thanks a lot to @tsu2 for the initial kind words and your slides
You will see me around with such an excellent start…rotfl!
Thanks Malcolm, going to hit the sever with Leaf within a few days.
I know about Cockpit, it came default with Fedora Server, non UI version, but it was pretty limited. Sure, it has a terminal, but as I may have hinted, I much prefer an interface to work with and Webmin is the best I have found so far. Sure, Cockpit will probably evolve, but Webmin has been around for a couple of decades and is still being updated so that is really the kind of effort I appreciate and endorse.
While browsing, I noticed that, you may have to, in addition, install the package “iscsiuio” –
Linux Broadcom NetXtremem II iscsi server
This tool is to be used in conjunction with the Broadcom NetXtreme II Linux driver (Kernel module name: “bnx2” and “bnx2x”), Broadcom CNIC driver, and the Broadcom iSCSI driver (Kernel module name: “bnx2i”). This user-space tool is used in conjunction with the following Broadcom Network Controllers: * bnx2: BCM5706, BCM5708, BCM5709 devices * bnx2x: BCM57710, BCM57711, BCM57711E, BCM57712, BCM57712E, BCM57800, BCM57810, BCM57840 devices This utility will provide the ARP and DHCP functionality for the iSCSI offload. The communication to the driver is done via user-space I/O (Kernel module name “uio”).
Please note that, the Leap 15.2 default Kernel also includes the bnx2 and bnx2x firmware: “/lib/modules/5.3.18-lp152.41-default/kernel/drivers/net/ethernet/broadcom/”
Hi
That’s part of the distribution (should get installed by default) uio module is there as well, there is also the YaST iSCSI Client Configuration module available. All the tools needed are there AFAIK
No, not as such, I have seen other tools to customize images https://build.opensuse.org/ but now I see it links to the same…
hmm…
But using Leaf I get three options out of which 2 may be relevant, but still ain’t…
JeOS is not relevant, not a VM, Kiwi, to be honest no clue. Leap is supposed to be the host so how can I use a docker image as host?
I think I should make a full installation of Leap and select configuration while installing it. At least it makes sense to me considering the three options in Build?
I had planned to ask this separately after reading up on it a bit more, but since you point…
On containers… a bolder move is to install TW with transaction updates. Then you’ll have podman v2 in repositories. Podman is Docker without a daemon, leading to a better experience alongside systemd, which might be useful for a server. It also can run completely rootless, although it doesn’t seem to be something you’ll need. Leap has podman v1, which I didn’t use so can’t advise.
Bolder? Is Podman anything like Portainer? Cause I tried that one but as usual I ended up in a small jungle of issues that lead me to scrap it for not getting it to run/work.
From what I understand TW seems to be more of a developer platform, while Leaf is more long term stable, and choosing between them I would probably go for Leaf, but considering I have nothing that works except Webmin in my intended setup, one may be as good as the other…
No, podman is a full replacement for docker, and it implements a very similar command line interface. podman was forked from the original docker a while ago. While similar, there’s a couple differences that really makes it stand out. When you call docker, you’re talking to a client program, which talks to a docker server listening to a socket, which only then runs the container. The server runs as root. That’s an effortless privilege escalation path for taking control of the machine. When you call podman, it forks itself into the process that runs the containerized environment. For a (desktop) development environment this is much better (no root involved), for a server, systemd can then manage the container life-cycle, capture logs, and so on. podman also can run as root, either logged in as root or using sudo.
YSK that both docker/podman would disable quotas on your btrfs filesystem (long-time bug), so don’t use the btrfs driver. Choose the overlay (non-default when running over btrfs) or fuse-overlay (default for rootless) driver. I heard there’s a way to avoid that bug but I haven’t got time to investigate.
Yeah, you got that right. That’s what I meant with bolder. Most of the time we really want a Leap experience, but I mentioned TW because you’ll get the latest podman & co. TW tends to break every new major kernel when there’s out-of-tree kernel modules, like NVIDIA/VirtualBox/VMWare. When that happens folks either rollback to a previous snapshots or boot a previous kernel. Both workarounds are features provided by openSUSE.
I know there may be others, but those three aren’t relevant fro my setup, so … no wait … I am not sure I know what I am talking about, I am assuming hardware and or hypervisor stuff and in that context none are relevant for me. Having TW break every now and then does not sound appealing, even if fixable.
Going to browse around for setup info… Need to have a clue what boxes to tick
