LEAP 15.2 as ethernet router - how to find network bottllenecks?

English Network/Internet
1

I have been using openSuSE for many years as a router with my own IPTables script. Also, I recently got FTTH with a 500 MBit/s download speed. My system is based on an Athlon 5350 AMD processor and I recently updated Leap to 15.2 from 15.1.

This weekend, because my wife and I wanted to ensure that we have internet access if something were to happen to this PC, the hardware is getting somewhat old at this point, we temporarily substituted an Archer TP-Link 1700 in as an Ethernet router.

With my Leap PC as the router, I clearly get 500 MBits/s in Firefox, to the Leap PC itself, when running speed tests. However, on all the other PCs on my network, all Windows 10 PCs, I typically only get 130 MBits/s download.

With the TP-Link 1700 in as router, I ran a speed test because I have known about the reduced speed issue for the PCs I have on my local network. The speed test on the Windows 10 PC I ran it on was 500MBits/s download when the speed on that PC was 130MBits/s with the Leap PC as router.

I am wondering if there is a reference that anyone knows of which presents step-by-step instructions for finding the network bottlenecks and if so, would you please post a reference link? I found this in the openSuSE docs, https://doc.opensuse.org/documentation/leap/tuning/html/book-sle-tuning/cha-tuning-network.html which mentions tools that can be used, but this page, at least, does not give a guide for how to use these tools. I also know that doc says network performance is automatically tuned with the most recent kernels. Still, there has to be a reason that my Leap PC performs worse than the TP-Link router, and I would like to discover what/where that is.

I am considering upgrading the hardware on my Leap PC, however, I do not want to blindly do that, expecting a performance improvement from the router, and then not have that pan out.

I found a page with indicated how to test MTU size, and I found that specifying an MTU of 1442 fractionally decreased the average ping delay; however, according to one other test I ran, there were no errors on the interfaces in the PC to speak of, so I am assuming that in general, the MTU value is likely only a part of the problem.

That said, would someone please recommend a reference that describes, in detail, how to locate the source of network bottlenecks? I do realize that it could be my IPTables Script, or hardware or both, something else, or a combination of things.

As a point of note, I think I will be able to work my way through it although my Linux knowledge, in general, is not at expert level; however, I have over 20-years of programming experience with Windows and never met anything in Linux where if it was adequately explained, I was not able to accomplish it. Aside from the IPTables routing functionality, I have set up my Leap PC with samba, DNS, time, and DHCP servers - all of which are at least functional.

Thanks in advance.

2

@wiyosaya:

Are you attempting to benchmark your network’s performance to the Internet or, between machines on your private LAN?

  • If, between machines connected to your private LAN, then you can use either “netperf
    ” or, “iperf”. - If, to the Internet then, you’ll need a public server on the Internet –
    Currently, AFAICS, there are only public “iperf” servers available on the Internet – <https://iperf.fr/iperf-servers.php>.

Both applications are available from the openSUSE repositories – “iperf” from the main repository and, “netperf” from the non-OSS repository.

3

@dcurtisfra: Thank you for your reply. I think I have reasonably determined that there is a bottleneck, and it appears to be the Leap PC, itself, when compared with the TP-Link router. There is no drop in speed test results across the TP-Link router, however, there is a drop in speed test results across the Leap PC. I am wondering if there is a way to determine what part(s) e.g., hardware/software, of my Leap PC are contributing to the bottleneck - if doing so is even possible. So to put it perhaps more simply, I would like to benchmark/breakdown the network performance between the two ethernet adapters on the Leap PC - one of which is my internet NIC, and the other is a NIC that is connected to my local LAN. So, I would like to understand the internal networking performance of the Leap PC, i.e., the contribution of the networking components in the Leap installation on this PC whether hardware or software or both.

<–Internet -->Leap PC<–local LAN–> where Leap PC, in the diagram, is reasonably the bottleneck; and is what I would like to understand in terms of what/where is the true source of the bottleneck.

Does that help?

Thanks again.

4

Regarding modifying the MTU, that should not be done unless you know somewhere… either your own equipment or your ISP… is using ancient (I mean, really really old like 20+ years). You’d have to be running something like a vintage WinXP or similar for hardware and software to be using those old MTU values.

Probably the most common causes for poor performance are incorrect or outdated device drivers or misonfiguration, ie set to half-duplex, mis-matched network devices like fast ethernet device connected to a gigabit device. Poorly made cables can easily cause signal loss.

When troubleshooting, it probably helps to start with whether your speed is somewhere in the right ballpark (at least 50% of the rated connection). You’ll probably have different strategies looking for the cause of a <very> slow connection vs an only fair performing connection.

If your network connection isn’t wired and running 10 year old hardware, the following article I wrote long ago is still 100% relevant today, it describes the reasons for changing the TCP/IP Congestion Control Algorithm and how to do it, and how to shift resources to your networking sub-system if you’re running today’s hardware… Note that your appliance router may have tiny networking resources allocated but because it’s optimized for its limited hardware work better than your PC with enormous hardware but by default set up to run as though it too has limited hardware.

https://sites.google.com/site/4techsecrets/optimize-and-fix-your-network-connection

You should also run some local tests using your openSUSE as a router without going through the Internet connection. Set up a machine on each side of your openSUSE and do some simple testing.
And, if you want to run tests through your ISP, you can be creative. If you don’t have a public server, then maybe you can run your tests through a different network connection (eg if your phone has a broadband connection). You’ll then need to figure out which connection and ISP might be the bottleneck if you do this.

HTH,
TSU

5

Nice story, but no actual information so far. Any details available for ‘Archer TP-Link 1700’? What is the actual model?

6

Yes, it’s possible – the teams working on embedded real-time systems do it all the time …

  • Have you ever paused to wonder why, the gentle international telecommunications industry has taken 145 years (1876 until now) to achieve the network speeds currently available to this planet’s human beings?

You could attempt to dig out the knowledge in Bell Labs to gain an insight into the theory – you need to start with Claude E. Shannon …

  • Once you’ve understood Shannon’s papers, you can begin to instrument the hardware box and, the microcode and software applications running on that hardware …
  • Then, in addition, you have to apply the knowledge which is currently spread throughout the telecommunications industry … >:)

So, what do you need?

  1. A hardware laboratory with test equipment which probes the pieces of hardware within the system being investigated – the wished for accuracy determines the price of the investment needed …
  2. A team of software developers who write the code which pumps the data streams into the system being investigated and, more code which receives the outgoing data streams and, more code which analyses the transit times – yet another cost versus quality factor …
  3. A stable, accurate clock source – for the time stamps on the data within the streams – for the speeds current today, either with NTP or, satellite clocks or, dedicated hardware …
7

Complete description of a utilized hardware is needed.
Why we need to guess?

8

Thanks for the respectful reply. My LAN is wired, gigibit, and all NICs connected to the LAN are gigabit; however, the Leap PC hardware is not that old (2014), but the CPU is only 4-cores and 4-threads. In addition, speed tests through the Leap PC to the internet give results of 1/5 that of running speed tests on the Leap PC - 130Mbit/s vs 500Mbit/s respectfully. I will see what I might do with the link you posted. It seems easy enough to follow even for a not-so-experienced Linux user like myself.:slight_smile:

Thanks, also, for the respectful reply. So, if I sum up your post, it is possible, however, there are no readily available applications to do so, and even if there were, it would take substantial time to collect and analyze the results. Based on that, I am unable to invest that kind of time right now.

What would you like to know? As I see it, it would be helpful for me if you were to post the items that would be helpful to you, along with any (necessary) commands that might help me gather that information. Otherwise, I would be guessing and, being an inexperienced user, likely to provide more information that tells you nothing, or keeps you guessing.

The model is Archer A7. Are there any other specifics that you would find useful?

9

I presume you mean this one – <https://www.tp-link.com/en/home-networking/wifi-router/archer-a7/&gt;.

  • Gigabit Ethernet connection to your ISP’s Broadband modem.
  • Gigabit Ethernet ports to your LAN.
  • WLAN – 5 GHz: 1300 Mbps (802.11ac) – 2.4 GHz: 450 Mbps (802.11n).

According to “Tom’s Guide”, it’s capable of 647.4 Mbps (WLAN) at a client distance of 5 feet – meaning, the test laboratory managed this throughput to one (5 GHz) WLAN client, presumably with a data stream attached to one of the Ethernet ports …I would suspect that, the port-to-port Gigabit Ethernet throughput approaches 1 Gbps assuming that, only one server and only one client is active …
To be honest, that’s perfectly OK for most private households – at least from my point of view …
[HR][/HR]Back to your original question –

  • The PC hardware most private people buy is, suitable for desktop use – with I/O performance and CPU performance ranging from “office
    ” to, “software development” to, “graphics processing” (artwork / photography) to, “gaming” … - In the professional area, the hardware is often “server
    ” – which is designed for disk I/O and network I/O performance – with some CPU performance for (often database) computing activities. - There is specialised hardware for CPU intensive computing and extreme network I/O performance – usually found as part of a “super computer” …

As a private person, I tend to purchase “enough” CPU and memory to cope with the throughput my current disks are capable of and, the throughput my current LAN is capable of …

  • The throughput to the Internet depends on the contract I’ve made with my ISP – who also supplies my telephone service – they also supply my e-Mail service, amongst other services, such as mobile telephony …

Yes, of course, I also occasionally take note of how much throughput is actually being achieved …

10

Did you try without any iptables rules?

11

Meaning that, you have two Ethernet controllers installed on the PC running openSUSE Leap.

  • The TP-Link router is dedicated hardware with enough I/O bandwidth within the box to pump data-streams from one Ethernet port to another.

Please tell us which Mainboard and, which Ethernet controllers, you are using as an openSUSE Leap router.

12

Thanks for all the replies.

That’s the one. As you noted in your next post, I did only benchmark ethernet. My wife and I use WiFi only minimally. I do understand that it has likely been highly optimized. With all the review sites out there, I doubt it would sell very many if it were not highly optimized. :wink:

Since my first PC, a 386sx-16, I have built my own. In most cases, I tend to emphasize computing/graphics power, and, honestly, networking has been somewhat of an afterthought.

When I built this Leap PC, I am reasonably sure that I only had a 15Mbit/s cable connection. I knew Linux, in general, would run on low-power machines, and the Athlon 5350 seemed ideal for it since its TDP is only 35W. I did, however, add an Adaptec 6405E SATA raid controller. Leap is installed on one of two mirror arrays (two SSDs). The other mirror array (two conventional hard drives) is intended as network storage - but gets very little traffic.

If I do rebuild this PC, I am considering A Ryzen 5 3600 on Gigabyte X570 Aorus Master. X570 AORUS MASTER (rev. 1.1/1.2) Key Features | Motherboard - GIGABYTE U.S.A. although, I will almost certainly turn off its WiFi in the BIOS and use the 2.5G and 1G onboard ethernet controllers in place of what I have now. I will connect the 2.5G ethernet NIC to the fiber ONT, and the 1G ethernet NIC to my LAN.

That’s a great question. I have not done that as I usually do not like to clear the rules for any length of time since it is my gateway PC and that firewall protects my home network (in the past, I have seen obvious signs of someone trying to locate unprotected computers). However, if I have time this weekend, I will try it as it will provide good information for the puzzle, and I can accomplish the task relatively quickly and then reload the firewall/NAT rules.

The mainboard is an Asus AM1M-A. There is no longer a product page for it at least on the US Asus Web site. Here is a Newegg link that is still active - https://www.newegg.com/asus-am1m-a/p/N82E16813132097?Item=N82E16813132097
I use the ethernet controller on the mainboard as my LAN ethernet port. According to the manual, that Ethernet port is built around a Realtek 8111GR PCIe LAN controller.

My other Ethernet controller is connected to the Fiber ONT. It a TP-Link 3468 V2.0 TG-3468 | Gigabit PCI Express Network Adapter | TP-Link and has a Realtek 8168e Ethernet controller on it - https://www.realtek.com/en/products/communications-network-ics/item/rtl8168e

I realize the mainboard and the ethernet controllers are not what might be called “server grade” however, they have served their purpose well. The much fatter pipe with the Fiber ISP, as I discovered only recently, has exposed what might be a lack of hardware power and/or tuning for the task.

Also, the Fiber ISP is the only internet capability I have right now. I dropped the only cable provider in our area when I had the fiber installed, and the DSL from our local phone company was only delivering about 2Mbits/s when we tried them out a few years back - apparently, our house is 30,000 ft from the phone company switch and, as I understand it, that distance kills performance for DSL

Thanks again.

13

One last note, however, I am not sure this is an important detail - in testing my internet speed with the Leap PC as the router, I have always gotten, as part of the results from the speed tests at https://www.dslreports.com/ that there is “buffer bloat”. I did not get that same message when the Archer A7 was in as the router.

I have tried to search for information relating to “Buffer Bloat” in the past, however, I have been unable to find anything meaningful relating to the message in general, or how to solve it. As such, I gave up on trying to do anything about it.

14

AMD Athlon 5350 – Desktop Kabini APU – AM1 socket – 4 CPU cores @ 2.05 GHz – integrated R3 (HD 8400) GPU – integrated (PCIe) Gigabit Ethernet – <SUSE Paste.

  • Is your 2nd Ethernet port a WiFi card?
  • If WiFi/WLAN, is it 2.4 GHz only or, does it support 5 GHz?

Searching the net for a throughput comparison will reveal quite a few suggestions but, this one has possibly the most relevant throughput information – <What is the actual real-life speed of wireless networks ? :: SG FAQ.
[HR][/HR]BTW, the Ethernet data path for your CPU is – Ethernet port on the Mainboard → AM1 socket → Kabini APU → AM1 socket → a PCIe slot → 2nd Ethernet controller or WiFi/WLAN card …

15

We’re crossing paths while writing … >:)

It seems that, the major difference between the dedicated router and the AMD Athlon APU box is, the Ethernet path between the Ethernet ports …

  • I was getting all fired up to show the difference between WiFi/WLAN throughput → 2.4 GHz versus 5 GHz – but that seems to be not relevant …

If you examine the Ethernet path in the Kabini system diagram, you’ll see that, the packets have to routed between PCIe slots via the AM1 APU socket –

  • I suspect that, the PCIe to PCIe data stream via the AM1 APU socket is being throttled by the hardware.
16

https://www.bufferbloat.net/projects/

Another case: Jumbo frame - Wikipedia

17

Setup for Jumbo Frames as follows: <https://wiki.archlinux.org/index.php/Jumbo_frames&gt;
[HR][/HR]This works around the “quantum issue” of the UNIX®/Linux kernel scheduler – ‘/proc/sys/kernel/sched_rr_timeslice_ms’ –


 > cat /proc/sys/kernel/sched_rr_timeslice_ms
100
 >

In other words –

Don’t attempt to perform I/O faster than “RR_TIMESLICE” …

  • Always attempt to fill buffers such that, at each scheduler time slice of 100 ms, enough data is transferred …
18

A neighboring inn had 0.15 Mbit/s with copper wire. That improved with FTTB to 50 Mbit/s, the smallest transmission rate available with their ISP. My connection is now 25 Mbit/s with fiber compared to 9 Mbit/s with copper wire. What matters even more is the lower latency with FTTB. My router is an ancient FRITZ!Box 7360, ethernet is Intel I219-V. Effective speed here is limited by transmission rates guaranteed by contract. If you don’t get that with your machine try a faster one running LEAP from a live stick. A dedicated router works best: “The Wireless router usually incorporates a Wireless access point, Ethernet switch, DSL modem and Router in a single box.” How to Set Up a Home Network- Beginners Guide KISS works best. This holds for home networks too.

19

My current DSL copper connection – only 17 km (downhill) from where you are – AVM FRITZ!Box 7490 data …

IPv6 – Receive: 55.0 Mbit/s – Transmit 11.1 Mbit/s
IPv4 (DS-Lite-Tunnel) – Receive: 55.0 Mbit/s – Transmit 11.1 Mbit/s
Plus VoIP telephony with ISDN extensions …

Negotiated DSL connection properties –

Receive Transmit
DSLAM data rate max. Mbit/s 116.8 32
DSLAM data rate min. Mbit/s 115.2 –
Cable capacity Mbit/s 141.131 36.088
Actual data rate Mbit/s 116.797 31.999
Min. effective data rate Mbit/s 114.088 31.995
Seamless rate adaption off off
Latency fast fast
Signal-to-noise ratio dB 13 12
20

Apologies for my belated responses. I feel like I have too many irons in the fire right now, and some of them are melting. lol!

Thanks for all the posts. I will try your suggestions with Jumbo Frames when I get the chance.

I dumped all my IP Tables rules this weekend, and replaced them briefly on the input, output, and forward chains with -P ACCEPT. It made no difference.

I then re-ran my rules script, and ran the speed test again. However, I noticed that on several PCs that had poor speed test performance, the speed test results were now the same as if I had run them on my Leap PC, that is, there was no speed degradation where previously there was, and the warnings about Buffer Bloat were gone. I am rather confused at that. The ISP’s network is new in our area, so perhaps the ISP did some tuning? To me, it does not seem likely that the ISP is doing something in the background that affects performance from my router to my LAN - why would they want to throttle NAT’d traffic that comes from one of their IP addresses - assuming they can or were doing something like that? I have changed nothing in my configuration.

The only PC that still has slow network performance on the Speed Test is another AMD APU (A10-7850) running Windows 10 on an ASRock FM2A88X Extreme6+ based PC, and using the Motherboard’s Ethernet port to connect to my LAN (Not that I am expecting help with that). I use that PC as an HTPC, and the speed is more than fast enough for everything I do right now.