Network sometimes become very slow

After recent kernel upgrades wire network sometimes become very slow after few hours of work ~ 50KB/sec while my Ethernet is 100Mb/sec. This happens regardless network management used: wicked or NetworkManager. PC restart does not help - only full power-off/on. What it could be and how to solve?

Definitely not normal behaviour. I would start by analysing your network traffic and related processes. Install nethogs and/or iftop to examine the traffic.

Identify the traffic with

iftop

and then locate the associated processes eg

ss -tup

When to start these? Now on when problem appears again? I would think kernel or associated processes set some wrong flags on Ethernet card driver settings to get this if only power off helps.

Nethogs (common situation):

NetHogs version 0.8.5

    PID USER     PROGRAM                                                                                                           DEV        SENT      RECEIVED
   2063 user  /usr/lib64/waterfox-classic/waterfox                                                                              enp6s7      0.064       0.035 KB/sec
  10573 user  /tmp/.mount_GkohFF/QGroundControl                                                                                 enp6s7      0.000       0.000 KB/sec
      ? root     192.168.0.102:60322-140.82.118.4:80                                                                                           0.000       0.000 KB/sec
   2634 user  ..sr/share/riot/riot-web --type=utility --field-trial-handle=13542205136582643125,2488800000229244219,131072 --d  enp6s7      0.000       0.000 KB/sec
   1938 user  /usr/bin/qbittorrent                                                                                              enp6s7      0.000       0.000 KB/sec
      ? root     unknown TCP                                                                                                                   0.000       0.000 KB/sec

  TOTAL                                                                                                                                        0.064       0.035 KB/sec

Yes, monitor live until the bandwidth issue becomes apparent. We can only speculate here without further information/logs/output etc. I note that you have qbittorrent running - perhaps that is a potential culprit here? (Just wondering if rate limiting needs to be applied?)

Network hardware details?

/usr/sbin/hwinfo --netcard

For a wired connection, check that the negotiation rate is as expected and not flapping. For example, assuming eth0

/usr/sbin/ethtool eth0

To watch live in a terminal window…

watch /usr/sbin/ethtool eth0

or filtered for speed and duplex

 watch '/usr/sbin/ethtool eth0 |egrep -i "speed|duplex"'

…and observe if any changes.

No, not applied. Current settings:

# /usr/sbin/hwinfo --netcard
20: PCI 607.0: 0200 Ethernet controller                         
  [Created at pci.386]
  Unique ID: VuTY.IQxIdIhhuH7
  Parent ID: qscc.UsCA9N2eQqB
  SysFS ID: /devices/pci0000:00/0000:00:14.4/0000:06:07.0
  SysFS BusID: 0000:06:07.0
  Hardware Class: network
  Model: "Realtek RTL-8100/8101L/8139 PCI Fast Ethernet Adapter"
  Vendor: pci 0x10ec "Realtek Semiconductor Co., Ltd."
  Device: pci 0x8139 "RTL-8100/8101L/8139 PCI Fast Ethernet Adapter"
  SubVendor: pci 0x10ec "Realtek Semiconductor Co., Ltd."
  SubDevice: pci 0x8139 "RTL-8100/8101L/8139 PCI Fast Ethernet Adapter"
  Revision: 0x10
  Driver: "8139too"
  Driver Modules: "8139too"
  Device File: enp6s7
  I/O Ports: 0xb000-0xbfff (rw)
  Memory Range: 0xfe120000-0xfe1200ff (rw,non-prefetchable)
  Memory Range: 0xfe100000-0xfe11ffff (ro,non-prefetchable,disabled)
  IRQ: 21 (984826 events)
  HW Address: 60:e3:27:05:0f:23
  Permanent HW Address: 60:e3:27:05:0f:23
  Link detected: yes
  Module Alias: "pci:v000010ECd00008139sv000010ECsd00008139bc02sc00i00"
  Driver Info #0:
    Driver Status: 8139too is active
    Driver Activation Cmd: "modprobe 8139too"
  Driver Info #1:
    Driver Status: 8139cp is active
    Driver Activation Cmd: "modprobe 8139cp"
  Config Status: cfg=no, avail=yes, need=no, active=unknown
  Attached to: #28 (PCI bridge)

26: PCI 500.0: 0200 Ethernet controller
  [Created at pci.386]
  Unique ID: D2Iq.BmqBU9Z2gDD
  Parent ID: WL76.sQx2zCXa1Z7
  SysFS ID: /devices/pci0000:00/0000:00:09.0/0000:05:00.0
  SysFS BusID: 0000:05:00.0
  Hardware Class: network
  Model: "Gigabyte Onboard Ethernet"
  Vendor: pci 0x10ec "Realtek Semiconductor Co., Ltd."
  Device: pci 0x8168 "RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller"
  SubVendor: pci 0x1458 "Gigabyte Technology Co., Ltd"
  SubDevice: pci 0xe000 "Onboard Ethernet"
  Revision: 0x0c
  Driver: "r8169"
  Driver Modules: "r8169"
  Device File: enp5s0
  I/O Ports: 0xc000-0xcfff (rw)
  Memory Range: 0xfe200000-0xfe200fff (rw,non-prefetchable)
  Memory Range: 0xda100000-0xda103fff (ro,non-prefetchable)
  IRQ: 36 (no events)
  HW Address: e0:d5:5e:39:4b:c3
  Permanent HW Address: e0:d5:5e:39:4b:c3
  Link detected: no
  Module Alias: "pci:v000010ECd00008168sv00001458sd0000E000bc02sc00i00"
  Driver Info #0:
    Driver Status: r8169 is active
    Driver Activation Cmd: "modprobe r8169"
  Config Status: cfg=no, avail=yes, need=no, active=unknown
  Attached to: #29 (PCI bridge)
# /usr/sbin/ethtool enp6s7
Settings for enp6s7:
        Supported ports:  TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 100Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 32
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: pumbg
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

**watch /usr/sbin/ethtool enp6s7:
**

Every 2,0s: /usr/sbin/ethtool enp6s7                                                                                           Gigabyte-Linux: Sun Feb  2 01:12:01 2020

Settings for enp6s7:
        Supported ports:  TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 100Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 32
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: pumbg
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

…and observe if any changes.[/QUOTE]

With egrep returns nothing - wrong parameters? I run:

 watch /usr/sbin/ethtool enp6s7 | egrep -i "Speed|Duplex"

I see, correct one:

watch '/usr/sbin/ethtool enp6s7 | egrep -i “speed|duplex”'

Every 2,0s: /usr/sbin/ethtool enp6s7 | egrep -i "speed|duplex" 

        Speed: 100Mb/s
        Duplex: Full

Running…

That’s because it needs to be encapsulated like this

 watch '/usr/sbin/ethtool enp6s7 | egrep -i "Speed|Duplex"'

Yes, but you don’t need to report the output, just observe for now. I don’t expect it to change ordinarily, but occasionally one can get flapping due to connection issues (usually hardware related).

One not uncommon cause is if you run any applications that spawn a very large number of network connections…
Like a torrent client, or running a ftp server (even as a workstation) under heavy load.

In an article I published long ago on a much earlier version of openSUSE, everything in it applies just as much today.
I describe a particular pattern where your system will gradually slow over a few minutes, come to a nearly complete stop (as you describe only kb/sec), suddenly resolves and then gradually slows again in a repeating cycle.

That situation is caused by

  • Inadequate resources allocated to networking because of the running torrent application
  • The default TCP/IP Congestion Control Algorithm installed in all Linux works best with 10/100 Fast Ethernet wired connections. Most people today connect using WiFi and may be further complicated by unusual wireless connections like satellite or phone carrier, or might even have Gigabit connections, and others might be effected by EMF.

I describe the situation,
How to diagnose,
And solutions

https://sites.google.com/site/4techsecrets/optimize-and-fix-your-network-connection

I also endorse the tools deano recommends.

TSU

If it was so, then killing the app or restarting PC would help. However only power off helps

As I said I have 10/100 Fast Ethernet wired connection - not WiFi. An it suddenly slows down. Other WiFi devices in my network at the same time still show high connection speed, so it is not the router problem as well.

Hi
Perhaps look at using atop and netatop, but as indicated I do wonder about your torrent feed, perhaps it is seeding/uploading?

I describe in detail because sometimes people don’t know what to look for…
For instance, they may not realize that running a torrent app can be very different than running other “normal” apps.
Or, they might notice the network slowing and not realize if they wait a few <minutes> longer that the situation might clear up only to repeat a cycle of slowing.
The above can be addressed by shifting system resources to networking as described in my article.
Even if you’re connecting with 10/100 Fast Ethernet, if there is anything that might cause congestion… other machines on your network, close to EMF sources, even network congestion beyond your Internet Gateway, depending on the suspected cause a TCP/IP Congestion Control algorithm might make some difference although it won’t eliminate a problem completely.

If other Hosts in your LAN aren’t experiencing similar issues more or less at the same time as your machine, it does suggest the problem is local to your machine. Depending on how your router or switch or hub that services your group of Hosts works, and any possible isolation or QoS, that <may> suggest you’re not affected by other Hosts… or not.

TSU

Yes, most time is seeding/uploading.

A hypothesis:

  • Failing network interface hardware (probably the data buffer DRAM) starts to introduce errors after a while, necessitating either retransmission or data rate reduction. The network chipset is reset following a power-off, but not by rebooting the operating system.
  • This could be either on the computer or at the other end of the cable
  • The kernel upgrades are most likely coincidental, but there is a possibility that kernel-firmware has changed the behaviour of the network hardware.

Suggestions:

  • Use “ifconfig” (from net-tools-deprecated package) to check TX and RX characteristics both when the system is running well and when slow.
  • If possible plug the other end of the cable into anther port on the switch or router, etc.
  • Try another net-card – a usb ethernet dongle would probably be easiest.
  • You might be able to try an older version of kernel-firmware from a Leap repository.

Please describe your network hardware (sudo hwinfo --netcard). Is it integrated or removable?

OK,
Then as I hypothesized,
This is likely your problem, scarcity of network resources.
To understand this problem enough to diagnose, you have to understand how a torrent application works…
The critical thing to know is that for however many other nodes you may be connected to, the torrent app will allocate a few kilobytes of resources (memory primarily) and use a network socket, possibly even assign a unique port.
An ordinary Linux installation doesn’t assume that these kinds of apps will be run, it’d be more common to run “ordinary” network apps that only make a few network connections, for instance a hundred or so connections on a web page that aggregates content like www.cnn.com, and assumes most people want more system resources for running apps locally on your machine. But a torrent app might be configured to connect to thousands, maybe multiple tens of thousands of other nodes even if you only configure less than a dozen active upload/download connections. Every one of those thousands needs some resources and it adds up.

My article describes how to resolve network sockets resource starvation starting from this page

But I do recommend you read the article from the beginning to understand what you are doing and the various tools and configs at your disposal. This is one topic that can be important… Primarily when your network connection isn’t wired Fast Ethernet or if you’re running any apps that makes enormously high numbers of network connections.

TSU

Not hardware problem: monitor always show 100 MB/s Full-duplex. Seems some NetworkManager bug reported here many times. I do not run torrent or something like this now. Issue starts after network cards switches in NM: for example between Ethernet and USB Ethernet (modem), or switch Network connection on/off many times in network tray applet. After this Ethernet could become VERY slow.

Any ideas?

Answering:

1) Slow now

# ifconfig

enp6s7: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.105  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::3f26:159:2ecc:f03d  prefixlen 64  scopeid 0x20<link>
        ether 60:e3:27:05:0f:23  txqueuelen 1000  (Ethernet)
        RX packets 745955  bytes 621386743 (592.6 MiB)
        RX errors 18419  dropped 93  overruns 1  frame 0
        TX packets 533097  bytes 174678544 (166.5 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

2) Plug the other end of the cable into anther port on the switch or router
Does not help

3) Switching to another card (integrated)
Helps :


# ifconfig
enp5s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.110  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::3b8:ff02:adb5:ae0c  prefixlen 64  scopeid 0x20<link>
        ether e0:d5:5e:39:4b:c3  txqueuelen 1000  (Ethernet)
        RX packets 116326  bytes 173739670 (165.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 60607  bytes 3517045 (3.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

4) Other firmware
Have not tried

5) My network hardware (sudo hwinfo --netcard). Is it integrated or removable?
Problem was with removable Realtek PCI network.

# hwinfo --netcard
20: PCI 607.0: 0200 Ethernet controller                         
  [Created at pci.386]
  Unique ID: VuTY.IQxIdIhhuH7
  Parent ID: qscc.UsCA9N2eQqB
  SysFS ID: /devices/pci0000:00/0000:00:14.4/0000:06:07.0
  SysFS BusID: 0000:06:07.0
  Hardware Class: network
  Model: "Realtek RTL-8100/8101L/8139 PCI Fast Ethernet Adapter"
  Vendor: pci 0x10ec "Realtek Semiconductor Co., Ltd."
  Device: pci 0x8139 "RTL-8100/8101L/8139 PCI Fast Ethernet Adapter"
  SubVendor: pci 0x10ec "Realtek Semiconductor Co., Ltd."
  SubDevice: pci 0x8139 "RTL-8100/8101L/8139 PCI Fast Ethernet Adapter"
  Revision: 0x10
  Driver: "8139too"
  Driver Modules: "8139too"
  Device File: enp6s7
  I/O Ports: 0xb000-0xbfff (rw)
  Memory Range: 0xfe120000-0xfe1200ff (rw,non-prefetchable)
  Memory Range: 0xfe100000-0xfe11ffff (ro,non-prefetchable,disabled)
  IRQ: 21 (1275426 events)
  HW Address: 60:e3:27:05:0f:23
  Permanent HW Address: 60:e3:27:05:0f:23
  Link detected: no
  Module Alias: "pci:v000010ECd00008139sv000010ECsd00008139bc02sc00i00"
  Driver Info #0:
    Driver Status: 8139too is active
    Driver Activation Cmd: "modprobe 8139too"
  Driver Info #1:
    Driver Status: 8139cp is active
    Driver Activation Cmd: "modprobe 8139cp"
  Config Status: cfg=no, avail=yes, need=no, active=unknown
  Attached to: #28 (PCI bridge)

26: PCI 500.0: 0200 Ethernet controller
  [Created at pci.386]
  Unique ID: D2Iq.BmqBU9Z2gDD
  Parent ID: WL76.sQx2zCXa1Z7
  SysFS ID: /devices/pci0000:00/0000:00:09.0/0000:05:00.0
  SysFS BusID: 0000:05:00.0
  Hardware Class: network
  Model: "Gigabyte Onboard Ethernet"
  Vendor: pci 0x10ec "Realtek Semiconductor Co., Ltd."
  Device: pci 0x8168 "RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller"
  SubVendor: pci 0x1458 "Gigabyte Technology Co., Ltd"
  SubDevice: pci 0xe000 "Onboard Ethernet"
  Revision: 0x0c
  Driver: "r8169"
  Driver Modules: "r8169"
  Device File: enp5s0
  I/O Ports: 0xc000-0xcfff (rw)
  Memory Range: 0xfe200000-0xfe200fff (rw,non-prefetchable)
  Memory Range: 0xda100000-0xda103fff (ro,non-prefetchable)
  IRQ: 36 (no events)
  HW Address: e0:d5:5e:39:4b:c3
  Permanent HW Address: e0:d5:5e:39:4b:c3
  Link detected: yes
  Module Alias: "pci:v000010ECd00008168sv00001458sd0000E000bc02sc00i00"
  Driver Info #0:
    Driver Status: r8169 is active
    Driver Activation Cmd: "modprobe r8169"
  Config Status: cfg=no, avail=yes, need=no, active=unknown
  Attached to: #29 (PCI bridge)

Will try to use onboard for now and check if problem will repeat. Happen today after PCI Ethernet -> USB modem (Ethernet emulation) switch and then back to PCI Ehternet. Did not have this problem with wicked however it is not so convenient as Network Manager.

Thanks, eng-int, switching from PCI Realtek to Gigabyte onboard seems works. So is it hardware problem and time to change PCI Ethernet or still could be NetworkManager? Is there is a way to reset Ethernet somehow without power-off? Thanks a lot!!!

Last summer my ADSL 2+ connection became slow and unreliable. It turned out to be a common hardware problem. I cleaned and re-plugged all connectors involved. The connection was reliable and speed close to technical limits again. This happened every few years since more than two decades.

This <bad word> happened again with onboard card now. Nothing criminal here:

# ifconfig
enp5s0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether e0:d5:5e:39:4b:c3  txqueuelen 1000  (Ethernet)
        RX packets 513900  bytes 562417644 (536.3 MiB)
        RX errors 0  dropped 1  overruns 0  frame 0
        TX packets 327703  bytes 96401316 (91.9 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp6s7: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.105  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::3f26:159:2ecc:f03d  prefixlen 64  scopeid 0x20<link>
        ether 60:e3:27:05:0f:23  txqueuelen 1000  (Ethernet)
        RX packets 207633  bytes 216860284 (206.8 MiB)
        RX errors 1  dropped 5  overruns 1  frame 0
        TX packets 203822  bytes 197170477 (188.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 1040  bytes 140004 (136.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1040  bytes 140004 (136.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Switched wire back to removable Ethernet PCI card - normal speed again. So think problem completely in this <bad word> NetworkManager, switching back to wicked. When this <bad word> NM will be fixed?! Bugs for years. Write bugreport to them?