nload display shows wget --limit-rate=10K is not really limit the rate to average 10K/s, is sometimes 1M/s, and other time is closer to zero. I want 10K/s or mroe or less during downloading at most of time, how can I do that?
Big or small files? On small files and low speed, wget --limit-rate is unpredictable. wget uses sleep to rate limit. If you’re downloading a small file, it won’t have time to adjust to the speed you want in time for it to complete the transfer. At low speed, it probably won’t get out of TCP slow start, which means that the bit tax will be high for TCP headers, which in turn means that bits on the wire do not correspond to wget actual data transfer. Another thing. wget rate limits bytes and nload reports by default to bits. use nload -u H to report bytes if you haven’t already.
Use CAKE as inbound rate limiter with Diffserv4.
https://www.bufferbloat.net/projects/codel/wiki/Cake/
https://www.bufferbloat.net/projects/codel/wiki/CakeRecipes/
Mark and classify wget traffic by firewall rules (iptables) as bulk traffic.
-j DSCP --set-dscp-class CS1
Limit bulk queue (threshold) rate with CAKE to 10 kbit/s.
Traffic shaping howto:
Have a look to /usr/local/sbin/netzwerkStart.sh and /etc/firewall/firewall_rules_ipv4.txt !!
How can you distinguish wget traffic from any other HTTP(S)/FTP/… traffic in iptables?
@dunne perhaps show an example? Works here as expected (Leap 15.5 or Tumbleweed)?
wget --limit-rate=10k https://download.opensuse.org/tumbleweed/repo/oss/noarch/kernel-source-6.4.9-1.2.noarch.rpm
......
kernel-source-6.4.9-1.2.noarch.rpm 0%[ ....] 39.60K 10.0KB/s <=== It's coming in as requested?
1 Like
You are comparing apples and oranges. wget will never show more than requested speed because it will simply pause when it hits rate limit, so from its point of view it never gets more that 10KB/s. But wget cannot limit the actual wire speed/bandwidth. So it is quite possible that server pushes the whole TCP window worth of data which is then consumed by wget (from the memory buffers on the client) with the rate of 10KB/s.
Or there is some other activity besides wget which is recorded by nload.
1 Like
Rate limited;
4+ hrs
No limit;
4.9 seconds…
Well, it does not match rate limit requested by wget (32KiB/s vs. 10KiB/s). And quick test here for the same command line
I do have some background activity so it shows it.
I do not argue that rate limiting in wget works.
@arvidjaar To my mind better to look at QoS rate limiting on a switch or router and a separate interface?
What other downloader can limit its download rate in average?