DoS with high volume of outbound duplicate ack

English Network/Internet
#1

Hi all,

I’m having an issue at the moment on an openSUSE 13.1 server. It looks as though we are experiencing a DDoS attack.

http://imgur.com/ZErPg8h

From multiple external IP addresses, we receive a number TCP ACK messages, from there our internal openSUSE server with an exponentially higher volume of dup ack reply messages (I’m reading more than 700,000 reply dup ack messages from my internal host in a single session!)

Does anyone know of any bugs that might be causing so many dup ack outbound messages in a single session?

(It looks to me like a DupACK Spoofing attack)
http://scenic.princeton.edu/network20q/wiki/index.php?title=Attacks_on_TCP_Congestion_Control

Any help would be greatly appreciated!

Cheers,
Josh

#2

I can’t see a way to edit my post, but here is a better image of the traffic:

http://imgur.com/51F3kRl

#3

Hi
This user had network issues (as in network provider)
http://serverfault.com/questions/526616/excessive-tcp-dup-ack-tcp-fast-retransmission-causing-issues-on-network-w

So it may be something out in your network, router etc…

#4

How about running this as root:

Code:

echo 1 > /proc/sys/net/ipv4/tcp_abc

http://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-14-tcp-congestion-control-with-a-misbehaving-receiver/

–
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…

#5

Hi,

Thank you for the quick responses!

I’ve had a read through those links and everything points to:

echo 1 > /proc/sys/net/ipv4/tcp_abc

I think you’ve both hit the nail on the head.

I’m going to try those commands and I’ll report back with the results.

Cheers,
Josh

#6

Hi,

I’ve attempted to make the changes, however, my version of openSUSE 13.1 “Bottle” does not have the tcp_abc file in the /proc/sys/net/ipv4/ directory.
I can see the file exists in one of my enterprise SUSE installs.

Does anyone know if I can just create the file, or if there is a way to verify the correct location?

Cheers,
Josh

#7

Add it to /etc/sysctl.conf.

net.ipv4.tcp_abc = yourdesiredvalue

Then sysctl -p to load the values, they will now take effect on every reboot.

#8

If you’re running openSUSE as a “real” server, you will need to modify a number of TCP parameters, including

  • TCP/IP Congestion Control algorithm. The default assumes end-user usage which is typically tiny files typically about 3kb in size, only a very small, limited number of connections. If you transfer large files and especially over speeds Gigabit speeds, possibly involving a network connection that’s not 802.3 (Fast Ethernet) you should change this.<br>

  • TCP/IP buffers. The default assumes limited connections in short bursts. If you have sustained connections, large number of simultaneous, transfer large files, etc then you need to re-allocate system resources to support these kinds of connections.

I did a write-up on this years ago for an earlier openSUSE, but all recommended settings and other changes are still applicable in 13.1 (and for the forseeable future). Should include all the info you need to know to make <intelligent> decisions.

For anyone who is running anything larger than a tiny, low load server,
https://sites.google.com/site/4techsecrets/optimize-and-fix-your-network-connection

HTH,
TSU

#9

Hi all,

I’ve attempted to

Add it to /etc/sysctl.conf.

net.ipv4.tcp_abc = yourdesiredvalue

Then sysctl -p to load the values, they will now take effect on every reboot.

I receive the message:

sysctl: cannot stat /proc/sys/net/ipv4/tcp_abc: No such file or directory

I cannot touch the file or create it. Has anyone been able to create it successfully?
(Tried rebooting as well)

Cheers,
Josh

#10

I tried this, as well, and the result is the same as joshua1909.

I’ve attempted to:

Add it to /etc/sysctl.conf.

net.ipv4.tcp_abc = yourdesiredvalue

Then sysctl -p to load the values, they will now take effect on every reboot.

I received the message:

sysctl: cannot stat /proc/sys/net/ipv4/tcp_abc: No such file or directory

Is there anything we can do? Must we reboot for this to work??

#11

Hi,

I have tested an older version of OpenSUSE (12.1) and the TCP_ABC flag is present. I believe this flag has been removed as is no longer required, because "Linux already implements other better window protection
heuristics"1.

1 http://comments.gmane.org/gmane.linux.network/257896

In my case I am still getting inbound DUP Ack attacks, however, the issue with the exponentially large number of replies from my host is fixed–and so I am able to combat the attack.
The outbound reply issue was due to, I believe, a combination of VMXNET3 drivers in the VM and a NIC issue on my VMware host.

In VMware, the primary NIC in my port group was offline and it was using the standby. Something here caused the DUP messages, I had other VMs on the same host that did not experience the issue.
So I believe it was a combination of VMXNET and the port group configuration. As soon as the offline NIC was removed from the port group the DUP error disappeared. I would not expect this behaviour, but that’s how it went…

Thank you everyone for your time.

Cheers,
Josh

#12

Hi
I would imagine it needs to be enabled in the kernel (as in the kernel re-copiled). Might be worth creating a bug and see if someone can confirm;
https://en.opensuse.org/openSUSE:Submitting_bug_reports