I'm part of a botnet?!

Hi again. For the last 4 or 5 days, my server is making strange traffic. Here’s a bit of tcpdump


22:10:48.572927 IP XXX.XXX.XXX.XXX.48002 > 162.93.227.34.443: Flags [S], seq 454828639, win 14600, options [mss 1460,sackOK,TS val 80439057 ecr 0,nop,wscale 7], length 0
22:10:48.572943 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47888: Flags .], ack 3346143438, win 1400, options [mss 1460], length 0
22:10:48.573432 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47892: Flags .], ack 2024936710, win 1400, options [mss 1460], length 0
22:10:48.573779 IP XXX.XXX.XXX.XXX.48004 > 162.93.227.34.443: Flags [S], seq 3330721355, win 14600, options [mss 1460,sackOK,TS val 80439057 ecr 0,nop,wscale 7], length 0
22:10:48.573807 IP XXX.XXX.XXX.XXX.48005 > 162.93.227.34.443: Flags [S], seq 1178102980, win 14600, options [mss 1460,sackOK,TS val 80439057 ecr 0,nop,wscale 7], length 0
22:10:48.574425 IP XXX.XXX.XXX.XXX.48006 > 162.93.227.34.443: Flags [S], seq 160888195, win 14600, options [mss 1460,sackOK,TS val 80439057 ecr 0,nop,wscale 7], length 0
22:10:48.574617 IP XXX.XXX.XXX.XXX.48007 > 162.93.227.34.443: Flags [S], seq 2760497392, win 14600, options [mss 1460,sackOK,TS val 80439057 ecr 0,nop,wscale 7], length 0
22:10:48.575075 IP XXX.XXX.XXX.XXX.48008 > 162.93.227.34.443: Flags [S], seq 524629507, win 14600, options [mss 1460,sackOK,TS val 80439057 ecr 0,nop,wscale 7], length 0
22:10:48.575181 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47905: Flags .], ack 2558824917, win 1400, options [mss 1460], length 0
22:10:48.575430 IP XXX.XXX.XXX.XXX.48009 > 162.93.227.34.443: Flags [S], seq 3296663687, win 14600, options [mss 1460,sackOK,TS val 80439057 ecr 0,nop,wscale 7], length 0
22:10:48.575930 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47897: Flags .], ack 1139190286, win 1400, options [mss 1460], length 0
22:10:48.575934 IP XXX.XXX.XXX.XXX.48011 > 162.93.227.34.443: Flags [S], seq 2133870181, win 14600, options [mss 1460,sackOK,TS val 80439057 ecr 0,nop,wscale 7], length 0
22:10:48.575973 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47910: Flags .], ack 3852466677, win 1400, options [mss 1460], length 0
22:10:48.576008 IP XXX.XXX.XXX.XXX.48010 > 162.93.227.34.443: Flags [S], seq 3226329832, win 14600, options [mss 1460,sackOK,TS val 80439057 ecr 0,nop,wscale 7], length 0
22:10:48.576176 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47891: Flags .], ack 365233685, win 1400, options [mss 1460], length 0
22:10:48.576206 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47899: Flags .], ack 2532448764, win 1400, options [mss 1460], length 0
22:10:48.576425 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47894: Flags .], ack 2683127351, win 1400, options [mss 1460], length 0
22:10:48.576736 IP XXX.XXX.XXX.XXX.48013 > 162.93.227.34.443: Flags [S], seq 931998935, win 14600, options [mss 1460,sackOK,TS val 80439058 ecr 0,nop,wscale 7], length 0
22:10:48.577159 IP XXX.XXX.XXX.XXX.48014 > 162.93.227.34.443: Flags [S], seq 4087074095, win 14600, options [mss 1460,sackOK,TS val 80439058 ecr 0,nop,wscale 7], length 0
22:10:48.577250 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47900: Flags .], ack 3479993579, win 1400, options [mss 1460], length 0
22:10:48.577531 IP XXX.XXX.XXX.XXX.48015 > 162.93.227.34.443: Flags [S], seq 497199545, win 14600, options [mss 1460,sackOK,TS val 80439058 ecr 0,nop,wscale 7], length 0
22:10:48.577927 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47904: Flags .], ack 1714908389, win 1400, options [mss 1460], length 0
22:10:48.578309 IP XXX.XXX.XXX.XXX.48016 > 162.93.227.34.443: Flags [S], seq 3107016326, win 14600, options [mss 1460,sackOK,TS val 80439058 ecr 0,nop,wscale 7], length 0
22:10:48.578426 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47898: Flags .], ack 3425055230, win 1400, options [mss 1460], length 0
22:10:48.578835 IP XXX.XXX.XXX.XXX.48017 > 162.93.227.34.443: Flags [S], seq 3441800893, win 14600, options [mss 1460,sackOK,TS val 80439058 ecr 0,nop,wscale 7], length 0
22:10:48.578927 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47901: Flags .], ack 2653327878, win 1400, options [mss 1460], length 0
22:10:48.579215 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47902: Flags .], ack 2573233621, win 1400, options [mss 1460], length 0
22:10:48.579347 IP XXX.XXX.XXX.XXX.48018 > 162.93.227.34.443: Flags [S], seq 2608457678, win 14600, options [mss 1460,sackOK,TS val 80439058 ecr 0,nop,wscale 7], length 0
22:10:48.579427 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47903: Flags .], ack 948204077, win 1400, options [mss 1460], length 0
22:10:48.579908 IP XXX.XXX.XXX.XXX.48019 > 162.93.227.34.443: Flags [S], seq 265323797, win 14600, options [mss 1460,sackOK,TS val 80439058 ecr 0,nop,wscale 7], length 0
22:10:48.580179 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47912: Flags .], ack 27697374, win 1400, options [mss 1460], length 0
22:10:48.580243 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47908: Flags .], ack 3446486171, win 1400, options [mss 1460], length 0
22:10:48.580422 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47913: Flags .], ack 249511405, win 1400, options [mss 1460], length 0
22:10:48.580680 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47915: Flags .], ack 913501692, win 1400, options [mss 1460], length 0
22:10:48.580938 IP 162.93.227.34.443 > XXX.XXX.XXX.XXX.47909: Flags .], ack 4004336091, win 1400, options [mss 1460], length 0
22:10:48.581094 IP XXX.XXX.XXX.XXX.48020 > 162.93.227.34.443: Flags [S], seq 1312176602, win 14600, options [mss 1460,sackOK,TS val 80439059 ecr 0,nop,wscale 7], length 0
22:10:48.581349 IP XXX.XXX.XXX.XXX.48021 > 162.93.227.34.443: Flags [S], seq 4059057971, win 14600, options [mss 1460,sackOK,TS val 80439059 ecr 0,nop,wscale 7], length 0

And an email from that IP’s ISP (I first tought that the server was attacking me):


Sir

  Please take another look at your TCPdump.  The spike in network traffic is your server attacking us.  Please immediately take this device offline.  You are currently part of a DDOS against us.

Thank You

There may be an exploit in apache2? If I restart the service, the traffic stops, but after a while it starts again.

www:~ # apache2ctl -v
Server version: Apache/2.2.22 (Linux/SUSE)
Server built:   2013-03-28 12:52:01.000000000 +0000
www:~ # cat /etc/issue
Welcome to openSUSE 12.3 "Dartmouth" - Kernel \r (\l).




www:~ #

Any ideas how I can stop this?

On 2013-04-24 21:26, CaracalSef wrote:
>
> Hi again. For the last 4 or 5 days, my server is making strange traffic.
> Here’s a bit of tcpdump

I’d suggest you ask in the security mail list.

openSUSE:Communication channels: Mailing lists

Security
Mailinglist Archive

I’d recommend you use a dedicated email for mail lists, not one you want
private.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

On Wed, 24 Apr 2013 19:26:01 +0000, CaracalSef wrote:

> Any ideas how I can stop this?

You could temporarily use iptables to drop traffic destined for the
target server (or route it to 127.0.0.1).

You can use ‘lsof -i’ to try to isolate the process that’s initiating the
connections, in the event that it’s a child of Apache2.

Consider the box compromised. You can try running rkhunter and see if it
turns anything up, and it probably wouldn’t hurt to run clamav against
the box just to make sure nothing that it can detect is actually running
on the box.

Best practice with a compromised box is to reinstall, patch, and re-
secure it.

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

Agree on re-pave, re-build, re-deploy.

Otherwise, I don’t see how tcpdump is going to tell you much. For any chance at analysis, you’d have to do a packet capture to determine things like protocol, payload, possibly service.

TSU

I agree. Also it might be of interest that that IP address seems to belong to a company whose business is aggressively scanning other people’s servers on behalf of the RIAA, MPAA, etc.

On 04/25/2013 01:04 AM, Jim Henderson wrote:
> Best practice with a compromised box is to reinstall, patch, and re-
> secure it.

just asking: wouldn’t that be a little better if it read:

“. . . to format, reinstall, patch . . .”


dd

On Thu, 25 Apr 2013 11:20:47 +0000, dd wrote:

> On 04/25/2013 01:04 AM, Jim Henderson wrote:
>> Best practice with a compromised box is to reinstall, patch, and re-
>> secure it.
>
> just asking: wouldn’t that be a little better if it read:
>
> “. . . to format, reinstall, patch . . .”

That’s essentially what I meant, but yes, to be complete, doing a fresh
installation on the drive is the best practice.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

https://devlog.websafe.pl/2013/03/27/outdated-joomla-websites-with-jce-used-to-attack-beneficial-data-processing-corp-and-regions-financial-corporation/

This was the exact same situation for me, too.

On Thu, 25 Apr 2013 21:46:06 +0000, CaracalSef wrote:

> http://tinyurl.com/c6zl5po
>
> This was the exact same situation for me, too.

Interesting, thanks for that info.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 04/25/2013 11:46 PM, CaracalSef wrote:
>
> http://tinyurl.com/c6zl5po
>
> This was the exact same situation for me, too.

wow, looks like you need to visit FormatLand.
but, i’m no security expert…

the first reply to your initial post recommended you join the
openSUSE security mail list.

if you overlooked that advice then, i strongly suggest you follow it
now by joining that list and posting your situation/symptoms to it…

maybe they tell you you don’t need to format to recover.

anyway, if the developers there are unaware of this Joomla hole, they
need to be!


dd
http://tinyurl.com/DD-Caveat

The problem nowadays is that once you’ve been “owned” it’s unlikely that you’ll be able to fix any problems on the box with any certainty. If you feel that the exploit is very contained, then you can consider just patching the hole after removing the exploit.

To a large degree if your box is mis-behaving by generating and not just relaying spam, it’s more than likely that the machine is not patchable (IMO). Truly knowing that the issue is restricted and addressable without re-formatting today requires talent.

So, perhaps lesson is that the best strategy is full backups so your restore doesn’t require manually re-building.

TSU

Thank you all for advices. I have backups, but in backups, the vulnerability still exists. I removed the script, patched the vulnerability and now I’m monitoring the activity on the server. For the last 2 days almost, there’s no strage behave.

Thank you all again. I feel kind of stupid now that I thought I was attack, when, in fact, my server attacked the others…