Hi,
Everyday my openSUSE 11.3 server starts a perl script that goes straight to 100% cpu usage. If you let run it won’t ever stop. I capped it once to 20% with cpulimit, but it keeps on going.
Now what scares me is that the command that seems to started it is /usr/sbin/ap
There is no such file… Could it be a rootkit? I don’t know where to look next, I usually end up killing the process. Within 24 hours it just starts again.
Any ideas?
Ricochet wrote:
> Hi,
>
> Everyday my openSUSE 11.3 server starts a perl script that goes
> straight to 100% cpu usage. If you let run it won’t ever stop. I capped
> it once to 20% with cpulimit, but it keeps on going.
>
> Now what scares me is that the command that seems to started it is
> /usr/sbin/ap
>
> There is no such file… Could it be a rootkit? I don’t know where to
> look next, I usually end up killing the process. Within 24 hours it just
> starts again.
>
> Any ideas?
>
>
i’m a little confused about a few things:
boot a live cd and look to see if there is a file /usr/sbin/ap on the
hard drive…and also look throughly inside /tmp to see if there are
any executable files…if so i’d say it is a pretty good bet you have
been cracked and rooted… CAREFUL: you must make sure you are looking
at the hard drive and not the live CD’s system…
–
DenverD
When it comes to chocolate, resistance is futile.
CAVEAT: http://is.gd/bpoMD [posted via NNTP w/openSUSE 10.3]
- how did you determine it is a perl script running?
TOP says PERL in the Command column
- if you know it is a perl script, do you know its location/name?
When I press C in TOP the Command column says /usr/sbin/ap
- how do you know it is started by /usr/sbin/ap
I don’t know what starts it, it’s just what I saw in the TOP column Command
** - why did you opt for cpulimit rather than nice?**
Since other processes aren’t that busy a renice doesn’t seem to work. It will only use less then 100% when another process claims more CPU time. As long as none will, it keeps using 99/100%
- when you kill the process what stops working on your system?
As far as I can tell, nothing
**
** - what operating system and version are you running? is it fully patched?**
OpenSuSE 11.3, fully patched
** - what DE?**
What?
- are you running apache?
Yes
- do you have any other indication that your machine might have been
rooted?
No
I’ll check with the Live CD later since I’m not at the same location atm.
Ricochet wrote:
- what DE?
What?
I think DenverD means “Desktop Environment” - e.g. KDE or GNOME or minimal text install
LOL, Duh… How could I not get that 
Gnome it is, but since it’s my server, it boots without X
Ricochet wrote:
> LOL, Duh… How could I not get that
>
> Gnome it is, but since it’s my server, it boots without X
thanks for answers…
i hope a real guru comes along soon because i’m not too sure about the
best was to proceed…i do think i’d take a look at the drive from a
known set of tools (that is, it is routine for rootkits to alter tools
such as ls so that evil files can’t be seen)…
if it is directly facing the internet and you may have been a little
lax in updates…well, i don’t have any idea how tight your security
has been but i wouldn’t hope to have any important personal financial
data on that box, nor accessible via the local network from that box…
and, it would be really nice to have a strong passwords on any boxes
on that net…
on the other hand, most crackers as not even close to being so clumsy
as to start up a red flag waving PERL script announcing: I’m here!
if i google this:
“/usr/sbin/ap” rootkit
i get a several interesting, and scary hits–but they seem to be
dealing with “ap-trapd” and not just “ap” and i think many are just
echos of the same one or two (by the way, are you also posting in
German?)…
given the fact that i’ve never run apache i won’t advise you to rush
to the machine and torch the evil within, instead:
have a look with a live CD, report findings and hope for competent
help to arrive here, soon…
–
DenverD
When it comes to chocolate, resistance is futile.
CAVEAT: http://is.gd/bpoMD [posted via NNTP w/openSUSE 10.3]
Me no speak German
I run the default firewall with only 80 and 443 open from the outside. I also run a wordpress and drupal site on it. The service is owned by wwwrun so it is web related I guess.
Thanks for your help so far!
hi,
i’m no guru either, but let’s find out what perl is doing:
lsof | grep perl
that will list you all opened files by perl. if in this list you find /usr/sbin/ap, take perl’s pid (the first number in this list) and look it up in /proc.
so lets say perl’s pid is 15423:
cd /proc/15423/fd
in this directory you will find all open file descriptors by this instance of perl. lets try to find /usr/sbin/ap here:
la | grep /usr/sbin/ap
if it’s there, you can open it directly using vim (or another editor) using it’s link (which will have a unique number as name) and have a look at it.
hope that helps
p.s.:
your thread as well? http://serversupportforum.de/forum/dedizierte-server/41294-usr-sbin-ap-und-99-cpu-last.html
brian j wrote:
> p.s.:
> your thread as well? http://tinyurl.com/237pkft
i found that one also and the OP here said he doesn’t do German…so,
there is probably info there to mine…but, not by me and
googleTranslate…no thanks.
we need a German guru, i think…
–
DenverD
When it comes to chocolate, resistance is futile.
CAVEAT: http://is.gd/bpoMD [posted via NNTP w/openSUSE 10.3]
what he found, where some undeliverable (spam) mails logged in /var/log/mail which suggested either a misuse of PHP or some other hack.
so the OP should check that as well, for example he could try to find some undeliverable mail:
grep suspended /var/log/mail
grep refused /var/log/mail
additonally they suggest, you post the result of the following attached to those i already gave.
ls -alh /proc/*/task/<PID>/
cat /proc/*/task/<PID>/status
cat /proc/*/task/<PID>/cmdline
just to be sure, check ssh connections:
netstat | grep ssh
before checking if the pid of your script may try to connect to the outside:
netstat -p | grep <pid>
Thanks for your help, I’ll try the perl commands later on.
For now I checked the mail. There is no suspended mail, but there are a lot of refusals. All of them seem to come from my own website, running on this server. It fails because my ISP is blocking port 25. I never cared for mail coming from my website. Root keeps getting mail from the site telling it can’t send over port 25. I kinda just deleted those mails.
I don’t think this is the problem, the last time the website tried to mail was October 10th.