Opensuse 11.4 Update Failure, SSH hack, or FSARCHIVER error?

Hello all,

First post here. I have been using OpenSuse 11.x for almost a year now, exclusively as my main machine.

I recently had a problem of some significance that I need help with to diagnose exactly what happened.

I recently was doing some development (via ethernet) (SSH, SFTP, HTTP, apache, mysql) and updating my machine. At the time I didn’t have “disable root login” in the SSH config file set correctly but I do have a complex password. But during the time frame, I also added the update aaa_base bugzilla #642289 #674192 and kdelibs4 #686652 via YAST. Within the hour KDE crashes when attempting to run fsarchiver. I reboot.

  1. Grub console appears. Never seen this before. (Googling and troubleshooting later…)
  2. I mount /dev/sda1 (root sys) in the Systemrescue CD wizard and only the directories:

dev
home
lost+found
media
proc
selinux
sys
tmp

…appeared…(with a filesize of 4096, see pic)… How can this be?

Grub couldn’t find a filesystem nor could the Install DVD recognize it. So it seems as if parts of the system were taken down…but by what? I ran the check filesystem via gpart (on the repair CD) and the disk is okay.

Lastly, I was able to restore an earlier version of my sys via a fsarchiver image and am up and running with minimal discomfort.

Has anyone seen anything like this??? This is very troubling for me as I try to be as security-conscious as possible and had significant problems upgrading from 11.3 to 11.4. I have a hardware and software firewall, etc…

Any insight would be greatly appreciated.

http://img51.imageshack.us/img51/7224/img1337w.jpg

This is the way ls works :slight_smile: (nothing to worry about). This is an output from my openSUSE and basically any linux I have used :

> ls -l
razem 40
drwxr-xr-x 2 test users 4096 03-12 14:10 bin
drwxr-xr-x 2 test users 4096 03-12 14:10 Dokumenty
drwxr-xr-x 2 test users 4096 03-12 14:10 Muzyka
drwxr-xr-x 2 test users 4096 03-12 14:10 Obrazy
drwxr-xr-x 2 test users 4096 03-12 14:10 Pobrane
drwxr-xr-x 2 test users 4096 03-12 14:10 public_html
drwxr-xr-x 2 test users 4096 03-12 14:10 Publiczny
drwxr-xr-x 2 test users 4096 03-12 14:11 Pulpit
drwxr-xr-x 2 test users 4096 03-12 14:10 Szablony
drwxr-xr-x 2 test users 4096 03-12 14:10 Wideo

I’ve never used fsarchiver so I don’t know anything about it (could You tell us something more about this application) but maybe it’s an fsarchiver bug ? Did You check the logs before restoring an eariler version of your system ?

Best regards,
Greg

Thanks for responding. I didn’t check my logs after KDE crashed because I didn’t think there was something significantly wrong with my system. Boy, was I was wrong. :shame: Where I did ls was on my root sys, after the crash. My home is on a different partition. On the root sys, several important directories were missing…that’s why Grub couldn’t fix or OpenSuse couldn’t repair…

Could someone of hacked my system with the “enable root login” in SSH config on and deleted directories on my root partition (even with an extremely strong password)?

When using fsarchiver, an imaging program, I did this command to save an image of the root sys:

QuickStart - FSArchiver


fsarchiver savefs /mnt/backup/gentoo-rootfs.fsa /dev/sda1 -v -A -a

I have never had a problem with Fsarchiver…although KDE crashed when it was running.

:dont-know:

Have you run fsck on the root partition. Note this must be done with out the partition mounted, thus you must do it from a bootable CD.

Sounds as if you had a file system crash. This can be caused by drive going bad.

It’s possible but very unlikely. A strong password is very important but even more effective way to secure ssh is to set it up to listen on a port different than 22. There is a log for checking who logged into your system and when ("/var/log/secure" as far as I can remember) :slight_smile: IMHO without the logs we can just speculate what happened untill it happens again but I hope it won’t and we will not have anything to talk about.

Best regards,
Greg

IMHO also it’s not very likely that someone else destroyed your root partition. Sure you did not make a typo causing this?

@gogalthorp

Have you run fsck on the root partition. Note this must be done with out the partition mounted, thus you must do it from a bootable CD.

Sounds as if you had a file system crash. This can be caused by drive going bad.

Yes, the drive is clean and relatively new. :expressionless: When I mounted the drive in Gparted there was some minor corruption that the program fixed. I still can’t account for the missing directories. :\

@glistwan, Knurpht

Thanks for your insight. Does it seem more likely that the updates crashed my system, or Fsarchiver (which was running as root) crashed it than a SSH hack? The symptom being missing critical directories?

Yes, I hope it doesn’t happen again. Now I know to make it a habit when something abnormal happens to check my logs immediately rather than just reboot.

Thanks again, gentlemen.

You’re welcome. I’d say it’s more likely that Fsarchiver is at fault but as I said that’s just speculation.

Best regards,
Greg

So you did have a damaged file system. If so it is sometimes not possible to reconstruct all the pieces. Did you look in the Lost & found directory and see if some of the missing stuff is there? Note that if this is root then it is almost impossible to re-piece things back together. You must either restore from backup or reinstall.

As to why, I suggest that you may have mis-configured the Fsarchiver program which started to overwrite the root partition and caused the damage. But that is speculation.

On 2011-05-16 19:06, tuxtoes wrote:
> Any insight would be greatly appreciated.

No idea. A bad filesystem crash, perhaps… Looking in the logs could
perhaps say something. Very difficult to guess.


Cheers / Saludos,

Carlos E. R.
(from 11.2 x86_64 “Emerald” at Telcontar)

I think I tracked down the issue. The updates may of crashed KDE when FSArchiver was running. SELinux was enabled when I was running FSArchiver

It’s also important that you make sure that SELinux is not enabled in the kernel running FSArchiver when you save a file-system which has been labeled by SELinux

QuickStart - FSArchiver

Thanks for the feedback :slight_smile:

Best regards,
Greg