I’ve a two server installations (so no X and other graphics stuff) of opensuse 12.2 to which I have SSH access and can sudo.
Network stuff is fast (SSH login, traceroute to the outside world, inbound and outbound smtp, inbound apache).
The machines are VMs on two different ESXi hosts behind two different NAT routers.
Both machines have 1024 megabyte memory and a single core XEON 2.5 Ghz CPU core.
Both machines are frequently updated (at least a couple of times a month but not on a regular schedule).
Two things are very slow on both machines:
1- yast “Network Services: HTTP Server”
2- yast “System Services (Runlevel): Services”
The first takes multiple minutes to start.
The second takes more than a minute per service to show the status.
When performing either of these 2 yast tasks on the machines, the whole affected machine feels slow (i.e. rpm -qa
or starting another yast session takes forever, starting joe
, top
or less
are slow, ssh
login times out)
free -m -h
indicates there is about 90 megabyte of free memory, and 690 megabyte of free buffers (I guess opensuse reserves most of the memory for buffers and little for processes).
top
does not indicate high CPU usage.
/var/log/messages
does not contain really strange stuff
As soon as the 2 slow yast tasks are done, the machines feel snappy again.
How can I start finding a cause?
snip:/home/jeroenp # free -m -h
total used free shared buffers cached
Mem: 997M 912M 84M 0B 36K 605M
-/+ buffers/cache: 306M 690M
Swap: 1.5G 24M 1.4G
top lines of top:
top - 09:44:44 up 8 days, 12:15, 4 users, load average: 30.37, 28.61, 23.47
Tasks: 182 total, 1 running, 181 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.3 us, 1.0 sy, 0.0 ni, 0.0 id, 98.7 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 1021008 total, 941552 used, 79456 free, 36 buffers
KiB Swap: 1532924 total, 25420 used, 1507504 free, 602836 cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3260 vscan 20 0 222m 96m 1060 S 0.0 9.7 0:09.14 /usr/sbin/amavi
3270 vscan 20 0 223m 95m 612 S 0.0 9.6 0:00.01 /usr/sbin/amavi
3271 vscan 20 0 223m 95m 612 S 0.0 9.6 0:00.01 /usr/sbin/amavi
27344 root 20 0 57536 20m 2072 D 0.0 2.0 0:00.50 rpm
26889 root 20 0 159m 19m 7468 S 0.3 1.9 0:11.66 y2base
27436 root 20 0 140m 11m 5424 S 0.0 1.2 0:00.40 y2base
26853 root 20 0 140m 11m 5424 S 0.0 1.2 0:00.42 y2base
/var/log/messages
Jun 20 08:20:23 snip sshd[26228]: Accepted keyboard-interactive/pam for jeroenp from 92.66.117.174 port 53992 ssh2
Jun 20 08:20:23 snip systemd-logind[488]: New session 914 of user jeroenp.
Jun 20 08:20:26 snip sudo: jeroenp : TTY=pts/0 ; PWD=/home/jeroenp ; USER=root ; COMMAND=/bin/bash
Jun 20 08:30:50 snip master[26676]: about to exec /usr/lib/cyrus/bin/ctl_cyrusdb
Jun 20 08:31:14 snip ctl_cyrusdb[26676]: checkpointing cyrus databases
Jun 20 08:31:14 snip ctl_cyrusdb[26676]: archiving log file: /var/lib/imap/db/log.0000000001
Jun 20 08:31:14 snip ctl_cyrusdb[26676]: last message repeated 2 times
Jun 20 08:31:14 snip ctl_cyrusdb[26676]: archiving database file: /var/lib/imap/annotations.db
Jun 20 08:31:14 snip ctl_cyrusdb[26676]: archiving database file: /var/lib/imap/mailboxes.db
Jun 20 08:31:14 snip ctl_cyrusdb[26676]: archiving log file: /var/lib/imap/db/log.0000000001
Jun 20 08:31:14 snip ctl_cyrusdb[26676]: done checkpointing cyrus databases
Jun 20 08:31:15 snip master[2656]: process 26676 exited, status 0
Jun 20 08:38:53 snip sshd[26772]: Accepted keyboard-interactive/pam for jeroenp from 92.66.117.174 port 56572 ssh2
Jun 20 08:38:53 snip sudo: jeroenp : TTY=pts/1 ; PWD=/home/jeroenp ; USER=root ; COMMAND=/bin/bash
Jun 20 08:38:54 snip systemd-logind[488]: New session 916 of user jeroenp.
Jun 20 08:41:40 snip sshd[27023]: Accepted keyboard-interactive/pam for jeroenp from 92.66.117.174 port 56665 ssh2
Jun 20 08:41:42 snip sudo: jeroenp : TTY=pts/2 ; PWD=/home/jeroenp ; USER=root ; COMMAND=/bin/bash
Jun 20 08:41:42 snip systemd-logind[488]: New session 917 of user jeroenp.
Jun 20 08:57:16 snip /USR/SBIN/CRON[27230]: (root) CMD (/usr/lib/AntiVir/guard/avupdate-guard --product=Scanner > /dev/null)
Jun 20 09:03:14 snip master[27304]: about to exec /usr/lib/cyrus/bin/ctl_cyrusdb
Jun 20 09:03:14 snip ctl_cyrusdb[27304]: checkpointing cyrus databases
Jun 20 09:03:14 snip ctl_cyrusdb[27304]: archiving log file: /var/lib/imap/db/log.0000000001
Jun 20 09:03:15 snip ctl_cyrusdb[27304]: last message repeated 2 times
Jun 20 09:03:15 snip ctl_cyrusdb[27304]: archiving database file: /var/lib/imap/annotations.db
Jun 20 09:03:15 snip ctl_cyrusdb[27304]: archiving database file: /var/lib/imap/mailboxes.db
Jun 20 09:03:15 snip ctl_cyrusdb[27304]: archiving log file: /var/lib/imap/db/log.0000000001
Jun 20 09:03:15 snip ctl_cyrusdb[27304]: done checkpointing cyrus databases
Jun 20 09:03:15 snip master[2656]: process 27304 exited, status 0
Jun 20 09:03:15 snip systemd[1]: Cannot add dependency job for unit ldap.service, ignoring: Unit ldap.service failed to load: No such file or directory. See system logs and 'systemctl status ldap.service' for details.
Jun 20 09:03:15 snip systemd[1]: Cannot add dependency job for unit ldap.service, ignoring: Unit ldap.service failed to load: No such file or directory. See system logs and 'systemctl status ldap.service' for details.
Jun 20 09:08:16 snip sshd[27347]: reverse mapping checking getaddrinfo for static.kpn.net [92.66.117.174] failed - POSSIBLE BREAK-IN ATTEMPT!
Jun 20 09:11:24 snip sudo: jeroenp : TTY=pts/3 ; PWD=/home/jeroenp ; USER=root ; COMMAND=/bin/bash
Jun 20 09:11:25 snip systemd-logind[488]: New session 921 of user jeroenp.
–jeroen