haveged

English Applications
#1

Just did the zypper dup for tumbleweed 20181110 and made a check with zypper ps -s after a reboot and came up with this.

How can you restart a deleted processes? Should haveged be reinstalled?

zypper ps -s
The following running processes use deleted files:

PID | PPID | UID | User | Command           | Service
----+------+-----+------+-------------------+--------
387 | 1    | 0   | root | haveged (deleted) | haveged

You may wish to restart these processes.
#2

On Mon, 12 Nov 2018 19:46:03 +0000, GDixon wrote:

> Just did the zypper dup for tumbleweed 20181110 and made a check with
> zypper ps -s after a reboot and came up with this.
>
> How can you restart a deleted processes? Should haveged be reinstalled?
>
>
>
>
> Code:
> --------------------
> zypper ps -s
> The following running processes use deleted files:
>
> PID | PPID | UID | User | Command | Service
> ----±-----±----±-----±------------------±-------
> 387 | 1 | 0 | root | haveged (deleted) | haveged
>
> You may wish to restart these processes.
> --------------------

For this service, you can just run:

sudo systemctl restart haveged

No need to reinstall the service - Linux keeps the existing process
running (and doesn’t free up the file inodes) until the service restarts.

There are a few services that may continue to hold files open after a
restart that zypper ps -s will incorrectly report. I see that on one of
my Leap systems on occasion - nothing really to worry about.

Jim

–
Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

#3

Thank you,

That did solve the situation.

Edit:
I spoke too soon. It does start the service but on either a reboot or shutdown the service isn’t started and needs to be started again.
Where can I make this change to start haveged permanent?

Greg

#4

I did check services manager and haveged is checked to start on boot.

#5

When your service is stopped,
Run the following and post the results

systemctl status havegod

Your result should include among possible other things

  • Current status of your service
  • Whether it ever started
  • If it started, that it definitely exited
  • A system log snippet that might relate to the start and stop.

HTH,
TSU

#6

Some puzzling restarts occur:

erlangen:~ # journalctl -b -u haveged.service 
-- Logs begin at Sun 2018-08-26 08:48:18 CEST, end at Tue 2018-11-13 06:50:10 CET. --
Nov 12 19:19:32 erlangen haveged[214]: haveged: listening socket at 3
Nov 12 19:19:33 erlangen haveged[214]: haveged: restart in new root: /sysroot
Nov 12 19:19:33 erlangen haveged[214]: haveged: listening socket at 3
Nov 12 19:19:33 erlangen haveged[214]: haveged: Fail:set_watermark()!
Nov 12 19:19:33 erlangen haveged[214]: haveged starting up
Nov 12 19:19:33 erlangen systemd[1]: haveged.service: Main process exited, code=exited, status=1/FAILURE
Nov 12 19:19:33 erlangen systemd[1]: haveged.service: Failed with result 'exit-code'.
Nov 12 19:19:33 erlangen systemd[1]: haveged.service: Service RestartSec=100ms expired, scheduling restart.
Nov 12 19:19:33 erlangen systemd[1]: haveged.service: Scheduled restart job, restart counter is at 1.
Nov 12 19:19:33 erlangen systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Nov 12 19:19:33 erlangen systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Nov 12 19:19:33 erlangen haveged[410]: haveged: listening socket at 3
Nov 12 19:19:34 erlangen systemd[1]: haveged.service: Service RestartSec=100ms expired, scheduling restart.
Nov 12 19:19:34 erlangen systemd[1]: haveged.service: Scheduled restart job, restart counter is at 2.
Nov 12 19:19:34 erlangen systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Nov 12 19:19:34 erlangen systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Nov 12 19:19:34 erlangen haveged[567]: haveged: listening socket at 3
erlangen:~ #

Any idea?

#7

According to what you just posted, your haveged service is running fine although after a few restarts in the beginning.
As I described in my post previous to this one, you need to run the following to return current status of the service

systemctl status haveged

To further verify that your system has no related issues, you can check the result of the following command which reads the size of your entropy cache, which should be larger than 1024

cat /proc/sys/kernel/random/entropy_avail

TSU

#8

Entropy cache is fine so it appears waiting for the next DUP would be wise and see where things lie.

Sometimes waiting is the best answer.

#9

You may try systemctl restart haveged:

erlangen:~ # journalctl --since today -u haveged.service 
-- Logs begin at Sun 2018-08-26 08:48:18 CEST, end at Tue 2018-11-13 22:15:18 CET. --
Nov 13 22:14:27 erlangen haveged[567]: haveged: Stopping due to signal 15
Nov 13 22:14:27 erlangen systemd[1]: Stopping Entropy Daemon based on the HAVEGE algorithm...
Nov 13 22:14:27 erlangen haveged[567]: haveged starting up
Nov 13 22:14:27 erlangen systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Nov 13 22:14:27 erlangen systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Nov 13 22:14:27 erlangen haveged[29491]: haveged: listening socket at 3
erlangen:~ # cat /proc/sys/kernel/random/entropy_avail
3755
erlangen:~ #
#10

Looking at a couple of my machines,
It looks like it’s not unusual for haveged to have to undergo at least one restart although none of mine had to restart more than once after the initial start.

I don’t think that restarts are anything to be concerned about,
The only real test of consequence is the entropy cache test I posted…
If that’s good, then your system is healthy.

TSU

#11

Zypper dup to the latest 20181112-0 cleared it so everything is peachy!

#12

The same here: Failures and restarts are gone with snapshot 20181112.

#13

Some days ago started on every new boot. I restart the service every day!
Seems that /usr/sbin/haveged is deleted after every new boot !


[FONT=monospace]80 | 1    | 0   | root    | haveged (deleted) | haveged
[/FONT]

systemctl status haveged


**●** haveged.service - Entropy Daemon based on the HAVEGE algorithm 
Loaded: loaded (/usr/lib/systemd/system/haveged.service; enabled; vendor preset: enabled) 
Active: **active (running)** since Sun 2020-10-11 07:40:33 EEST; 2h 22min ago 
Docs: man:haveged(8) 
http://www.issihosts.com/haveged/ 
Main PID: 180 (haveged) 
Tasks: 1 (limit: 4915) 
Memory: 5.0M 
CGroup: /system.slice/haveged.service 
└─180 @usr/sbin/haveged -w 1024 -v 0 -F

[FONT=monospace]

sudo journalctl -b -u haveged.service 
-- Logs begin at Sun 2020-10-11 04:40:35 EEST, end at Sun 2020-10-11 10:04:05 EEST. -- 
Οκτ 11 07:40:33 FOSTW haveged[180]: haveged: command socket is listening at fd 3

cat /proc/sys/kernel/random/entropy_avail 
3682

[/FONT]

[FONT=monospace][FONT=monospace][FONT=monospace][FONT=monospace][FONT=monospace]ls -l 
-rwxr-xr-x 1 root root 31496 Aug  25 15:57 /usr/sbin/haveged*
[/FONT][/FONT]$ file /usr/sbin/haveged 
/usr/sbin/haveged: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=0a6fd315a7ba8238afed7f83ac3ede20b1f0eae2, for GNU/Linux 3.2.0, stripped[/FONT][/FONT][/FONT]

[FONT=monospace][FONT=monospace][FONT=monospace][FONT=monospace]

[/FONT][/FONT][/FONT][/FONT]after restarting the service
[FONT=monospace][FONT=monospace][FONT=monospace][FONT=monospace][FONT=monospace]

sudo journalctl -b -u haveged.service 
-- Logs begin at Sun 2020-10-11 04:40:35 EEST, end at Sun 2020-10-11 10:18:19 EEST. -- 
Οκτ 11 07:40:33 FOSTW haveged[180]: haveged: command socket is listening at fd 3 
Οκτ 11 10:17:37 FOSTW haveged[180]: haveged: Stopping due to signal 15 
Οκτ 11 10:17:37 FOSTW haveged[180]: haveged starting up 
Οκτ 11 10:17:37 FOSTW systemd[1]: Stopping Entropy Daemon based on the HAVEGE algorithm... 
Οκτ 11 10:17:37 FOSTW systemd[1]: haveged.service: Succeeded. 
Οκτ 11 10:17:37 FOSTW systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. 
Οκτ 11 10:17:37 FOSTW systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. 
Οκτ 11 10:17:37 FOSTW haveged[683]: haveged: command socket is listening at fd 3

[/FONT][/FONT][/FONT] [/FONT][/FONT]

#14

This is almost two years old. Do you really think people are still watching this thread?

#15

I’m not sure what is really happening. It is probably some system data structures being change to make it appear as if “/usr/sbin/haveged” is being deleted.

When I reboot:

The timestamp on “/usr/sbin” is not changed;
the inode number for “/usr/sbin/haveged” is not changed.

So I don’t think it is really being deleted. But “lsof” does agree with “zypper ps” that it has been deleted.

I expect it has something to do with the way that “haveged” works.

On a system where I am not using any encrypted partition, but I am using an ecryptfs private directory, I do not see this happening with “haveged”. That is, it does not show up in the output of “zypper ps” on that system. And “lsof -p pid-of-process” does not show a “deleted”. I’m only seeing this strange behaviour where I am using an encrypted partition.

Added note: it may be that “haveged” is started from the “initrd”, and that will look to be deleted after the switch over to the normal root.

#16

Thanks you very much for your answer nrickert.


sudo lsof -p 683 
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs 
Output information may be incomplete. 
lsof: WARNING: can't stat() fuse file system /run/user/1000/doc 
Output information may be incomplete. 
COMMAND PID USER   FD   TYPE             DEVICE SIZE/OFF    NODE NAME 
haveged 683 root  cwd    DIR              254,1     4096       2 / 
haveged 683 root  rtd    DIR              254,1     4096       2 / 
haveged 683 root  txt    REG              254,1    31496 1583256 /usr/sbin/haveged 
haveged 683 root  mem    REG              254,1  2106872 1049556 /lib64/libc-2.32.so 
haveged 683 root  mem    REG              254,1   104592 1517087 /usr/lib64/libhavege.so.2.0.0 
haveged 683 root  mem    REG              254,1   200920 1049539 /lib64/ld-2.32.so 
haveged 683 root    0r   CHR                1,3      0t0    2051 /dev/null 
haveged 683 root    1u  unix 0xffff9c4f5f421800      0t0 1149840 type=STREAM 
haveged 683 root    2u  unix 0xffff9c4f5f421800      0t0 1149840 type=STREAM 
haveged 683 root    3u  sock                0,8      0t0 1151489 protocol: UNIX 
haveged 683 root    4u   CHR                1,8      0t0    2055 /dev/random 
$ sudo ls -l /lib64/libc-2.32.so 
-rwxr-xr-x 1 root root 2106872 Oct   6 18:29 /lib64/libc-2.32.so 
$ sudo ls -l /lib64/ld-2.32.so 
-rwxr-xr-x 1 root root 200920 Oct   6 18:29 /lib64/ld-2.32.so 
$ sudo ls -l /usr/lib64/libhavege.so.2* 
lrwxrwxrwx 1 root root     18 Aug  25 15:57 /usr/lib64/libhavege.so.2 -> libhavege.so.2.0.0 
-rwxr-xr-x 1 root root 104592 Aug  25 15:57 /usr/lib64/libhavege.so.2.0.0 
$ sudo ls -l /usr/sbin/haveged 
-rwxr-xr-x 1 root root 31496 Aug  25 15:57 /usr/sbin/havege

libc & ld changed, maybe these are the problematic files or haveged should be renewed.

#17

Are they 2 different files?


sudo lsof -p 180 
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs 
Output information may be incomplete. 
COMMAND PID USER   FD   TYPE             DEVICE SIZE/OFF  NODE NAME 
haveged 180 root  cwd    DIR                0,2        0     2 / 
haveged 180 root  rtd    DIR                0,2        0     2 / 
haveged 180 root  txt    REG                0,2    31496 13831 /usr/sbin/haveged (deleted) 
haveged 180 root  DEL    REG                0,2          13161 /lib64/libc-2.32.so 
haveged 180 root  DEL    REG                0,2          13681 /usr/lib64/libhavege.so.2.0.0 
haveged 180 root  DEL    REG                0,2          13159 /lib64/ld-2.32.so 
haveged 180 root    0r   CHR                1,3      0t0  1030 /dev/null 
haveged 180 root    1u  unix 0xffff99680802d400      0t0 14399 type=STREAM 
haveged 180 root    2u  unix 0xffff99680802d400      0t0 14399 type=STREAM 
haveged 180 root    3u  sock                0,8      0t0 14426 protocol: UNIX 
haveged 180 root    4u   CHR                1,8      0t0  1034 /dev/random 
$ l /usr/sbin/haveged 
-rwxr-xr-x 1 root root 31496 Aug  25 15:57 /usr/sbin/haveged*

[FONT=monospace]/usr/sbin/haveged: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=0a6fd315a7ba8238afed7f83ac3ede20b1f0eae2, for GNU/Linux 3.2.0, stripped
 [/FONT]
#18

I’m looking more closely at this.

I am comparing two Tumbleweed systems (runningn in VMs). On one of them “haveged” does not show as deleted. On the other, it does show as deleted.

On the system where it shows as deleted, “haveged” is started at around 1.69 seconds from boot – I am using the output from “dmesg” to determine this. And the switch over from the temporary root (based on the initramfs) to the final root as at around 22 seconds into the boot.

On the other system, where it does not show as deleted, the switch to the permanent root is much earlier and “haveged” is started after that switch. So I think that’s the real cause of the issue.

If I remember correctly, until recently “haveged” was restarted after the switch to the permanent root. And now that restart is not happening. I’m not sure whether that should be considered a bug – or maybe we should just ignore it.

I should perhaps mention the difference between the two systems. The one where “haveged” shows as deleted is using an encrypted LVM, and the root file system is part of that LVM. So “haveged” is possibly required for the crypto before the permanent root can be mounted.

#19

haveged.service has ‘Restart=always’. What is its status?

erlangen:~ # systemctl status haveged.service 
● haveged.service - Entropy Daemon based on the HAVEGE algorithm
     Loaded: loaded (/usr/lib/systemd/system/haveged.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2020-10-12 18:21:28 CEST; 2h 40min ago
       Docs: man:haveged(8)
             http://www.issihosts.com/haveged/
   Main PID: 220 (haveged)
      Tasks: 1 (limit: 4915)
     Memory: 3.7M
     CGroup: /system.slice/haveged.service
             └─220 @usr/sbin/haveged -w 1024 -v 0 -F

Oct 12 18:21:28 erlangen haveged[220]: haveged: command socket is listening at fd 3
Warning: journal has been rotated since unit was started, output may be incomplete.
erlangen:~ # 


erlangen:~ # journalctl -b -u haveged.service --no-hostname --quiet --output short-monotonic 
    2.197343] haveged[220]: haveged: command socket is listening at fd 3
erlangen:~ #
#20

After restarted

systemctl status haveged.service 
**●** haveged.service - Entropy Daemon based on the HAVEGE algorithm 
Loaded: loaded (/usr/lib/systemd/system/haveged.service; enabled; vendor preset: enabled) 
Active: **active (running)** since Mon 2020-10-12 21:10:13 EEST; 2h 38min ago 
Docs: man:haveged(8) 
http://www.issihosts.com/haveged/ 
Main PID: 22629 (haveged) 
Tasks: 1 (limit: 4915) 
Memory: 3.6M 
CGroup: /system.slice/haveged.service 
└─22629 /usr/sbin/haveged -w 1024 -v 0 -F

It seems that the original file is deleted and another took its place.