Page 1 of 3 123 LastLast
Results 1 to 10 of 28

Thread: System takes longer to start due to "flush journal" job

  1. #1

    Default System takes longer to start due to "flush journal" job

    Following from this thread, but made a new one since issue is different indeed.

    This week I had noticed my system took a bit longer than before to start (plymouth took longer to finish).
    Last time I saw this behavior was when still doing experiments on Leap 42.3, by switching from Network Manager to Wicked network service. When hitting Esc key to switch from plymouth to console I noticed (IIRC) Wicked needed to set some "start jobs", which took some additional seconds.
    But this is no longer the case; I stick to Network Manager full time. So again I pressed Esc to plymouth and found out the process that's now taking additional seconds to start:
    Code:
    a start job is running for Flush Journal to Persistent Storage [...]
    I have no idea if Leap 42.3 does this ever since clean installation; never checked for it out of fear of pressing Esc key... But if it does, it's now taking longer to finish since this week, out of the blue...
    I had not even run updates just until today. Neither updates fixed it.

    Any ideas?
    Thanks beforehand.

  2. #2
    Join Date
    Jun 2008
    Location
    Podunk
    Posts
    32,336
    Blog Entries
    15

    Default Re: System takes longer to start due to "flush journal" job

    Hi
    Not doing or configured maintenance, maybe lots of coredumps?

    Code:
    coredumpctl list
    If there are lot's, then maybe worth investigating or ignore?

    Clean up stuff if it's high disk usage?

    Code:
    du -sh /var/log/journal/
    journalctl --vacuum-time=2d
    By default journalctl is configured to auto, maybe configure to volatile?

    See the systemd coredump.conf and journald.conf and man pages for how your wanting it to run...
    Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
    SUSE SLE, openSUSE Leap/Tumbleweed (x86_64) | GNOME DE
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below... Thanks!

  3. #3

    Default Re: System takes longer to start due to "flush journal" job

    Must coredumpctl be run as root, or list can be run as normal user?
    Do you need me to post something from coredumpctl? But don't core dumps -memory dumps- store sensitive data?
    My /var/log/journal/ directory is 161 Mb. Is this too much or very short?

  4. #4
    Join Date
    Jun 2008
    Location
    Podunk
    Posts
    32,336
    Blog Entries
    15

    Default Re: System takes longer to start due to "flush journal" job

    Quote Originally Posted by F_style View Post
    Must coredumpctl be run as root, or list can be run as normal user?
    Do you need me to post something from coredumpctl? But don't core dumps -memory dumps- store sensitive data?
    My /var/log/journal/ directory is 161 Mb. Is this too much or very short?
    Hi
    Yes, as root user, nope nothing needed here from the output, just if there are and how many?

    Have you tweaked the journald.conf file at all?

    Something may have triggered it (since it's auto) to start storing...
    Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
    SUSE SLE, openSUSE Leap/Tumbleweed (x86_64) | GNOME DE
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below... Thanks!

  5. #5

    Default Re: System takes longer to start due to "flush journal" job

    So, /var/log/journal/ directory is 161 Mb
    "coredumpctl list" output gives 9 entries, oldest dating from circa 2 weeks ago
    No, I have never ever touched journald.conf. This thread is the first time I have ever talked about this kind of stuff.

  6. #6
    Join Date
    Jun 2008
    Location
    Podunk
    Posts
    32,336
    Blog Entries
    15

    Default Re: System takes longer to start due to "flush journal" job

    Hi
    Have a read of this thread, check the logs for the flush time and look at systemd-analyze;
    https://forums.opensuse.org/showthre...g-far-too-long

    There are a few others if you use the advanced forum search.
    Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
    SUSE SLE, openSUSE Leap/Tumbleweed (x86_64) | GNOME DE
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below... Thanks!

  7. #7
    Join Date
    Sep 2012
    Posts
    7,100

    Default Re: System takes longer to start due to "flush journal" job

    Quote Originally Posted by F_style View Post
    My /var/log/journal/ directory is 161 Mb.
    Log files themselves are in subdirectory with name based on machine UUID.

    In general, systemd starts journald as soon as possible; journald starts with logs in RAM. When file systems are mounted, journald copies content from RAM to permanent storage. The only way to avoid it is to disable persistent storage, but then you lose possibility to analyze historical logs (they are lost after reboot).

    It is impossible to give blanket answer to "why it takes so long". For a start, we do not even know how long it takes and why you think it is not normal.

  8. #8

    Default Re: System takes longer to start due to "flush journal" job

    Ok, I managed to get more accurate information.

    Total plymouth screen (from when it starts to when it ends) time: 1 minute exactly.
    Time it takes since plymouth finishing to login screen ready: 21 seconds.
    Total: 1 minute 21 seconds to get to login screen since moment plymouth starts.
    Now I'm a bit more sure system does take a bit longer than circa 2 weeks ago; I knew I felt something... And all of this out of the blue without doing *any* tweaking to configurations or system updates at all.
    Code:
    journalctl -b | grep "Time spent on flushing"
    oct 16 13:06:35 linux-cmn7 systemd-journald[434]: Time spent on flushing to /var is 31.907949s for 917 entries.
    Code:
    user1@linux-cmn7:~> ls -lh /var/log/journal/<long-UUID-number>
    total 168M
    -rw-r-----  1 root systemd-journal  40M sep 27 14:11 system@<(16DigitHexNumber)-(16DigitHexNumber)>.journal~
    -rw-r-----  1 root systemd-journal 104M oct 16 13:23 system.journal
    -rw-r-----+ 1 root systemd-journal 8.0M sep 27 14:12 user-xxxx@<(16DigitHexNumber)-(16DigitHexNumber)>.journal~
    -rw-r-----+ 1 root systemd-journal  16M oct 16 13:17 user-xxxx.journal
    31 seconds to flush, W T H !?
    In journald.conf file *all* fields are commented out, meaning it's taking all defaults. So I read the respective manual page, looked for the terms SystemMaxFileSize and SystemMaxUse, and at the end of terms concluded max file size must be taking 512 Mb, given a root file system of 40 Gb.
    So the files I have are within range in this case, but aren't they still a bit large?
    By the way, I do have a strange habit of powering off and turning on system again often several times a day...

    Any ideas?
    Thanks beforehand.

  9. #9

    Default Re: System takes longer to start due to "flush journal" job

    Now I'm desperate

    journalctl --vacuum-time=1d
    it only deleted a couple of journal files but didn't free that much space (just circa ~50 Mb). Used the variant size=50M, and it does nothing!! "0B were freed for journal"
    Code:
    journalctl -b | grep "Time spent on flushing"
    oct 16 13:16:21 linux-cmn7 systemd-journald[434]: Time spent on flushing to /var is 34.216276s for 933 entries.
    Code:
    user1@linux-cmn7:~> ls -lh /var/log/journal/<long-UUID-number>
    total 121M
    -rw-r-----  1 root systemd-journal 104M oct 16 13:33 system.journal
    -rw-r-----+ 1 root systemd-journal  16M oct 16 13:27 user-xxxx.journal
    So all along it's been only logging to the same pair of files, and I cannot do anything to stop it!!?? I clearly just proved that journal logging is consuming boot time, and it increases with each boot. Can anyone help please? I don't want this!

  10. #10
    Join Date
    Jun 2008
    Location
    Podunk
    Posts
    32,336
    Blog Entries
    15

    Default Re: System takes longer to start due to "flush journal" job

    Hi
    So, you have reviewed the current logs (journalctl) to see there is nothing untoward happening on your system? You ran systemd-analyze blame and critical-chain to confirm there is nothing masking your issue since things happen in parallel?
    Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
    SUSE SLE, openSUSE Leap/Tumbleweed (x86_64) | GNOME DE
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below... Thanks!

Page 1 of 3 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •