Page 2 of 2 FirstFirst 12
Results 11 to 12 of 12

Thread: journalctl slow response time...

  1. #11
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,278
    Blog Entries
    2

    Default Re: journalctl slow response time...

    Quote Originally Posted by robin_listas View Post
    <snip>

    I can not do any testing myself, because I use syslog instead for long
    term logs; I only allow systemd to keep the journal of the current
    session, and even then it is big:

    Code:
    Telcontar:~ # journalctl --disk-usage
    Journals take up 408.3M on disk.
    Telcontar:~ #
    
    cer@Telcontar:~> uptime
    15:27  up 12 days 18:55,  72 users,  load average: 0,83, 0,57, 0,60
    cer@Telcontar:~>
    
    Telcontar:~ # time journalctl > logs
    
    real    1m14.254s
    user    0m9.986s
    sys     0m2.741s
    Telcontar:~ # ls -lh logs
    -rw-r--r-- 1 root root 44M Dec 25 15:28 logs
    Telcontar:~ #
    A second run is faster:

    Telcontar:~ # time journalctl -l > logs

    Code:
    real    0m11.065s
    user    0m9.680s
    sys     0m1.372s
    Telcontar:~ # ls -lh logs
    -rw-r--r-- 1 root root 44M Dec 25 15:31 logs
    Telcontar:~ #
    which indicates that the main time factor is the hard disk reading. If I
    had my 2 year log in the journal, it would be terribly big and slow
    (consider: mine has 400 MB in 12 days).

    Still the text dump is an order of magnitude smaller than the journal.



    --
    Cheers / Saludos,

    Carlos E. R.
    (from 13.1 x86_64 "Bottle" at Telcontar)
    Carlos,
    You fall in that category of extremely large journals (Yours reads over 400MB).
    In the links I posted, you can restrict size to something much smaller (25MB is probably very small, something around 100MB might be a max but YMMV). I don't know that keeping 400MB of logs is useful (but again, everyone has their own reasons to keep every logfile since installation).

    That said, there are some native equivalents of tail and head which should be used which might provide better performance
    Code:
    # The following initially prints the most recent 10 lines so is equivalent to tail, but will also continuously update
    journalctl --follow
    # Displays the specified time frame
    journalctl --since=-1day
    # Similar to "--since" but display entries up to a specified date
    journalctl --until=2014-12-25
    or
    journalctl --until=now
    Since I'm not experiencing a slow journalctl, I can't test whether the above commands fixes the problem.

    You'll also find in the links I posted (Linux Pottering's comments) why the legacy syslog returns data quickly... The full syslog is stored as a series of smaller files so parsing only the relevant small logfile is quick.

    TSU

  2. #12
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: journalctl slow response time...

    On 2014-12-26 17:56, tsu2 wrote:
    >
    > robin_listas;2685118 Wrote:



    >> which indicates that the main time factor is the hard disk reading. If I
    >> had my 2 year log in the journal, it would be terribly big and slow
    >> (consider: mine has 400 MB in 12 days).
    >>
    >> Still the text dump is an order of magnitude smaller than the journal.


    > Carlos,
    > You fall in that category of extremely large journals (Yours reads over
    > 400MB).
    > In the links I posted, you can restrict size to something much smaller
    > (25MB is probably very small, something around 100MB might be a max but
    > YMMV). I don't know that keeping 400MB of logs is useful (but again,
    > everyone has their own reasons to keep every logfile since
    > installation).


    No, I don't have a need for them, because what I use is syslog, and it is much smaller, covering months of logs.

    What you see is the current session logs, the runtime, which are automatically deleted on reboot. And it is large because systemd is keeping nntp and mail debug logs; I know how to prune them from syslog (rotated out), but I don't know how to selectively prune systemd log records.

    The other problem is that the runtime logs do not seem to be compacted, as the dump to text is much smaller.


    > That said, there are some native equivalents of tail and head which
    > should be used which might provide better performance


    That will be interesting for the OP.


    > Since I'm not experiencing a slow journalctl, I can't test whether the
    > above commands fixes the problem.
    >
    > You'll also find in the links I posted (Linux Pottering's comments) why
    > the legacy syslog returns data quickly... The full syslog is stored as a
    > series of smaller files so parsing only the relevant small logfile is
    > quick.


    LOL.


    Look:


    Code:
    Telcontar:~ # ls /var/log/messages*
    /var/log/messages               /var/log/messages-20140301.xz  /var/log/messages-20140506.xz  /var/log/messages-20140907.xz
    /var/log/messages-20120816.bz2  /var/log/messages-20140305.xz  /var/log/messages-20140525.xz  /var/log/messages-20141001.xz
    /var/log/messages-20131205.xz   /var/log/messages-20140402.xz  /var/log/messages-20140629.xz  /var/log/messages-20141028.xz
    /var/log/messages-20140123.xz   /var/log/messages-20140411.xz  /var/log/messages-20140726.xz  /var/log/messages-20141130.xz
    Telcontar:~ # time zgrep "Restarting kernel threads" /var/log/messages*xz
    /var/log/messages-20131205.xz:<0.4> 2013-11-05 11:22:32 Telcontar kernel - - - [112002.376799] Restarting kernel threads ... done.
    /var/log/messages-20131205.xz:<0.4> 2013-11-05 19:14:00 Telcontar kernel - - - [130783.602403] Restarting kernel threads ... done.
    ....
    /var/log/messages-20141130.xz:<0.4> 2014-11-29 21:55:33 Telcontar kernel - - - [921439.104283] Restarting kernel threads ... done.
    /var/log/messages-20141130.xz:<0.4> 2014-11-30 14:13:34 Telcontar kernel - - - [941075.361466] Restarting kernel threads ... done.
    real    0m0.865s
    user    0m0.443s
    sys     0m0.115s
    Telcontar:~ #

    As you see, running a "query" on a year of syslog logs took less than a second - and it is not cached.
    Overall, a much more efficient method ;-)


    --
    Cheers / Saludos,

    Carlos E. R.
    (from 13.1 x86_64 "Bottle" at Telcontar)

Page 2 of 2 FirstFirst 12

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •