I’m using sarg with squid3 on openSUSE 12.2
Has anyone seen or heard of this problem with sarg before?
SARG: Records in file: 428, reading: 100.00%
SARG: Records read: 428, written: 0, excluded: 0
SARG: Squid log format
SARG: No records found
SARG: End
I’ll post a configuration and system summary later today.
Hi, here’s the rest of the information as promised.
It’s as if SARG doesn’t understand the log file or can’t find any records.
I think this error is due to the fact that the information (date range) it is looking for is no longer in the log file (It’s rotated out).
Surely sarg should cater for that?
I have about four months of log files. The daily report works, but the weekly and monthly fails.
Here’s the daily run:
/usr/bin/sarg -z -x -f /etc/sarg.conf -d 27/03/2013 -o /srv/www/sarg/Daily
SARG: Init
SARG: Loading configuration from /etc/sarg.conf
SARG: TAG: access_log /var/log/squid/access.log
SARG: TAG: font_face Tahoma,Verdana,Arial
SARG: TAG: output_dir /srv/www/sarg
SARG: TAG: user_ip yes
SARG: TAG: date_format e
SARG: TAG: lastlog 90
SARG: TAG: overwrite_report yes
SARG: TAG: max_elapsed 28800000
SARG: TAG: show_successful_message no
SARG: TAG: show_read_statistics yes
SARG: TAG: www_document_root /srv/www/htdocs
SARG: TAG: download_suffix "zip,arj,bzip,gz,ace,doc,iso,adt,bin,cab,com,dot,drv$,lha,lzh,mdb,mso,ppt,rtf,src,shs,sys,exe,dll,mp3,avi,mpg,mpeg"
SARG: Deleting temporary directory "/tmp/sarg"
SARG: Parameters:
SARG: Hostname or IP address (-a) =
SARG: Useragent log (-b) =
SARG: Exclude file (-c) =
SARG: Date from-until (-d) = 27/03/2013-27/03/2013
SARG: Email address to send reports (-e) =
SARG: Config file (-f) = /etc/sarg.conf
SARG: Date format (-g) = Europe (dd/mm/yyyy)
SARG: IP report (-i) = No
SARG: Keep temporary files (-k) = No
SARG: Input log (-l) = /var/log/squid/access.log
SARG: Resolve IP Address (-n) = No
SARG: Output dir (-o) = /srv/www/sarg/Daily/
SARG: Use Ip Address instead of userid (-p) = Yes
SARG: Accessed site (-s) =
SARG: Time (-t) =
SARG: User (-u) =
SARG: Temporary dir (-w) = /tmp/sarg
SARG: Debug messages (-x) = Yes
SARG: Process messages (-z) = Yes
SARG: Previous reports to keep (--lastlog) = 90
SARG:
SARG: sarg version: 2.3.5 Jan-11-2013
SARG: Reading access log file: /var/log/squid/access.log
SARG: Records in file: 2990, reading: 100.00%
SARG: Records read: 2990, written: 2990, excluded: 0
SARG: Squid log format
SARG: Period covered by log files: 27/03/2013-27/03/2013
SARG: (info) date=27/03/2013
SARG: (info) period=27 Mar 2013
SARG: Period: 27 Mar 2013
SARG: (info) outdirname=/srv/www/sarg/Daily/27Mar2013-27Mar2013
SARG: Sorting log /tmp/sarg/127_0_0_1.user_unsort
SARG: Making file: /tmp/sarg/127_0_0_1
SARG: Sorting log /tmp/sarg/192_168_10_52.user_unsort
SARG: Making file: /tmp/sarg/192_168_10_52
SARG: Sorting log /tmp/sarg/192_168_10_55.user_unsort
SARG: Making file: /tmp/sarg/192_168_10_55
SARG: (info) Dansguardian report not produced because no dansguardian configuration file was provided
SARG: (info) No redirector logs provided to produce that kind of report
SARG: (info) Denied report not produced because it is empty
SARG: (info) Authentication failures report not produced because it is empty
SARG: (info) Redirector report not generated because it is empty
SARG: Sorting file: /tmp/sarg/127_0_0_1.utmp
SARG: Making report: 127.0.0.1
SARG: Sorting file: /tmp/sarg/192_168_10_52.utmp
SARG: Making report: 192.168.10.52
SARG: Sorting file: /tmp/sarg/192_168_10_55.utmp
SARG: Making report: 192.168.10.55
SARG: Making index.html
SARG: Purging temporary file sarg-general
SARG: End
And the weekly
/usr/bin/sarg -z -x -f /etc/sarg.conf -d 20/03/2013-26/03/2013 -o /srv/www/sarg/Weekly
SARG: Init
SARG: Loading configuration from /etc/sarg.conf
SARG: TAG: access_log /var/log/squid/access.log
SARG: TAG: font_face Tahoma,Verdana,Arial
SARG: TAG: output_dir /srv/www/sarg
SARG: TAG: user_ip yes
SARG: TAG: date_format e
SARG: TAG: lastlog 90
SARG: TAG: overwrite_report yes
SARG: TAG: max_elapsed 28800000
SARG: TAG: show_successful_message no
SARG: TAG: show_read_statistics yes
SARG: TAG: www_document_root /srv/www/htdocs
SARG: TAG: download_suffix "zip,arj,bzip,gz,ace,doc,iso,adt,bin,cab,com,dot,drv$,lha,lzh,mdb,mso,ppt,rtf,src,shs,sys,exe,dll,mp3,avi,mpg,mpeg"
SARG: Deleting temporary directory "/tmp/sarg"
SARG: Parameters:
SARG: Hostname or IP address (-a) =
SARG: Useragent log (-b) =
SARG: Exclude file (-c) =
SARG: Date from-until (-d) = 20/03/2013-26/03/2013
SARG: Email address to send reports (-e) =
SARG: Config file (-f) = /etc/sarg.conf
SARG: Date format (-g) = Europe (dd/mm/yyyy)
SARG: IP report (-i) = No
SARG: Keep temporary files (-k) = No
SARG: Input log (-l) = /var/log/squid/access.log
SARG: Resolve IP Address (-n) = No
SARG: Output dir (-o) = /srv/www/sarg/Weekly/
SARG: Use Ip Address instead of userid (-p) = Yes
SARG: Accessed site (-s) =
SARG: Time (-t) =
SARG: User (-u) =
SARG: Temporary dir (-w) = /tmp/sarg
SARG: Debug messages (-x) = Yes
SARG: Process messages (-z) = Yes
SARG: Previous reports to keep (--lastlog) = 90
SARG:
SARG: sarg version: 2.3.5 Jan-11-2013
SARG: Reading access log file: /var/log/squid/access.log
SARG: Records in file: 3006, reading: 100.00%
SARG: Records read: 3006, written: 0, excluded: 0
SARG: Squid log format
SARG: No records found
SARG: End
Here’s my system’s information
cat /etc/SuSE-release
openSUSE 12.2 (x86_64)
VERSION = 12.2
CODENAME = Mantis
uname -a
Linux xxxxxxxxxx 3.4.33-2.24-default #1 SMP Tue Feb 26 03:34:33 UTC 2013 (5f00a32) x86_64 x86_64 x86_64 GNU/Linux
free -m
total used free shared buffers cached
Mem: 15998 1980 14018 0 26 438
-/+ buffers/cache: 1515 14483
Swap: 2007 0 2007
grep -vE ‘^$|^#’ /etc/sarg.conf
access_log /var/log/squid/access.log
font_face Tahoma,Verdana,Arial
output_dir /srv/www/sarg
user_ip yes
date_format e
lastlog 90
overwrite_report yes
max_elapsed 28800000
show_successful_message no
show_read_statistics yes
www_document_root /srv/www/htdocs
download_suffix "zip,arj,bzip,gz,ace,doc,iso,adt,bin,cab,com,dot,drv$,lha,lzh,mdb,mso,ppt,rtf,src,shs,sys,exe,dll,mp3,avi,mpg,mpeg"
zypper info squid
Information for package squid:
Repository: openSUSE-12.2-Oss
Name: squid
Version: 2.7.STABLE9-9.1.3
Arch: x86_64
Vendor: openSUSE
Installed: No
Status: not installed
Installed Size: 4.5 MiB
Summary: Squid WWW proxy server
Description:
The stable version of the Squid WWW Proxy Server.
Home page: http://www.squid-cache.org
I updated sarg to a new version available for 12.2
zypper info sarg
Information for package sarg:
Repository: openSUSE-12.2-home:elbuffo_sarg
Name: sarg
Version: 2.3.5-15.1
Arch: x86_64
Vendor: obs://build.opensuse.org/home:elbuffo
Installed: Yes
Status: up-to-date
Installed Size: 1.5 MiB
Summary: Squid Analysis Report Generator
Description:
Sarg -- Squid Analysis Report Generator is a tool that allows you to
view "where" your users are going to on the Internet. Sarg generate
reports in html, with fields such as: users, IP Addresses, bytes,
sites, and times.
Thanks
This confirms it, sarg does not find the data.
mkdir /tmp/squid
cp /var/log/squid/access.log /tmp/squid/
cp /var/log/squid/access.log-2013032* /tmp/squid/
cd /tmp/squid/
xz -d access.log*.xz
cat access.log* | sort > access.log_new
/usr/bin/sarg -z -x -f /etc/sarg.conf -d 20/03/2013-26/03/2013 -o /srv/www/sarg/Weekly -l /tmp/squid/access.log_new
SARG: Init
SARG: Loading configuration from /etc/sarg.conf
SARG: TAG: access_log /var/log/squid/access.log
SARG: TAG: font_face Tahoma,Verdana,Arial
SARG: TAG: output_dir /srv/www/sarg
SARG: TAG: user_ip yes
SARG: TAG: date_format e
SARG: TAG: lastlog 90
SARG: TAG: overwrite_report yes
SARG: TAG: max_elapsed 28800000
SARG: TAG: show_successful_message no
SARG: TAG: show_read_statistics yes
SARG: TAG: www_document_root /srv/www/htdocs
SARG: TAG: download_suffix "zip,arj,bzip,gz,ace,doc,iso,adt,bin,cab,com,dot,drv$,lha,lzh,mdb,mso,ppt,rtf,src,shs,sys,exe,dll,mp3,avi,mpg,mpeg"
SARG: Deleting temporary directory "/tmp/sarg"
SARG: Parameters:
SARG: Hostname or IP address (-a) =
SARG: Useragent log (-b) =
SARG: Exclude file (-c) =
SARG: Date from-until (-d) = 20/03/2013-26/03/2013
SARG: Email address to send reports (-e) =
SARG: Config file (-f) = /etc/sarg.conf
SARG: Date format (-g) = Europe (dd/mm/yyyy)
SARG: IP report (-i) = No
SARG: Keep temporary files (-k) = No
SARG: Input log (-l) = /tmp/squid/access.log_new
SARG: Resolve IP Address (-n) = No
SARG: Output dir (-o) = /srv/www/sarg/Weekly/
SARG: Use Ip Address instead of userid (-p) = Yes
SARG: Accessed site (-s) =
SARG: Time (-t) =
SARG: User (-u) =
SARG: Temporary dir (-w) = /tmp/sarg
SARG: Debug messages (-x) = Yes
SARG: Process messages (-z) = Yes
SARG: Previous reports to keep (--lastlog) = 90
SARG:
SARG: sarg version: 2.3.5 Jan-11-2013
SARG: Reading access log file: /tmp/squid/access.log_new
SARG: Records in file: 75923, reading: 100.00%
SARG: Records read: 75923, written: 65303, excluded: 2
SARG: Squid log format
SARG: Period covered by log files: 20/03/2013-26/03/2013
SARG: (info) date=28/03/2013
SARG: (info) period=20 Mar 2013-26 Mar 2013
SARG: Period: 20 Mar 2013-26 Mar 2013
SARG: (info) outdirname=/srv/www/sarg/Weekly/20Mar2013-26Mar2013
SARG: Sorting log /tmp/sarg/127_0_0_1.user_unsort
SARG: Making file: /tmp/sarg/127_0_0_1
SARG: Sorting log /tmp/sarg/xxxxxxxxxxxxxxxxxxxx.user_unsort
SARG: Making file: /tmp/sarg/xxxxxxxxxxxxxxxxxxxx
SARG: Sorting log /tmp/sarg/xxxxxxxxxxxxxxxxxxxx.user_unsort
SARG: Making file: /tmp/sarg/xxxxxxxxxxxxxxxxxxxx
SARG: Sorting log /tmp/sarg/xxxxxxxxxxxxxxxxxxxx.user_unsort
SARG: Making file: /tmp/sarg/xxxxxxxxxxxxxxxxxxxx
SARG: (info) Dansguardian report not produced because no dansguardian configuration file was provided
SARG: (info) No redirector logs provided to produce that kind of report
SARG: (info) Denied report not produced because it is empty
SARG: (info) Authentication failures report not produced because it is empty
SARG: (info) Redirector report not generated because it is empty
SARG: Sorting file: /tmp/sarg/127_0_0_1.utmp
SARG: Making report: 127.0.0.1
SARG: Sorting file: /tmp/sarg/xxxxxxxxxxxxxxxxxxxx.utmp
SARG: Making report: xxxxxxxxxxxxxxxxxxxx
SARG: Sorting file: /tmp/sarg/xxxxxxxxxxxxxxxxxxxx.utmp
SARG: Making report: xxxxxxxxxxxxxxxxxxxx
SARG: Sorting file: /tmp/sarg/xxxxxxxxxxxxxxxxxxxx.utmp
SARG: Making report: xxxxxxxxxxxxxxxxxxxx
SARG: Making index.html
SARG: Purging temporary file sarg-general
SARG: End
So either sarg has to understand what the OS is doing, which I doubt they will do since there are just too many different ones out there and it falls outside the scope of the application. Or the OS has to hand sarg the log files on a silver platter.
Looking at /usr/sbin/sarg-reports you can see that no such provision is made
weekly ()
{
WEEKLYOUT=$HTMLOUT/$WEEKLY
mkdir -p $WEEKLYOUT
create_index_html
$SARG -f $CONFIG -d $WEEKAGO-$YESTERDAY -o $WEEKLYOUT >$ERRORS 2>&1
exclude_from_log
}
GrEaT!! Not feeling like hacking the script at the moment.
If I had to fix it (which I might have to), I’ll grab the log location from the sarg config file, string the relevant ones together and send it off to sarg with the -l parameter as above.
Hopefully someone at SUSE does that and rolls out a fix first, or maybe it’s fixed in 12.3 and I can pillage that file.