Receiving router log information in rsyslog?

martinprowe · January 29, 2014, 11:16am

Hi,

I would like to receive log information from my router at my SuSE 13.1 64bit workstation.
I have been able to turn on the router log output and I can see the TCP packages from it, using Wireshark.

However, up to now I have not been able to receive this data at my workstation.
Wireshark says that there is no open port at my rsyslog destination.
This is confirmed with Zenmap - I have no port (514) open on my workstation.

I have tried reading/editing my /etc/rsyslog/remote.conf - with no success!
I have no firewall or AppArmour running on this lab net.

Regards, Martin

robin_listas · January 29, 2014, 12:28pm

On 2014-01-29 11:26, martinprowe wrote:
>
> Hi,
>
> I would like to receive log information from my router at my SuSE 13.1
> 64bit workstation.
> I have been able to turn on the router log output and I can see the TCP
> packages from it, using Wireshark.

TCP? Not UDP?

> I have tried reading/editing my /etc/rsyslog/remote.conf - with no
> success!

You do not say what you edited.

This is how I do mine:

/etc/rsyslog.d/remote.conf


# UDP Syslog Server:
$ModLoad imudp.so         # provides UDP syslog reception
##$UDPServerAddress 10.10.0.1 # force to listen on this IP only,
##                            # needs SYSLOG_REQUIRES_NETWORK=yes.
$UDPServerRun 514         # start a UDP syslog server at
# standard port 514

–
Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

martinprowe · January 29, 2014, 2:53pm

Hi Robin,

TCP/UDP? I wouldn’t mind either. Once I have something working, I can fine tune later. However, I think I am seeing TCP conversations in Wireshark.
The router is using the DD_WRT code.
So I have been trying to enable both in /etc/rsyslog/remote.conf. I have not seen any clues to suggest that they are mutually exclusive?

remote.conf? Yep, mine is just like yours. But still no go? Do I need to do anything else?

The only additional information I can think of adding is the (very limited) information I am see in the log files.

/var/log/messages: When I stop and restart the rsyslog service.

2014-01-29T13:30:10.993487+00:00 gzunder systemd[1]: Stopping System Logging Service...
2014-01-29T13:30:10.994873+00:00 gzunder systemd[1]: Stopped System Logging Service.
2014-01-29T13:30:12.013319+00:00 gzunder systemd[1]: [/usr/lib/systemd/system/klog.service:26] Unknown lvalue 'Names' in section 'Unit'
2014-01-29T13:30:31.283658+00:00 gzunder systemd[1]: [/usr/lib/systemd/system/klog.service:26] Unknown lvalue 'Names' in section 'Unit'
2014-01-29T13:30:52.149803+00:00 gzunder systemd[1]: Reloading.

Does that tell us anything?

Oh and there’s this:

gzunder:~ # lsmod | grep ^i
iptable_nat            13011  0 
iptable_mangle         12695  0 
iptable_filter         12810  0 
ip_tables              27239  3 iptable_nat,iptable_mangle,iptable_filter
iTCO_wdt               13480  0 
iTCO_vendor_support    13718  1 iTCO_wdt
i7core_edac            28216  0 
i2c_i801               22444  0 
gzunder:~ #

No imudp.so or imtcp.so?

Regards, Martin

robin_listas · January 29, 2014, 4:43pm

On 2014-01-29 14:56, martinprowe wrote:
>
> Hi Robin,
>
> -TCP/UDP?- I wouldn’t mind either. Once I have something working, I
> can fine tune later. However, I think I am seeing TCP conversations in
> Wireshark.

Curious. Syslog is normally UDP. It is your router who decides.

> The router is using the DD_WRT code.
> So I have been trying to enable both in /etc/rsyslog/remote.conf. I have
> not seen any clues to suggest that they are mutually exclusive?

Dunno.

> -remote.conf?- Yep, mine is just like yours. But still no go? Do I
> need to do anything else?

In “/etc/rsyslog.conf” there is this comment:


## === When you're using remote logging, enable on-disk queues ===
## === in rsyslog.d/remote.conf. When neccesary also set the   ===
## === SYSLOG_REQUIRES_NETWORK=yes in /etc/sysconfig/syslog,   ===
## === e.g. when rsyslog has to receive on a specific IP only. ===

12.3 removed that entry, but rsyslog still says it is needed. Go figure.

You also need entries to pick messages from the router and write them
somewhere. In my case:


if      ($source == 'router') then \
-/var/log/router
& ~

which works because my dns server does reverse resolution on the router IP.

But I don’t think that’s your issue yet.

The only additional information I can think of adding is the (very
limited) information I am see in the log files.

-/var/log/messages:- When I stop and restart the rsyslog service.

Code:
2014-01-29T13:30:10.993487+00:00 gzunder systemd[1]: Stopping System Logging Service...
2014-01-29T13:30:10.994873+00:00 gzunder systemd[1]: Stopped System Logging Service.
2014-01-29T13:30:12.013319+00:00 gzunder systemd[1]: [/usr/lib/systemd/system/klog.service:26] Unknown lvalue ‘Names’ in section ‘Unit’
2014-01-29T13:30:31.283658+00:00 gzunder systemd[1]: [/usr/lib/systemd/system/klog.service:26] Unknown lvalue ‘Names’ in section ‘Unit’
2014-01-29T13:30:52.149803+00:00 gzunder systemd[1]: Reloading.

Does that tell us anything?

No, that’s irrelevant. I guess that systemd changed some syntax; some
units use “Names” and apparently it is not recognized.

What says this:


systemctl status rsyslog.service

–
Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

martinprowe · January 29, 2014, 5:29pm

In “/etc/rsyslog.conf” there is this comment:

=== When you’re using remote logging, enable on-disk queues ===

=== in rsyslog.d/remote.conf. When neccesary also set the ===

=== SYSLOG_REQUIRES_NETWORK=yes in /etc/sysconfig/syslog, ===

=== e.g. when rsyslog has to receive on a specific IP only. ===

Yep. Seen it. And tried it. No difference!

You also need entries to pick messages from the router and write them
somewhere. In my case:

if ($source == ‘router’) then
-/var/log/router
& ~

Is that in /etc/rsyslog.conf?

But I don’t think that’s your issue yet.

No. I agree. I need to get imudp.so loaded and a port open before I can worry about where to put the incoming info!
Between these overs, I have just done a fresh install of 13.1 64bit, and there must be more than just editing remote.conf - because this new build doesn’t open port 514 either?

What says this:

systemctl status rsyslog.service

gzunder:~ # systemctl status rsyslog.service
rsyslog.service - System Logging Service
   Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled)
   Active: active (running) since Wed 2014-01-29 13:30:52 GMT; 2h 33min ago
  Process: 6989 ExecStartPre=/usr/sbin/rsyslog-service-prepare (code=exited, status=0/SUCCESS)
 Main PID: 6993 (rsyslogd)
   CGroup: /system.slice/rsyslog.service
           └─6993 /usr/sbin/rsyslogd -n

Jan 29 13:30:52 gzunder systemd[1]: Started System Logging Service.
Jan 29 13:34:28 gzunder systemd[1]: Started System Logging Service.
Jan 29 14:06:28 gzunder systemd[1]: Started System Logging Service.
gzunder:~ #

Also, I have just renamed /etc/rsyslog.d/remote.conf in the hope that if I stop/start rsyslog, it will throw an error. Thus proving that remote.conf is being read.
But no. Thus, I’m wondering that the reason that the additional modules (imudp.so & imtcp.so) are not being loaded is that remote.conf is not being read?

Significant? How is remote.conf invoked?

Regards, Martin

nrickert · January 29, 2014, 5:44pm

I do the same as Carlos. That is, I uncomment two lines in “remote.conf”.

I also have to modify the firewall to allow UDP port 514. That with the “Allowed Services” option of the firewall settings. I have to click the “Advanced” button to allow port 514.

robin_listas · January 29, 2014, 9:28pm

On 2014-01-29 17:36, martinprowe wrote:

> Is that in /etc/rsyslog.conf?

Yes.

>> But I don’t think that’s your issue yet.
>
> No. I agree. I need to get imudp.so loaded and a port open before I can
> worry about where to put the incoming info!
> Between these overs, I have just done a fresh install of 13.1 64bit, and
> there must be more than just editing remote.conf - because this new
> build doesn’t open port 514 either?

Huh?

I tried:


nmap -sS -sU -T4 -A localhost"
....
514/udp   open|filtered syslog

So it is open on my 12.3. I’m not using it on 13.1 (yet)

>> What says this:

> Code:
> --------------------
> gzunder:~ # systemctl status rsyslog.service
> rsyslog.service - System Logging Service
> Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled)
> Active: active (running) since Wed 2014-01-29 13:30:52 GMT; 2h 33min ago
> Process: 6989 ExecStartPre=/usr/sbin/rsyslog-service-prepare (code=exited, status=0/SUCCESS)
> Main PID: 6993 (rsyslogd)
> CGroup: /system.slice/rsyslog.service
> └─6993 /usr/sbin/rsyslogd -n
>
> Jan 29 13:30:52 gzunder systemd[1]: Started System Logging Service.
> Jan 29 13:34:28 gzunder systemd[1]: Started System Logging Service.
> Jan 29 14:06:28 gzunder systemd[1]: Started System Logging Service.
> gzunder:~ #
> --------------------

Well, so it is running.

> Also, I have just renamed /etc/rsyslog.d/remote.conf in the hope that if
> I stop/start rsyslog, it will throw an error. Thus proving that
> remote.conf is being read.
> But no. Thus, I’m wondering that the reason that the additional modules
> (imudp.so & imtcp.so) are not being loaded is that remote.conf is not
> being read?

I doubt it. You can insert a syntax error instead.

> Significant? How is remote.conf invoked?

I don’t remember…

Another thing to consider is apparmour interfering.

–
Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

martinprowe · January 30, 2014, 12:16pm

Hi again,

Okay, after sleeping on this problem and checking your advice, my state of play is this;

Clean install of SuSE/KDE 13.1 from openSUSE-13.1-DVD-x86_64.iso. Add Packman repo and full upgrade.
Disable Firewall and AppArmour using YaST2.

gzunder:~ # iptables -L -n -v
Chain INPUT (policy ACCEPT 37392 packets, 12M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 43556 packets, 5784K bytes)                                          
 pkts bytes target     prot opt in     out     source               destination                  
gzunder:~ #

Uncomment two lines in /etc/rsyslog.d/remote.conf

# ######### Receiving Messages from Remote Hosts ########## 
# TCP Syslog Server:
# provides TCP syslog reception and GSS-API (if compiled to support it)
#$ModLoad imtcp.so             # mpr load module
#$UDPServerAddress 192.168.0.1 # mpr 10.10.0.1 # force to listen on this IP only,
#                            # needs SYSLOG_REQUIRES_NETWORK=yes.
#$InputTCPServerRun 514        # mpr <port> # Starts a TCP server on selected port

# UDP Syslog Server:
$ModLoad imudp.so                    # edited by mpr ## provides UDP syslog reception
#$UDPServerAddress 192.168.0.1 #edited by mpr ## force to listen on this IP only,
##                                              # needs SYSLOG_REQUIRES_NETWORK=yes.
$UDPServerRun 514                    # edited by mpr ##start a UDP syslog server at standard port 514

No errors that I can find when starting/restarting rsyslog, so I assume imudp.so has loaded okay?
Any Idea how to test?

But has it worked? Is port 514 open? No…

gzunder:~ # nmap -sS -sU localhost

Starting Nmap 6.40 ( http://nmap.org ) at 2014-01-30 10:31 GMT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.0000060s latency).
Not shown: 1990 closed ports
PORT      STATE         SERVICE
25/tcp    open          smtp
111/tcp   open          rpcbind
631/tcp   open          ipp
5901/tcp  open          vnc-1
10000/tcp open          snet-sensor-mgmt
111/udp   open          rpcbind
177/udp   open          xdmcp
631/udp   open|filtered ipp
5353/udp  open|filtered zeroconf

Other udp ports are open. So I’m guessing that IF imudp.so has loaded and that there are no filters or other services blocking port 514, there MUST be a problem with 13.1? I guess I need to do a test build of 12.3.
Incidentally, have played with alternative udp ports (disables zeroconf and setting remote.conf to use 5353) with no luck.

I have no idea how to raise a bug report, so I’ll wait. I guess it will get fixed sometime? If anyone running 13.1 can confirm, I’d appreciate it.

Regards, Martin

nrickert · January 30, 2014, 3:11pm

I don’t have “nmap” installed here. I check with:


 % netstat -u -a | grep 514
udp        0      0 0.0.0.0:514             0.0.0.0:*
udp        0      0 :::514

robin_listas · January 30, 2014, 4:03pm

On 2014-01-30 15:16, nrickert wrote:

> I don’t have “nmap” installed here. I check with:
>
> Code:
> --------------------
>
> % netstat -u -a | grep 514
> udp 0 0 0.0.0.0:514 0.0.0.0:*
> udp 0 0 :::514
>
> --------------------

On my 12.3, where remote logging is working, that command produces zero
output. Weird. Ah, no, it appears as:


udp        0      0 *:syslog                *:*

–
Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

nrickert · January 30, 2014, 10:22pm

That looks right. I have “netstat” aliased to “netstat -n”, so I get the “514” rather than “syslog”.

I’ve used that alias for so many years, I didn’t think about it.

martinprowe · January 31, 2014, 5:05pm

Hi again,

Another day - more information …

Clean install of 12.3 (using openSUSE-12.3-DVD-x86_64.iso) - After editing remote.conf. Works as expected.
Clean install of 13.1 (using openSUSE-13.1-DVD-x86_64.iso) - Does not work!!
Upgrade 12.3 to 13.1 (using above DVD images) - Works!!

Question: nrickett - did you upgrade to 13.1?

Best regards, Martin

robin_listas · January 31, 2014, 5:28pm

On 2014-01-31 17:06, martinprowe wrote:
>
> Hi again,
>
> Another day - more information …
>
> 1. Clean install of 12.3 (using openSUSE-12.3-DVD-x86_64.iso) - After
> editing remote.conf. Works as expected.
>
> 2. Clean install of 13.1 (using openSUSE-13.1-DVD-x86_64.iso) - Does
> not work!!
>
> 3. Upgrade 12.3 to 13.1 (using above DVD images) - Works!!

Then compare all the config files, between the two 13.1 systems.

I don’t have a 13.1 install with remote logging running. I intended to
do it in an old laptop which I used as 24*7 server, but it just died.

–
Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

martinprowe · January 31, 2014, 5:55pm

Oh … I wish I could think of a witty repost to that Robin!

Like I said earlier, I guess that someone cleverer than I, will fix it sooner or later?
In the meantime, it is quicker for me to throw together a 12.3 VirtualBox to watch those logs …

Thank you all for your help, I think that it closes it for me.

Best regards, Martin

robin_listas · January 31, 2014, 6:14pm

On 2014-01-31 17:56, martinprowe wrote:
>
> robin_listas;2620990 Wrote:
>> Then compare all the config files, between the two 13.1 systems.
>
> Oh … I wish I could think of a witty repost to that Robin!

It is not that difficult, it is just one directory and one file
(rsyslog.d and rsyslog.conf).

> Like I said earlier, I guess that someone cleverer than I, will fix it
> sooner or later?

Maybe.

–
Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

nrickert · January 31, 2014, 6:16pm

Yes, I am running 13.1, installed from the 64bit DVD image (but “burned” to a USB).

Incidentally, you misspelled my name (not offended, just mentioning in case others wonder).

martinprowe · February 1, 2014, 1:22pm

Opps… I am sorry. Just gross carelessness…:shame:

But, if I may, can I clarify the above statement?
Did you reformat your disc and install 13.1 or did you select upgrade when you booted from the 13.1(DVD/USB) install media?
The upgrade route, I think, keeps all your apps and configs and just changes the repos and installs latest code.

Robin: I have checked the two rsyslog.conf - no difference. But that does not surprise me. My problem runs deeper than that?
Something (permissions/conflicts?) is stopping the required socket from opening (even if the rsyslog config asks for it).

Best regards, Martin

robin_listas · February 1, 2014, 2:18pm

On 2014-02-01 13:26, martinprowe wrote:

> Robin: I have checked the two rsyslog.conf - no difference. But that
> does not surprise me. My problem runs deeper than that?
> Something (permissions/conflicts?) is stopping the required socket from
> opening (even if the rsyslog config asks for it).

I intend to upgrade my desktop machine to 13.1, and there I log router
messages, so I will see then (in about a month, perhaps less). I do not
use freshly installed 13.1 systems to try this.

I do have a small test partition with freshly installed 13.1, though.
And another virtual install. I can try the config in there, but not
actually rerouting router messages to them to try them.

–
Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

nrickert · February 1, 2014, 4:50pm

I reformatted “/” and “/boot” and did a clean install. I retained the existing “/home”. Since “rsyslog” lives entirely on the root file system, it is part of what was reformatted and reinstalled.

On “/home” I do have a small archive of changes that I make to config files. I used that to modify “remote.conf”. Based on that archive, it looks as if “remote.conf” in 13.1 is identical to what it was in whatever I was running in Aug 2012. I simply made the same changes that I had made for that earlier version.


--- remote.conf 2012/08/24 05:27:47     1.1
+++ remote.conf 2012/08/24 05:29:19     1.1.0.1
@@ -44,10 +44,10 @@
 #$InputTCPServerRun <port> # Starts a TCP server on selected port
 
 # UDP Syslog Server:
-#$ModLoad imudp.so         # provides UDP syslog reception
+$ModLoad imudp.so         # provides UDP syslog reception
 ##$UDPServerAddress 10.10.0.1 # force to listen on this IP only,
 ##                            # needs SYSLOG_REQUIRES_NETWORK=yes.
-#$UDPServerRun 514         # start a UDP syslog server at standard port 514
+$UDPServerRun 514         # start a UDP syslog server at standard port 514
 
 
 ########### Encrypting Syslog Traffic with TLS ##########

Note that copy/paste might have converted some TAB characters to spaces.

The “-” lines are the old version, the “+” lines are after my changes. The lines starting with space are there for context. And everything is indented by one character (a “+” or “-” or " "). It’s “diff -u” format.