Rsyslog repeating messages thousands of times

This one has me stumped.

I’m running Leap 15.1 in a small site having 4-5 VMs with one being the centralized rsyslog server.
Everything was fine until I upgraded the rsyslog server to Leap 15.3 and then rsyslog lots its mind.
It’s configuration files were preserved through the 15.1 dup → 15.2 dup → 15.3 upgrade
and switch over to new “sle” and “backport” repos.

After the upgrades I noticed that rsyslog was still receiving logs from the other servers but it was logging
at a furious rate. Since the logging VM was down for a few hours during the upgrades I thought that other
servers were catching up on backlogged messages but I eventually determined that messages were being
repeated.

I determined that the problem was with the upgraded rsyslog server itself by shutting down one of the
servers that logs to it and still its messages were being logged… and not only that but the very same message
including timestamp, server name, application/app pid (eg. CRON[]) and message were being
repeated. In the case of the cron example, a single message was repeated over 12,000 times after the
source of the message was shutdown.

I had to stop and disable rsyslogd because it wrote 10Gb of logs in about an hour.
I thought that someone might have seen this before but I haven’t found a similar thread yet.

Darren

I thought that someone might have seen this before but I haven’t found a similar thread yet.

Set ‘$RepeatedMsgReduction on’ in rsyslog.conf and enable rsyslog again.

Thank you for responding.
This is on my default and confirmed on in my instance.

rsyslog.service was disabled over the holidays and when I tried to enable/start it found that it would not start
until I fixed 2 dependent services; var-run.mount (/var/run needs to be a directory and not a symlink)
and syslog.socket’s “[Socket]” section needed “Service=syslog” before rsyslog.service would start.

It ran amok for a while but eventually settled down and seems to be back to normal.