CUPS Printers on Virtual Machines worked Monday do not work Today

When I print on any virtual machine, the printers shared on my servers used to appear on the print choices.

Today - every Virtual Machine (all OpenSUSE 15.2) the only printer is “Print to File”

Anyone else see this issue.

It worked Monday - the last time I printed. I tried to day - nada - restarted cups on the VM and the server - no difference. Even rebooted to see if that is the reason.

I never had to add a printer to the VM’s for the printers to appear - CUPS seemed to do that on its startup.

My zypper update logs show both ldap and devicemapper got upgraded since I printed last - are they the issue?

Not a firewall issue?

no firewall - I use iptables - no changes to it in 2 years. The server and Virtual Machines were not booted since the last kernel update 6 days ago when the printers disappeared.

No other server (all have hard coded ip address) can see the printers too but the servers with printers attached can see all the printers.

Well iptables is the user end to the Linux firewall, but as you claim it has not been changed I guess we can rule that out.

The server and Virtual Machines were not booted since the last kernel update 6 days ago when the printers disappeared.

No other server (all have hard coded ip address) can see the printers too but the servers with printers attached can see all the printers.

Right, so check cupsd on the server…

sudo lsof -i:631

Config not changed for some reason?

egrep -i "listen|port" /etc/cups/cupsd.conf

This might be useful to review with respect to printer sharing
https://wiki.archlinux.org/index.php/CUPS/Printer_sharing#Printer_sharing

I did what you suggested on the server with the printer, a server without any printer and a virtual machine

Server

LLR1:~ # lsof -i:631
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
cupsd   1057 root    7u  IPv4  27415      0t0  TCP *:ipp (LISTEN)
cupsd   1057 root    8u  IPv6  27416      0t0  TCP *:ipp (LISTEN)
LLR1:~ # egrep -i "listen|port" /etc/cups/cupsd.conf
#Port 631
Listen *:631
Listen localhost:631
Listen /run/cups/cups.sock
# 2018-08-12,20:33:36 modify_cupsd_conf Listen localhost
# 2018-08-12,20:43:06 modify_cupsd_conf Listen all
LLR1:~ # ll /etc/cups/cupsd.conf
-rw-r----- 1 root lp 3746 Aug 12  2018 /etc/cups/cupsd.conf
LLR1:~ # cat /etc/cups/cupsd.conf
LogLevel normal
# Allow remote access
#Port 631
Listen *:631
Listen localhost:631
Listen /run/cups/cups.sock
# Share local printers on the local network.
Browsing On
BrowseLocalProtocols dnssd
DefaultAuthType Basic
WebInterface Yes
<Location />
Allow @LOCAL
Order allow,deny
Allow 127.0.0.2
</Location>
<Location /admin>
  Order allow,deny
</Location>
<Location /admin/conf>
  AuthType Default
  Require user @SYSTEM
  Order allow,deny
</Location>
<Policy default>
  JobPrivateAccess default
  JobPrivateValues default
  SubscriptionPrivateAccess default
  SubscriptionPrivateValues default
  <Limit Create-Job Print-Job Print-URI Validate-Job>
    Order deny,allow
  </Limit>
  <Limit Send-Document Send-URI Hold-Job Release-Job Restart-Job Purge-Jobs Set-Job-Attributes Create-Job-Subscription Renew-Subscription Cancel-Subscription Get-Notifications Reprocess-Job Cancel-Current-Job Suspend-Current-Job Resume-Job Cancel-My-Jobs Close-Job CUPS-Move-Job CUPS-Get-Document>
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>
  <Limit CUPS-Add-Modify-Printer CUPS-Delete-Printer CUPS-Add-Modify-Class CUPS-Delete-Class CUPS-Set-Default CUPS-Get-Devices>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>
  <Limit Pause-Printer Resume-Printer Enable-Printer Disable-Printer Pause-Printer-After-Current-Job Hold-New-Jobs Release-Held-New-Jobs Deactivate-Printer Activate-Printer Restart-Printer Shutdown-Printer Startup-Printer Promote-Job Schedule-Job-After Cancel-Jobs CUPS-Accept-Jobs CUPS-Reject-Jobs>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>
  <Limit Cancel-Job CUPS-Authenticate-Job>
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>
  <Limit All>
    Order deny,allow
  </Limit>
</Policy>
<Policy authenticated>
  JobPrivateAccess default
  JobPrivateValues default
  SubscriptionPrivateAccess default
  SubscriptionPrivateValues default
  <Limit Create-Job Print-Job Print-URI Validate-Job>
    AuthType Default
    Order deny,allow
  </Limit>
  <Limit Send-Document Send-URI Hold-Job Release-Job Restart-Job Purge-Jobs Set-Job-Attributes Create-Job-Subscription Renew-Subscription Cancel-Subscription Get-Notifications Reprocess-Job Cancel-Current-Job Suspend-Current-Job Resume-Job Cancel-My-Jobs Close-Job CUPS-Move-Job CUPS-Get-Document>
    AuthType Default
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>
  <Limit CUPS-Add-Modify-Printer CUPS-Delete-Printer CUPS-Add-Modify-Class CUPS-Delete-Class CUPS-Set-Default>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>
  <Limit Pause-Printer Resume-Printer Enable-Printer Disable-Printer Pause-Printer-After-Current-Job Hold-New-Jobs Release-Held-New-Jobs Deactivate-Printer Activate-Printer Restart-Printer Shutdown-Printer Startup-Printer Promote-Job Schedule-Job-After Cancel-Jobs CUPS-Accept-Jobs CUPS-Reject-Jobs>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>
  <Limit Cancel-Job CUPS-Authenticate-Job>
    AuthType Default
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>
  <Limit All>
    Order deny,allow
  </Limit>
</Policy>
<Policy allowallforanybody>
  JobPrivateAccess all
  JobPrivateValues none
  SubscriptionPrivateAccess all
  SubscriptionPrivateValues none
  <Limit All Validate-Job Cancel-Jobs Cancel-My-Jobs Close-Job CUPS-Get-Document>
    Order deny,allow
    Allow from all
  </Limit>
</Policy>
DefaultPolicy default
# 2018-08-12,20:33:36 modify_cupsd_conf Listen localhost
# 2018-08-12,20:43:04 modify_cupsd_conf Allow @LOCAL
# 2018-08-12,20:43:05 modify_cupsd_conf Browsing On
# 2018-08-12,20:43:06 modify_cupsd_conf Listen all
# 2018-08-12,21:40:34 modify_cupsd_conf Browsing On
LLR1:~ # 

Server no printer

LLR2:~ #  lsof -i:631
COMMAND PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
cupsd   926 root    6u  IPv6  24914      0t0  TCP localhost:ipp (LISTEN)
cupsd   926 root    7u  IPv4  24915      0t0  TCP localhost:ipp (LISTEN)
LLR2:~ # egrep -i "listen|port" /etc/cups/cupsd.conf
# Only listen for connections from the local machine.
Listen localhost:631
Listen /run/cups/cups.sock

LLR2:~ # ll /etc/cups/cupsd.conf
-rw-r----- 1 root lp 8329 May 16  2020 /etc/cups/cupsd.conf
LLR2:~ # cat /etc/cups/cupsd.conf
#
# Configuration file for the CUPS scheduler.  See "man cupsd.conf" for a
# complete description of this file.
#

# Log general information in error_log - change "warn" to "debug"
# for troubleshooting...
LogLevel warn
PageLogFormat

# Only listen for connections from the local machine.
Listen localhost:631
Listen /run/cups/cups.sock

# Show shared printers on the local network.
Browsing On
BrowseLocalProtocols dnssd

# Default authentication type, when authentication is required...
DefaultAuthType Basic

# Web interface setting...
WebInterface Yes

# Restrict access to the server...
<Location />
  Order allow,deny
</Location>

# Restrict access to the admin pages...
<Location /admin>
  Order allow,deny
</Location>

# Restrict access to configuration files...
<Location /admin/conf>
  AuthType Default
  Require user @SYSTEM
  Order allow,deny
</Location>

# Restrict access to log files...
<Location /admin/log>
  AuthType Default
  Require user @SYSTEM
  Order allow,deny
</Location>

# Set the default printer/job policies...
<Policy default>
  # Job/subscription privacy...
  JobPrivateAccess default
  JobPrivateValues default
  SubscriptionPrivateAccess default
  SubscriptionPrivateValues default

  # Job-related operations must be done by the owner or an administrator...
  <Limit Create-Job Print-Job Print-URI Validate-Job>
    Order deny,allow
  </Limit>

  <Limit Send-Document Send-URI Hold-Job Release-Job Restart-Job Purge-Jobs Set-Job-Attributes Create-Job-Subscription Renew-Subscription Cancel-Subscription Get-Notifications Reprocess-Job Cancel-Current-Job Suspend-Current-Job Resume-Job Cancel-My-Jobs Close-Job CUPS-Move-Job CUPS-Get-Document>
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>

  # All administration operations require an administrator to authenticate...
  <Limit CUPS-Add-Modify-Printer CUPS-Delete-Printer CUPS-Add-Modify-Class CUPS-Delete-Class CUPS-Set-Default CUPS-Get-Devices>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>

  # All printer operations require a printer operator to authenticate...
  <Limit Pause-Printer Resume-Printer Enable-Printer Disable-Printer Pause-Printer-After-Current-Job Hold-New-Jobs Release-Held-New-Jobs Deactivate-Printer Activate-Printer Restart-Printer Shutdown-Printer Startup-Printer Promote-Job Schedule-Job-After Cancel-Jobs CUPS-Accept-Jobs CUPS-Reject-Jobs>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>

  # Only the owner or an administrator can cancel or authenticate a job...
  <Limit Cancel-Job CUPS-Authenticate-Job>
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>

  <Limit All>
    Order deny,allow
  </Limit>
</Policy>

# Set the authenticated printer/job policies...
<Policy authenticated>
  # Job/subscription privacy...
  JobPrivateAccess default
  JobPrivateValues default
  SubscriptionPrivateAccess default
  SubscriptionPrivateValues default

  # Job-related operations must be done by the owner or an administrator...
  <Limit Create-Job Print-Job Print-URI Validate-Job>
    AuthType Default
    Order deny,allow
  </Limit>

  <Limit Send-Document Send-URI Hold-Job Release-Job Restart-Job Purge-Jobs Set-Job-Attributes Create-Job-Subscription Renew-Subscription Cancel-Subscription Get-Notifications Reprocess-Job Cancel-Current-Job Suspend-Current-Job Resume-Job Cancel-My-Jobs Close-Job CUPS-Move-Job CUPS-Get-Document>
    AuthType Default
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>

  # All administration operations require an administrator to authenticate...
  <Limit CUPS-Add-Modify-Printer CUPS-Delete-Printer CUPS-Add-Modify-Class CUPS-Delete-Class CUPS-Set-Default>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>

  # All printer operations require a printer operator to authenticate...
  <Limit Pause-Printer Resume-Printer Enable-Printer Disable-Printer Pause-Printer-After-Current-Job Hold-New-Jobs Release-Held-New-Jobs Deactivate-Printer Activate-Printer Restart-Printer Shutdown-Printer Startup-Printer Promote-Job Schedule-Job-After Cancel-Jobs CUPS-Accept-Jobs CUPS-Reject-Jobs>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>

  # Only the owner or an administrator can cancel or authenticate a job...
  <Limit Cancel-Job CUPS-Authenticate-Job>
    AuthType Default
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>

  <Limit All>
    Order deny,allow
  </Limit>
</Policy>

# Set the kerberized printer/job policies...
<Policy kerberos>
  # Job/subscription privacy...
  JobPrivateAccess default
  JobPrivateValues default
  SubscriptionPrivateAccess default
  SubscriptionPrivateValues default

  # Job-related operations must be done by the owner or an administrator...
  <Limit Create-Job Print-Job Print-URI Validate-Job>
    AuthType Negotiate
    Order deny,allow
  </Limit>

  <Limit Send-Document Send-URI Hold-Job Release-Job Restart-Job Purge-Jobs Set-Job-Attributes Create-Job-Subscription Renew-Subscription Cancel-Subscription Get-Notifications Reprocess-Job Cancel-Current-Job Suspend-Current-Job Resume-Job Cancel-My-Jobs Close-Job CUPS-Move-Job CUPS-Get-Document>
    AuthType Negotiate
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>

  # All administration operations require an administrator to authenticate...
  <Limit CUPS-Add-Modify-Printer CUPS-Delete-Printer CUPS-Add-Modify-Class CUPS-Delete-Class CUPS-Set-Default>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>

  # All printer operations require a printer operator to authenticate...
  <Limit Pause-Printer Resume-Printer Enable-Printer Disable-Printer Pause-Printer-After-Current-Job Hold-New-Jobs Release-Held-New-Jobs Deactivate-Printer Activate-Printer Restart-Printer Shutdown-Printer Startup-Printer Promote-Job Schedule-Job-After Cancel-Jobs CUPS-Accept-Jobs CUPS-Reject-Jobs>
    AuthType Default
    Require user @SYSTEM
    Order deny,allow
  </Limit>

  # Only the owner or an administrator can cancel or authenticate a job...
  <Limit Cancel-Job CUPS-Authenticate-Job>
    AuthType Negotiate
    Require user @OWNER @SYSTEM
    Order deny,allow
  </Limit>

  <Limit All>
    Order deny,allow
  </Limit>
</Policy>

# The policy below is added by SUSE during build of our cups package.
# The policy 'allowallforanybody' is totally open and insecure and therefore
# it can only be used within an internal network where only trused users exist
# and where the cupsd is not accessible at all from any external host, see
# http://en.opensuse.org/SDB:CUPS_and_SANE_Firewall_settings
# Have in mind that any user who is allowed to do printer admin tasks
# can change the print queues as he likes - e.g. send copies of confidental
# print jobs from an internal network to any external destination, see
# http://en.opensuse.org/SDB:CUPS_in_a_Nutshell
# For documentation regarding 'Managing Operation Policies' see
# http://www.cups.org/documentation.php/doc-1.7/policies.html
<Policy allowallforanybody>
  # Allow anybody to access job's private values:
  JobPrivateAccess all
  # Make none of the job values to be private:
  JobPrivateValues none
  # Allow anybody to access subscription's private values:
  SubscriptionPrivateAccess all
  # Make none of the subscription values to be private:
  SubscriptionPrivateValues none
  # Allow anybody to do all IPP operations:
  # Currently the IPP operations Validate-Job Cancel-Jobs Cancel-My-Jobs Close-Job CUPS-Get-Document
  # must be additionally exlicitly specified because those IPP operations are not included
  # in the "All" wildcard value - otherwise cupsd prints error messages of the form
  # "No limit for Validate-Job defined in policy allowallforanybody and no suitable template found."
  <Limit Validate-Job Cancel-Jobs Cancel-My-Jobs Close-Job CUPS-Get-Document>
    Order deny,allow
    Allow from all
  </Limit>
  # Since CUPS > 1.5.4 the "All" wildcard value must be specified separately,
  # otherwise clients like "lpstat -p" just hang up,
  # see https://bugzilla.opensuse.org/show_bug.cgi?id=936309
  # and https://www.cups.org/str.php?L4659
  <Limit All>
    Order deny,allow
    Allow from all
  </Limit>
</Policy>
# Explicitly set the CUPS 'default' policy to be used by default:
DefaultPolicy default

LLR2:~ # 


Virtual Box VM

VM1:~ # lsof -i:631
COMMAND  PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
cupsd   3917 root    5u  IPv6 4349470      0t0  TCP localhost:ipp (LISTEN)
cupsd   3917 root    8u  IPv4 4349471      0t0  TCP localhost:ipp (LISTEN)
VM1:~ # egrep -i "listen|port" /etc/cups/cupsd.conf
# Only listen for connections from the local machine.
Listen localhost:631
Listen /run/cups/cups.sock
VM1:~ # ll /etc/cups/cupsd.conf
-rw-r----- 1 root lp 8329 May 16  2020 /etc/cups/cupsd.conf
VM1:~ # cat /etc/cups/cupsd.conf
**** same as LLr2 ***
VM1:~ # 


I added the Listen *:631 and restarted cups - no change.

Is Avahi active on the hosts as well?

sudo systemctl status avahi-daemon

server:

LLR1:~ # systemctl status avahi-daemon
● avahi-daemon.service - Avahi mDNS/DNS-SD Stack
   Loaded: loaded (/usr/lib/systemd/system/avahi-daemon.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2021-01-21 09:38:17 CST; 4h 18min ago
 Main PID: 1060 (avahi-daemon)
   Status: "Server startup complete. Host name is LLR1.local. Local service cookie is 4069120057."
    Tasks: 1
   CGroup: /system.slice/avahi-daemon.service
           └─1060 avahi-daemon: running [LLR1.local]

Jan 21 09:38:17 LLR1 avahi-daemon[1060]: Network interface enumeration completed.
Jan 21 09:38:17 LLR1 avahi-daemon[1060]: Server startup complete. Host name is LLR1.local. Local service cookie is 4069120057.
Jan 21 09:38:17 LLR1 avahi-daemon[1060]: Service "LLR1" (/etc/avahi/services/ssh.service) successfully established.
Jan 21 09:38:17 LLR1 avahi-daemon[1060]: Service "LLR1" (/etc/avahi/services/sftp-ssh.service) successfully established.
Jan 21 09:38:21 LLR1 avahi-daemon[1060]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.20.101.
Jan 21 09:38:21 LLR1 avahi-daemon[1060]: New relevant interface eth0.IPv4 for mDNS.
Jan 21 09:38:21 LLR1 avahi-daemon[1060]: Registering new address record for 192.168.20.101 on eth0.IPv4.
Jan 21 09:38:22 LLR1 avahi-daemon[1060]: Joining mDNS multicast group on interface eth0.IPv6 with address fe80::a64c:c8ff:fe2b:5>
Jan 21 09:38:22 LLR1 avahi-daemon[1060]: New relevant interface eth0.IPv6 for mDNS.
Jan 21 09:38:22 LLR1 avahi-daemon[1060]: Registering new address record for fe80::a64c:c8ff:fe2b:582e on eth0.*.
LLR1:~ # 

VM

VM1:~ #  systemctl status avahi-daemon
● avahi-daemon.service - Avahi mDNS/DNS-SD Stack
   Loaded: loaded (/usr/lib/systemd/system/avahi-daemon.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2021-01-18 23:16:09 CST; 2 days ago
 Main PID: 890 (avahi-daemon)
   Status: "Server startup complete. Host name is VM1.local. Local service cookie is 1355585992."
    Tasks: 1
   CGroup: /system.slice/avahi-daemon.service
           └─890 avahi-daemon: running [VM1.local]

Jan 21 13:09:46 VM1 avahi-daemon[890]: Withdrawing address record for fe80::a02:7ff:fe77:f856 on eth0.
Jan 21 13:09:46 VM1 avahi-daemon[890]: Leaving mDNS multicast group on interface eth0.IPv6 with address fe80:>
Jan 21 13:09:46 VM1 avahi-daemon[890]: Interface eth0.IPv6 no longer relevant for mDNS.
Jan 21 13:09:46 VM1 avahi-daemon[890]: Withdrawing address record for 192.168.20.241 on eth0.
Jan 21 13:09:46 VM1 avahi-daemon[890]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.1>
Jan 21 13:09:46 VM1 avahi-daemon[890]: Interface eth0.IPv4 no longer relevant for mDNS.
Jan 21 13:09:46 VM1 avahi-daemon[890]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.1>
Jan 21 13:09:46 VM1 avahi-daemon[890]: New relevant interface eth0.IPv4 for mDNS.
Jan 21 13:09:46 VM1 avahi-daemon[890]: Registering new address record for 192.168.20.241 on eth0.IPv4.
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
VM1:~ # 

What is reported on the VM?

avahi-browse -art
nmap LLR1.local
nmap LLR2.local

My results

VM1:~ # avahi-browse -art
VM1:~ # nmap LLR1
Starting Nmap 7.70 ( https://nmap.org ) at 2021-01-21 14:46 CST
Nmap scan report for LLR1 (192.168.20.101)
Host is up (0.00057s latency).
rDNS record for 192.168.20.101: llr1
Not shown: 992 filtered ports
PORT     STATE  SERVICE
22/tcp   open   ssh
80/tcp   closed http
139/tcp  open   netbios-ssn
161/tcp  closed snmp
443/tcp  closed https
445/tcp  open   microsoft-ds
631/tcp  open   ipp
5900/tcp open   vnc
MAC Address: A4:4C:C8:2B:58:2E (Dell)

Nmap done: 1 IP address (1 host up) scanned in 9.62 seconds
VM1:~ # nmap LLR2
Starting Nmap 7.70 ( https://nmap.org ) at 2021-01-21 14:47 CST
Nmap scan report for LLR2 (192.168.20.102)
Host is up (0.00051s latency).
rDNS record for 192.168.20.102: llr2
Not shown: 995 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
139/tcp  open  netbios-ssn
445/tcp  open  microsoft-ds
5900/tcp open  vnc
5901/tcp open  vnc-1
MAC Address: 8C:04:BA:3F:B9:55 (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 1.85 seconds
VM1:~ # 

I note that LLR2.local nmap results not listing port 631 (CUPS IPP service).

It does not have a printer attached to it - rmr does port 631 is only needed on systems with printers

VM1:~ # nmap rmr
Starting Nmap 7.70 ( https://nmap.org ) at 2021-01-21 15:35 CST
Nmap scan report for rmr (192.168.20.112)
Host is up (0.00041s latency).
Other addresses for rmr (not scanned): 192.168.20.210
Not shown: 994 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
139/tcp  open  netbios-ssn
445/tcp  open  microsoft-ds
631/tcp  open  ipp
5900/tcp open  vnc
5901/tcp open  vnc-1
MAC Address: A4:4C:C8:12:23:62 (Dell)

Nmap done: 1 IP address (1 host up) scanned in 1.67 seconds
VM1:~ # 


I just tested my Raspberry Pi 4 - they see the printers. they are running Ubuntu MATE 20.04 and it prints fine from there.

VM1:~ # nmap rpi01
Starting Nmap 7.70 ( https://nmap.org ) at 2021-01-21 15:47 CST
Nmap scan report for rpi01 (192.168.20.171)
Host is up (0.0014s latency).
Not shown: 993 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
53/tcp   open  domain
80/tcp   open  http
111/tcp  open  rpcbind
139/tcp  open  netbios-ssn
445/tcp  open  microsoft-ds
5900/tcp open  vnc
MAC Address: DC:A6:32:B2:7E:78 (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 1.67 seconds
VM1:~ # 

Ok, not easy to know this from a distance.

Ok, and does avahi-browse report the shared printers (from the rpi hosts) as a matter of interest? You may need to install the relevant package first.

it has to be one of these updates as only OpenSUSE 15.2 has the issues - since all the machines I have have the same thing installed - here is what got installed since monday - I deleted the fetch and installs to make it readable

LLR2:~ # zypper -n up

The following 16 packages are going to be upgraded:
  cpp7 gcc7 libasan4 libcilkrts5 libgfortran4 libstdc++6-devel-gcc7 libubsan0
  libvlc5 libvlccore9 vlc vlc-codec-gstreamer vlc-lang vlc-noX vlc-opencv vlc-qt
  vlc-vdpau

16 packages to upgrade.

LLR2:~ # zypper -n up

The following 4 packages are going to be upgraded:
  libldap-2_4-2 libldap-2_4-2-32bit libldap-data openldap2-client

4 packages to upgrade.

LLR2:~ # zypper -n up

The following package is going to be upgraded:
  chromium

1 package to upgrade.

LLR2:~ # zypper -n up

The following 8 packages are going to be upgraded:
  device-mapper gnome-software gnome-software-lang libdevmapper-event1_03
  libdevmapper1_03 libdevmapper1_03-32bit liblvm2cmd2_03 lvm2

8 packages to upgrade.

LLR2:~ # zypper -n up

The following package is going to be upgraded:
  mdadm

1 package to upgrade.

LLR2:~ # zypper -n up 

The following package is going to be upgraded:
  dnsmasq

1 package to upgrade.

My openSUSE Leap 15.2 system is fully updated, and I can’t replicate the issue you’re observing from a client perspective. Both ‘lpinfo -v’ and ‘avahi-browse -art’ report the CUPS printer I have shared on a CUPS server I have at hand.

FWIW, the CUPS dnssd backend (run from the client machine) returned…

~> sudo /usr/lib/cups/backend/dnssd
[sudo] password for root: 
DEBUG: sent=0, count=0
DEBUG: Querying "Brother\032DCP-7055\032\064\032Xxxx\226\128\153s\032MacBook\032Air._ipp._tcp.local"...
DEBUG: Querying "Brother\032DCP-7055\032\064\032Xxxx\226\128\153s\032MacBook\032Air._ipps._tcp.local"...
DEBUG: sent=0, count=2
network dnssd://Brother%20DCP-7055%20%40%20Xxxx%E2%80%99s%20MacBook%20Air._ipp._tcp.local/ "unknown" "Brother DCP-7055 @ Xxxx MacBook Air" "" ""
DEBUG: sent=2, count=2
DEBUG: sent=2, count=0
~> 

You should find out why Avahi (DNS-SD) is not discovering the CUPS printers on the affected hosts.

How can I test if Bonjour is working - dnssd uses Bonjour and something is preventing it from talking on eth0 since Tuesday but servers with printers see each other which means Bonjour is running on them and they see each other

The server with a printer sees:

LLR1:~ # /usr/lib/cups/backend/dnssd
DEBUG: Ignoring local service Brother HL-L2395DW series @ LLR1.
DEBUG: Ignoring local service Brother HL-L2395DW series @ LLR1.
DEBUG: Ignoring local service Brother HL-L2395DW series @ LLR1.
DEBUG: Querying "EPSON\032ET-2720\032Series\032\064\032RMR._ipp._tcp.local"...
DEBUG: Querying "EPSON\032ET-2720\032Series\032\064\032RMR._ipps._tcp.local"...
DEBUG: Querying "EPSON\032ET-2720\032Series\032\064\032RMR._printer._tcp.local"...
DEBUG: sent=0, count=3
DEBUG2: query_callback(browser=0x5596376f2400, interfaceIndex=2, protocol=1, event=0, fullName="EPSON\032ET-2720\032Series\032\064\032RMR._ipp._tcp.local", rrclass=1, rrtype=16, rdata=0x5596376f38c0, rdlen=472, flags=5, context=0x5596376f16c0)
DEBUG2: query_callback: "txtvers=1".
DEBUG2: query_callback: "qtotal=1".
DEBUG2: query_callback: "rp=printers/EPSON_ET-2720_Series".
DEBUG2: query_callback: "ty=Epson ET-2720 Series - epson-inkjet-printer-escpr 1.7.7-1lsb3.2 (Seiko Epson Corporation LSB 3.2)".
DEBUG2: query_callback: "adminurl=https://RMR.local.:631/printers/EPSON_ET-2720_Series".
DEBUG2: query_callback: "note=Rose's PC".
DEBUG2: query_callback: "priority=0".
DEBUG2: query_callback: "product=(EPSON ET-2720 Series)".
DEBUG2: query_callback: "pdl=application/octet-stream,application/pdf,application/postscript,image/jpeg,image/png,image/pwg-raster".
DEBUG2: query_callback: "UUID=7215b299-8916-3513-4681-5800f700a884".
DEBUG2: query_callback: "TLS=1.2".
DEBUG2: query_callback: "Color=T".
DEBUG2: query_callback: "printer-state=3".
DEBUG2: query_callback: "printer-type=0x100E".
DEBUG2: query_callback(browser=0x5596376f2400, interfaceIndex=-1, protocol=-1, event=2, fullName="EPSON\032ET-2720\032Series\032\064\032RMR._ipp._tcp.local", rrclass=1, rrtype=16, rdata=(nil), rdlen=0, flags=0, context=0x5596376f16c0)
DEBUG2: query_callback(browser=0x5596376f1bc0, interfaceIndex=2, protocol=1, event=0, fullName="EPSON\032ET-2720\032Series\032\064\032RMR._ipps._tcp.local", rrclass=1, rrtype=16, rdata=0x5596376f3c60, rdlen=472, flags=5, context=0x5596376f1540)
DEBUG2: query_callback: "txtvers=1".
DEBUG2: query_callback: "qtotal=1".
DEBUG2: query_callback: "rp=printers/EPSON_ET-2720_Series".
DEBUG2: query_callback: "ty=Epson ET-2720 Series - epson-inkjet-printer-escpr 1.7.7-1lsb3.2 (Seiko Epson Corporation LSB 3.2)".
DEBUG2: query_callback: "adminurl=https://RMR.local.:631/printers/EPSON_ET-2720_Series".
DEBUG2: query_callback: "note=Rose's PC".
DEBUG2: query_callback: "priority=0".
DEBUG2: query_callback: "product=(EPSON ET-2720 Series)".
DEBUG2: query_callback: "pdl=application/octet-stream,application/pdf,application/postscript,image/jpeg,image/png,image/pwg-raster".
DEBUG2: query_callback: "UUID=7215b299-8916-3513-4681-5800f700a884".
DEBUG2: query_callback: "TLS=1.2".
DEBUG2: query_callback: "Color=T".
DEBUG2: query_callback: "printer-state=3".
DEBUG2: query_callback: "printer-type=0x100E".
DEBUG2: query_callback(browser=0x5596376f1bc0, interfaceIndex=-1, protocol=-1, event=2, fullName="EPSON\032ET-2720\032Series\032\064\032RMR._ipps._tcp.local", rrclass=1, rrtype=16, rdata=(nil), rdlen=0, flags=0, context=0x5596376f1540)
DEBUG2: query_callback(browser=0x5596376f3570, interfaceIndex=2, protocol=1, event=0, fullName="EPSON\032ET-2720\032Series\032\064\032RMR._printer._tcp.local", rrclass=1, rrtype=16, rdata=0x5596376f1e14, rdlen=1, flags=5, context=0x5596376f1aa0)
DEBUG2: query_callback(browser=0x5596376f3570, interfaceIndex=-1, protocol=-1, event=2, fullName="EPSON\032ET-2720\032Series\032\064\032RMR._printer._tcp.local", rrclass=1, rrtype=16, rdata=(nil), rdlen=0, flags=0, context=0x5596376f1aa0)
network dnssd://EPSON%20ET-2720%20Series%20%40%20RMR._ipp._tcp.local/cups?uuid=7215b299-8916-3513-4681-5800f700a884 "EPSON EPSON ET-2720 Series" "EPSON ET-2720 Series @ RMR" "MFG:EPSON;MDL:ET-2720 Series;CMD:PDF,PS,JPEG,PNG,PWG;" ""
DEBUG: sent=3, count=3
LLR1:~ # 

Every VM and non printer server sees:

VM1:~ # /usr/lib/cups/backend/dnssd
DEBUG: sent=0, count=0
DEBUG: sent=0, count=0
VM1:~ # 

I think I found the problem but how to fix

VM1:/var/log/cups # >access_log
VM1:/var/log/cups # >error_log
VM1:/var/log/cups # ll
total 0
-rw-r--r-- 1 root lp 0 Jan 22 08:08 access_log
-rw-r--r-- 1 root lp 0 Jan 22 08:08 error_log
VM1:/var/log/cups # systemctl restart cups
VM1:/var/log/cups # cat error_log
E [22/Jan/2021:08:08:38 -0600] Unable to open listen socket for address [v1.::1]:631 - Address already in use.
E [22/Jan/2021:08:08:38 -0600] Unable to open listen socket for address 127.0.0.1:631 - Address already in use.
VM1:/var/log/cups # 
VM1:/var/log/cups # netstat -an | grep 631
tcp        0      0 0.0.0.0:631             0.0.0.0:*               LISTEN     
tcp6       0      0 :::631                  :::*                    LISTEN     
unix  3       ]         STREAM     CONNECTED     26310    
unix  3       ]         STREAM     CONNECTED     3631433  
unix  3       ]         STREAM     CONNECTED     3631231  @/dbus-vfs-daemon/socket-mc2KtvV8
unix  3       ]         STREAM     CONNECTED     3631434  @/dbus-vfs-daemon/socket-8FY54nty
unix  3       ]         STREAM     CONNECTED     3631430  @/dbus-vfs-daemon/socket-S12FhYHR
unix  3       ]         STREAM     CONNECTED     3631427  @/dbus-vfs-daemon/socket-AYEVB8mx
unix  2       ]         SEQPACKET  CONNECTED     4279631  
unix  3       ]         STREAM     CONNECTED     45631    @/tmp/.X11-unix/X0
VM1:/var/log/cups #