389ds + SSSD: Unable to login: 6 (Permission denied)

I’m struggling to correctly setup a 389ds server with SSSD in openSuse Leap 15.3. I followed the instructions found here: https://doc.opensuse.org/documentation/leap/security/html/book-security/cha-security-ldap.html
Everything went smooth, but when I try to login as an ldap user, it fails.
journalctl tells me, when I try to login using ssh:

$ ssh -l robin
Connection closed by port 22

sep 23 22:11:08 izanami sshd[2742]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=  user=robin
sep 23 22:11:08 izanami sshd[2742]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost= user=robin
sep 23 22:11:08 izanami sshd[2742]: pam_sss(sshd:account): Access denied for user robin: 6 (Permission denied)
sep 23 22:11:08 izanami sshd[2740]: error: PAM: User account has expired for robin from
sep 23 22:11:08 izanami sshd[2740]: fatal: monitor_read: unpermitted request 104

The same with sudo -u:

izanami:~ # sudo -i -u robin
sudo: PAM account management error: Permission denied
sudo: a password is required

sep 23 22:12:38 izanami sudo[2744]: pam_sss(sudo-i:account): Access denied for user robin: 6 (Permission denied)
sep 23 22:12:38 izanami sudo[2744]:     root : PAM account management error: Permission denied; TTY=pts/0 ; PWD=/home/robin ; USER=robin ; COMMAND=/bin/bash

The user is correctly defined in 389ds:

# dsidm ldap user get robin                               
dn: uid=robin,ou=people,dc=sicho,dc=home
cn: Robin Roevens
displayName: Robin Roevens
gidNumber: 100
homeDirectory: /home/robin
loginShell: /bin/bash
objectClass: top
objectClass: nsPerson
objectClass: nsAccount
objectClass: nsOrgPerson
objectClass: posixAccount
uid: robin
uidNumber: 1000
userPassword: ****

The user is not expired:

# dsidm ldap account entry-status uid=robin,ou=people,dc=sicho,dc=home
Entry DN: uid=robin,ou=people,dc=sicho,dc=home
Entry Creation Date: 20210923192359Z (2021-09-23 19:23:59)
Entry Modification Date: 20210923195836Z (2021-09-23 19:58:36)
Entry State: activated

and correctly seen by getent passwd:

# getent passwd robin

user robin is member of the ldap group server_admins:

# dsidm ldap group get server_admins
dn: cn=server_admins,ou=groups,dc=sicho,dc=home
cn: server_admins
member: uid=robin,ou=people,dc=sicho,dc=home
objectClass: top
objectClass: groupOfNames
objectClass: nsMemberOf

This group is not seen by getent group, I assume since it has no objectClass: posixGroup. But I don’t think this is a problem; as this is only used in sssd settings as ldap_access_filter: (memberOf=cn=server_admins,ou=groups,dc=sicho,dc=home)

The MemberOf 389-ds plugin is enabled:

# dsconf ldap plugin memberof show  
dn: cn=MemberOf Plugin,cn=plugins,cn=config
cn: MemberOf Plugin
memberofattr: memberOf
memberofgroupattr: member
nsslapd-plugin-depends-on-type: database
nsslapd-pluginDescription: memberof plugin
nsslapd-pluginEnabled: on
nsslapd-pluginId: memberof
nsslapd-pluginInitfunc: memberof_postop_init
nsslapd-pluginPath: libmemberof-plugin
nsslapd-pluginType: betxnpostoperation
nsslapd-pluginVendor: 389 Project
objectClass: top
objectClass: nsSlapdPlugin
objectClass: extensibleObject

# dsconf ldap plugin memberof status 
Plugin 'MemberOf Plugin' is enabled

My sssd.conf:

# sssd.conf
# Generated by 389 Directory Server - dsidm
# For more details see man sssd.conf and man sssd-ldap
# Be sure to review the content of this file to ensure it is secure and correct
# in your environment.

# Uncomment this for more verbose logging.
# debug_level=3

# Cache hashes of user authentication for offline auth.
cache_credentials = True
id_provider = ldap
auth_provider = ldap
access_provider = ldap
chpass_provider = ldap
ldap_schema = rfc2307bis
ldap_search_base = dc=sicho,dc=home
ldap_uri = ldaps://localhost
# If you have DNS SRV records, you can use the following instead. This derives
# from your ldap_search_base.
# ldap_uri = _srv_

ldap_tls_reqcert = demand
# To use cacert dir, place *.crt files in this path then run:
# /usr/bin/openssl rehash /etc/openldap/certs
# or (for older versions of openssl)
# /usr/bin/c_rehash /etc/openldap/certs
ldap_tls_cacertdir = /etc/openldap/certs

# Path to the cacert
# ldap_tls_cacert = /etc/openldap/certs/ca.crt

# Only users who match this filter can login and authorise to this machine. Note
# that users who do NOT match, will still have their uid/gid resolve, but they
# can't login.
ldap_access_filter = (memberOf=cn=server_admins,ou=groups,dc=sicho,dc=home)

enumerate = false
access_provider = ldap
ldap_user_member_of = memberof
ldap_user_gecos = cn
ldap_user_uuid = nsUniqueId
ldap_group_uuid = nsUniqueId
# This is really important as it allows SSSD to respect nsAccountLock
ldap_account_expire_policy = rhds
ldap_access_order = filter, expire
# Setup for ssh keys
# Inside /etc/ssh/sshd_config add the lines:
#   AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys
#   AuthorizedKeysCommandUser nobody
# You can test with the command: sss_ssh_authorizedkeys <username>
ldap_user_ssh_public_key = nsSshPublicKey

# This prevents an issue where the Directory is recursively walked on group
# and user look ups. It makes the client faster and more responsive in almost
# every scenario.
ignore_group_members = False

services = nss, pam, ssh, sudo
config_file_version = 2

domains = ldap
homedir_substring = /home

I tried commenting out these lines:

#ldap_access_filter = (memberOf=cn=server_admins,ou=groups,dc=sicho,dc=home)
#ldap_account_expire_policy = rhds
#ldap_access_order = filter, expire

and restarted sssd. But still the same errors and ldap users can’t login.

I tried debug level 3, but all I see in the sssd logs are succesfull lookups and authentications. But still PAM is denying access.

PAM and nsswitch where configured by following the instructions in the sections Configure PAM (SUSE) and Configure NSS (SUSE) at https://www.port389.org/docs/389ds/howto/howto-sssd.html

So I’m out of idea’s and have no clue why PAM is denying access…

What’s the output from:

ls -l /usr/bin/sudo

I’ll note that the output I see here is:

% ls -l /usr/bin/sudo
-rwsr-xr-x 1 root root 184616 Mar  5  2021 /usr/bin/sudo

It is:

# ls -l /usr/bin/sudo
-rwsr-xr-x 1 root root 184616  6 mrt  2021 /usr/bin/sudo

But I don’t think sudo has anything to do with ssh or even console login not working.

Console login btw gives me:

Welcome to openSUSE Leap 15.3 - Kernel 5.3.18-59.19-default (tty1).


izanami login: robin

Permission denied

sep 24 21:28:41 izanami login[1950]: pam_unix(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty1 ruser= rhost=  user=robin
sep 24 21:28:42 izanami login[1950]: pam_sss(login:auth): authentication success; logname=LOGIN uid=0 euid=0 tty=/dev/tty1 ruser= rhost= user=robin
sep 24 21:28:42 izanami login[1950]: pam_sss(login:account): Access denied for user robin: 6 (Permission denied)
sep 24 21:28:42 izanami login[1950]: Permission denied

I set debug level to 7 and now I see a bit more info in sssd logs:

(2021-09-24 22:06:35): [be[ldap]] [dp_pam_handler] (0x0100): Got request with the following data
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): command: SSS_PAM_ACCT_MGMT
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): domain: ldap
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): user: robin@ldap
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): service: sshd
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): tty: ssh
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): ruser: 
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): rhost:
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): authtok type: 0
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): newauthtok type: 0
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): priv: 1
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): cli_pid: 5238
(2021-09-24 22:06:35): [be[ldap]] [pam_print_data] (0x0100): logon name: not set
(2021-09-24 22:06:35): [be[ldap]] [dp_attach_req] (0x0400): DP Request [PAM Account #6]: New request. Flags [0000].
(2021-09-24 22:06:35): [be[ldap]] [dp_attach_req] (0x0400): Number of active DP request: 1
(2021-09-24 22:06:35): [be[ldap]] [sss_domain_get_state] (0x1000): Domain ldap is Active
(2021-09-24 22:06:35): [be[ldap]] [sdap_access_send] (0x0400): Performing access check for user [robin@ldap]
(2021-09-24 22:06:35): [be[ldap]] [sdap_access_filter_send] (0x0400): Performing access filter check for user [robin@ldap]
(2021-09-24 22:06:35): [be[ldap]] [sdap_access_filter_send] (0x0400): Checking filter against LDAP
(2021-09-24 22:06:35): [be[ldap]] [sdap_get_generic_ext_step] (0x0400):  calling ldap_search_ext with  (&(uid=robin)(objectclass=posixAccount)(memberOf=cn=server_admins,ou=groups,dc=sicho,dc=home))][uid=robin,ou=people,dc=sicho,dc=home].
(2021-09-24 22:06:35): [be[ldap]] [sdap_get_generic_op_finished] (0x0400): Search result: Success(0), no errmsg set
(2021-09-24 22:06:35): [be[ldap]] [sdap_access_filter_done] (0x0100):  User [robin@ldap] was not found with the specified filter. Denying  access.
(2021-09-24 22:06:35): [be[ldap]] [sdap_access_filter_done] (0x0400): Access denied by online lookup

It seems that it fails on the memberOf attribute. And when I check my user in the ds, it indeed does not contain a memberOf attribute.
I did enable the memberOf plugin after I had joined my user in the server_admins group and it seems enabling that plugin does not add memberOf attributes to already existing group memberships.
After removing my user from the group again, and then re-adding it, I got a memberOf attribute assigned to my user.
And now login works correctly.

Too bad I didn’t find any decent documentation about this, and SUSE docs only mention that you should enable the plugin for the SSSD filtering to work; but they mention it only during SSSD configuration, after they already assigned the user to the group.
The memberof plugin property

nsslapd-pluginType: betxnpostoperation

could have given me a hint… but I’m not that experienced with 389ds to notice that in time (or know if it is even possible for a plugin to ‘initialize’ on your existing ldap entries)…

Hi Robin,

I just submitted this:

Was thinking maybe by now you are quite expereinced with 389-ds + sssd, and you may have some insight.

Thanks a lot in advance.