ssh problem: connection closed after appearent login

Can not connect to my ssh-server (OpenSuSE 12.1) after restarting it after recent (May, 5) power break. Before that everything worked OK. On client side it takes login/password, prints message of the day then close connection. Debug shows basically nothing:

debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1
Password:
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 0
debug1: Authentication succeeded (keyboard-interactive).
Authenticated to elis.dvo.ru ([94.198.20.37]:22).
debug1: channel 0: new [client-session]
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug2: callback start
debug2: client_session2_setup: id 0
debug2: fd 3 setting TCP_NODELAY
debug2: channel 0: request pty-req confirm 1
debug1: Sending environment.
debug1: Sending env LANG = ru_RU.UTF-8
debug2: channel 0: request env confirm 0
debug2: channel 0: request shell confirm 1
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug2: channel 0: rcvd eow
debug2: channel 0: close_read
debug2: channel 0: input open -> closed
Last login: Sun May 6 18:40:51 2012 from 109.110.40.158

Hi, everybody

debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug2: channel 0: obuf empty
debug2: channel 0: close_write
debug2: channel 0: output drain -> closed
debug2: channel 0: rcvd close
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
Connection to elis.dvo.ru closed.
Transferred: sent 2232, received 1856 bytes, in 0.1 seconds
Bytes per second: sent 38549.3, received 32055.3
debug1: Exit status 254

On server side:

sshd[14133]: error: PAM: pam_open_session(): Permission denied

(/var/log/messages)

Users are authorized via NIS on another machine. Local logins on the ssh-server work fine, so the problem should not be with NIS.

Any ideas ?

Please, next time use CODE tags around comnputer output: http://forums.opensuse.org/english/information-new-users/advanced-how-faq-read-only/451526-posting-code-tags-guide.html
It will make that text and the whole post better readable.

Definetly. But what about the problem ?


OpenSSH_5.8p2, OpenSSL 1.0.0e 6 Sep 2011
debug1: Reading configuration data /etc/ssh/ssh_config

...

debug1: Authentications that can continue: publickey,keyboard-interactive

...

debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 0
debug1: Authentication succeeded (keyboard-interactive).
Authenticated to elis.dvo.ru ([94.198.20.37]:22).
debug2: fd 6 setting O_NONBLOCK
debug1: channel 0: new [client-session]
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug2: callback start
debug2: client_session2_setup: id 0
debug2: fd 3 setting TCP_NODELAY
debug2: channel 0: request pty-req confirm 1
debug1: Sending environment.
debug1: Sending env LANG = ru_RU.UTF-8
debug2: channel 0: request env confirm 0
debug2: channel 0: request shell confirm 1
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug2: channel 0: rcvd eow
debug2: channel 0: close_read
debug2: channel 0: input open -> closed
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug2: channel 0: obuf empty
debug2: channel 0: close_write
debug2: channel 0: output drain -> closed
debug2: channel 0: rcvd close
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 2 clearing O_NONBLOCK
Connection to elis.dvo.ru closed.
Transferred: sent 2088, received 1760 bytes, in 0.0 seconds
Bytes per second: sent 51289.6, received 43232.7
debug1: Exit status 254

On server side:

sshd[14133]: error: PAM: pam_open_session(): Permission denied

(/var/log/messages)



That looks like a problem with the PAM setup on the server. It probably affects other kinds of login, and not just ssh.

No, it’s OK with the local logins. May I also add that users home is an NFS share ? Mount options are default and local logins work as usual, read/write et all. No “unlimited” on “nofiles” in limits.conf, which caused the similar problem for the other users.

I have checked several possible reasons why this could have happened (based on Google search): /etc/nologin, “nofile unlimited” in limits.conf, nothing worked.
This ssh problem persists for local users, even within the same machine. At the same time outgoing ssh connections work just fine. Is there any way to debug PAM or to get more detailed information on this problem ?

I have exactly the same problem. It worked for a while and now closes whenever there is an ssh login. Everything else works fine.

What happens, if you do a


su -c 'rcsshd restart'

then try to access.

I tried that and it didn’t work. I’m wondering about Avahi or the ipmi interface possibly causing a problem.

I can only add that I have

pam - A Security Tool that Provides Authentication for Applications Version: 1.1.4-9.2.2 Installed: 1.1.4-9.2.2

installed. Reinstall of opensshd did not make any effect. As /bin/login prints /etc/motd does it mean that it is users shell which crashes?

On 2012-05-08 20:46, npjohn01 wrote:

> I tried that and it didn’t work.

What did not work, the command, or accessing later?

Run “rcrpmconfigcheck” on the server and post it here.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

It did not work for me either.

On 2012-05-10 12:06, nurmi e wrote:
> It did not work for me either.

Same comment as previously done, with no answer.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

Here it is


elis:/etc/ssh # rcrpmconfigcheck
Searching for unresolved configuration files                                                                                                          done
Please check the following files (see /var/adm/rpmconfigcheck):
    /etc/ssh/sshd_config.rpmsave

The contents of sshd_config and sshd_config.rpmsave are as follows:


sshd_config:PasswordAuthentication no
sshd_config:UsePAM yes
sshd_config:X11Forwarding yes
sshd_config:Subsystem sftp      /usr/lib/ssh/sftp-server
sshd_config:AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
sshd_config:AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
sshd_config:AcceptEnv LC_IDENTIFICATION LC_ALL
sshd_config:AllowTcpForwarding yes
sshd_config:Compression yes
sshd_config:MaxAuthTries 6
sshd_config:PrintMotd yes
sshd_config:Protocol 2
sshd_config:PubkeyAuthentication yes
sshd_config:RSAAuthentication yes

sshd_config.rpmsave:PermitRootLogin no
sshd_config.rpmsave:PasswordAuthentication no
sshd_config.rpmsave:UsePAM yes
sshd_config.rpmsave:X11Forwarding yes
sshd_config.rpmsave:Subsystem   sftp    /usr/lib/ssh/sftp-server
sshd_config.rpmsave:AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
sshd_config.rpmsave:AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
sshd_config.rpmsave:AcceptEnv LC_IDENTIFICATION LC_ALL

comments and blank lines excluded.

On 2012-05-11 03:36, nurmi e wrote:
>
> Here it is
>
> Code:
> --------------------
>
> elis:/etc/ssh # rcrpmconfigcheck
> Searching for unresolved configuration files done
> Please check the following files (see /var/adm/rpmconfigcheck):
> /etc/ssh/sshd_config.rpmsave
>
> --------------------

Aha, interesting. That file is the old configuration before the upgrade.
The active file is different. Some of those changes are the culprit. You
will have to play with the differences in that server.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

sshd_conf.rpmsave most probably appeared as a result of openssh update which was undertaken in a desperate move to fix the situation. SSH already did not work at that moment. But I will compare in with the working system.

On 2012-05-11 14:16, nurmi e wrote:
>
> sshd_conf.rpmsave most probably appeared as a result of openssh update
> which was undertaken in a desperate move to fix the situation. SSH
> already did not work at that moment. But I will compare in with the
> working system.

Ah.
I can post what I have in mine, but I don’t keep a server:


> Telcontar:~ # cat /etc/ssh/sshd_config |  egrep -v "^:space:]]*$|^#"
> PasswordAuthentication no
> UsePAM yes
> X11Forwarding yes
> Subsystem       sftp    /usr/lib64/ssh/sftp-server
> AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
> AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
> AcceptEnv LC_IDENTIFICATION LC_ALL
> Telcontar:~ #

However, in this the crucial part here is what PAM contains… the main one
should be this:


Telcontar:~ # cat /etc/pam.d/sshd
#%PAM-1.0
auth     requisite      pam_nologin.so
auth     include        common-auth
account  requisite      pam_nologin.so
account  include        common-account
password include        common-password
session  required       pam_loginuid.so
session  include        common-session
session  optional       pam_lastlog.so   silent noupdate showfailed
Telcontar:~ #

but don’t ask me to explain what it means because I know next to nothing
about pam.

You said that the issue happened after a reboot - this is because you did
not reboot after an update of something. Many updates in Linux are not
applied till you restart the affected applications, or reboot. The saying
that you never need to reboot Linux after an update (like in Windows) is false.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

Seems /etc/ssh/sshd_config is not a reason. When I replaced it with a copy of sshd_config from the working machine nothing happened. That is it still did not let remote logins. Finally I just reinstalled the system from scratch and it works now. At least so far …

On 2012-05-12 09:16, nurmi e wrote:
>
> Seems /etc/ssh/sshd_config is not a reason. When I replaced it with a
> copy of sshd_config from the working machine nothing happened. That is
> it still did not let remote logins. Finally I just reinstalled the
> system from scratch and it works now. At least so far …

You did not have a look at pam, then.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

I’ve got the same problem on a new installed suse 12.2 on a PIII my home gateway. I’ve work remotely with firefox only for a month without problems. Once i’ve configure samba as PDC via YaST. Then this problem apears with putty on Win-Xp. Then i stop samba server and problem disapear till next reboot and after login.
Today i atempt connect with same accounts in vain. But i connect with root accout without problem and reboot system romotely. But once. This problem exists with all accounts now. pam_open_session remove session after enter correct password.