Can't ssh into OpenSUSE-13.1

chigurh · July 16, 2014, 9:12am

I can not ssh into openSUSE box behind OpenBSD PF firewall. I can ssh into FreeBSD & CentOS box behind PF. I can not start sshd via systemctl/systemd either.

xxxx:1$ ssh -v xxxx@172.16.0.115
OpenSSH_6.6, OpenSSL 1.0.1c 10 May 2012
debug1: Reading configuration data /home/xxxx/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to 172.16.0.115 [172.16.0.115] port 62636 .
debug1: connect to address 172.16.0.115 port 62636 : Connection refused
ssh: connect to host 172.16.0.115 port 62636 : Connection refused

My sshd_config -

# cat /etc/ssh/sshd_config 
Port 62636 
AddressFamily any
Protocol 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
KeyRegenerationInterval 1h
ServerKeyBits 2048
SyslogFacility AUTH
LogLevel INFO
PermitRootLogin no
StrictModes yes
MaxAuthTries 6
MaxSessions 10
RSAAuthentication yes
PubkeyAuthentication yes
PasswordAuthentication yes
PermitEmptyPasswords no
UsePAM yes
AllowAgentForwarding yes
AllowTcpForwarding yes
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes
PrintMotd yes
PrintLastLog yes
TCPKeepAlive yes
UsePrivilegeSeparation sandbox          # Default for new installations.
UseDNS yes
Subsystem       sftp    /usr/lib/ssh/sftp-server
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL

ssh_config -

# cat /etc/ssh/ssh_config
Host *
   ForwardAgent yes
   ForwardX11 yes
   ForwardX11Trusted yes
   RSAAuthentication yes
   PasswordAuthentication yes
   CheckHostIP yes
   StrictHostKeyChecking ask
   IdentityFile ~/.ssh/identity
   IdentityFile ~/.ssh/id_rsa
   IdentityFile ~/.ssh/id_dsa
   Port 62636 
   Protocol 2
   SendEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
   SendEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
   SendEnv LC_IDENTIFICATION LC_ALL
   VisualHostKey no
   HashKnownHosts yes

When I run systemctl start sshd.service the sshd fails to start -

# systemctl --failed
UNIT            LOAD   ACTIVE SUB    DESCRIPTION
sshd.service    loaded failed failed OpenSSH Daemon

Relevant SuSEfirewall2 settings -

FW_SERVICES_EXT_TCP="62636 "
FW_CONFIGURATIONS_EXT="sshd"
FW_SERVICES_DMZ_TCP="62636"
FW_CONFIGURATIONS_DMZ="sshd"
FW_SERVICES_ACCEPT_EXT="172.16.0.1/24,tcp,62636"
FW_SERVICES_ACCEPT_DMZ=""
FW_SERVICES_ACCEPT_INT="172.16.0.1/24,tcp,62636"
FW_SERVICES_ACCEPT_RELATED_DMZ=""
FW_SERVICES_ACCEPT_RELATED_INT=""

I can ssh in OpenBSD from openSUSE -

# netstat -an | grep :62636 
tcp        0      0 172.16.0.115:34271      172.16.0.1:62636         ESTABLISHED

sshd.service appears in broken state -

# systemctl status sshd.service
sshd.service - OpenSSH Daemon
   Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled)
   Active: failed (Result: start-limit) since Tue 2014-07-15 21:34:40 PDT; 32min ago
  Process: 2750 ExecStart=/usr/sbin/sshd -D $SSHD_OPTS (code=exited, status=1/FAILURE)
  Process: 2747 ExecStartPre=/usr/sbin/sshd-gen-keys-start (code=exited, status=0/SUCCESS)
 Main PID: 2750 (code=exited, status=1/FAILURE)

Jul 15 21:34:40 xxxx systemd[1]: sshd.service: main process exited, code=exited, status=1/FAILURE
Jul 15 21:34:40 xxxx systemd[1]: Unit sshd.service entered failed state.
Jul 15 21:34:40 xxxx systemd[1]: sshd.service holdoff time over, scheduling restart.
Jul 15 21:34:40 xxxx systemd[1]: Stopping OpenSSH Daemon...
Jul 15 21:34:40 xxxx systemd[1]: Starting OpenSSH Daemon...
Jul 15 21:34:40 xxxx systemd[1]: sshd.service start request repeated too quickly, refusing to start.
Jul 15 21:34:40 xxxx systemd[1]: Failed to start OpenSSH Daemon.
Jul 15 21:34:40 xxxx systemd[1]: Unit sshd.service entered failed state.

My ip addr -

# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 74:d4:35:e3:07:50 brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.115/24 brd 172.16.0.255 scope global enp1s0
       valid_lft forever preferred_lft forever
    inet6 fe80::76d4:35ff:fee3:750/64 scope link 
       valid_lft forever preferred_lft forever
3: enp2s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
    link/ether 74:d4:35:e3:07:4e brd ff:ff:ff:ff:ff:ff
4: vmnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:50:56:c0:00:01 brd ff:ff:ff:ff:ff:ff
    inet 172.16.158.1/24 brd 172.16.158.255 scope global vmnet1
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fec0:1/64 scope link 
       valid_lft forever preferred_lft forever
5: vmnet8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:50:56:c0:00:08 brd ff:ff:ff:ff:ff:ff
    inet 172.16.119.1/24 brd 172.16.119.255 scope global vmnet8
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fec0:8/64 scope link 
       valid_lft forever preferred_lft forever

ip route -

# ip route
default via 172.16.0.1 dev enp1s0  proto static 
172.16.0.0/24 dev enp1s0  proto kernel  scope link  src 172.16.0.115  metric 1 
172.16.119.0/24 dev vmnet8  proto kernel  scope link  src 172.16.119.1 
172.16.158.0/24 dev vmnet1  proto kernel  scope link  src 172.16.158.1

I operate ssh on different port than 62636, all the other parameters are same. Have diffed the sshd_config & ssh_config on all hosts and they are about same.

This sshd failure persists on 4 openSUSE machines (3 desktops & 1 notebook), I am sure it is something small but have no clue where to look. My firewall+gateway is not blocking anything on localnet. Please let me know how to fix this.

Miuku · July 16, 2014, 10:24am

Set LogLevel to DEBUG2 in /etc/ssh/sshd_config -> LogLevel DEBUG2

Then start the service (systemctl start sshd) and look at /var/log/messages to see why the service isn’t starting.

Miuku · July 16, 2014, 11:12am

Come to think of it - did you remove # in front of the HostKey before you started sshd for the first time?

If so, the system may expect that you have your own hostkeys and doesn’t try to create them on the first startup.

Suworow · July 16, 2014, 12:05pm

Check your firewall settings first of all.
Have a look into the firewall log.

A common pitfall:
You setup ssh in YAST and you open the ssh port there.
But this will open port 22 only.
Your port is NOT the common ssh port 22, however.

This might be the source of the trouble.

Be sure your ssh port is open in your firewall.

chigurh · July 16, 2014, 1:08pm

Well, I set the firewall to default and then opened my custom port. Now it all started working. I think I got confused with gui firewall configuration. Editing /etc/sysconfig/SuSEfirewall2 is much more simpler than this YaST firewall gui.

Thanks everyone!