Broken Cyrillic encoding at ssh connection

Good afternoon!

I am experiencing this problem since installing openSUSE 42.3. In previous oS versions there were no problems with this.

At my machine Cyrillic letters are displayed properly. But at remote host, connected through ssh, I obtain such result:


$ ls task/SU2DATA/
?????????????????? ??????????

And the most unpleasant thing is that it multiplies substrings when I try to edit string with Cyrillic text through mcedit or vi.
This is an initial line (there is Russian phrase “Магнитная масса” between “SU2DATA/” and “/TBCDGT_PHI=0.08m”):

*OUTPUTPATH = …/SU2DATA/�~\агни�~Bна�~O ма�~A�~Aа/TBCDGT_PHI=0.08m/4x14x14x14/b3.02/tw/TeslaK40s-02/
*When I just delete the last “4” in “4x14x14x14” substring, I obtain
*OUTPUTPATH = …/SU2DATA/�~\агни�~Bна�~O ма�~A�~Aа/TBCDGT_PHI=0.08m/4x14x14x1x14x14x14/b3.02/tw/TeslaK40s-02/

*This behavior is the same at the all remote machines I tried. Some of them return the following message at the connection:
/usr/bin/manpath: can’t set the locale; make sure $LC_ and $LANG are correct

*The result of locale command at my machine is


> locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME=ce_RU.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

at the remote machine (with error message…):


> locale
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME=ce_RU.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Could anybody help me to fix this, please?

In Internet I have found some advices to fix this, but they don’t work for me. I have tried to add the following lines into ~/.ssh/config file:
*Host **
SendEnv LC_
to add the following line into /etc/ssh/ssh_config:
SendEnv LANG LC_

sa well as
SendEnv LC_TIME=“ce_RU.UTF-8”
SendEnv LC_ALL=“en_US.UTF-8”
the following lines into /etc/environment:
LANG=“en_US.UTF-8”
LC_TIME=“en_US.UTF-8”
LC_ALL=“en_US.UTF-8”
Nothing helps.

Please post output of “env | grep -E ‘LANG|LC_’” from this system.


> env | grep -E 'LANG|LC_'
**LANG**=en_US.UTF-8
**LC_**TIME=ce_RU.UTF-8

Hmm … looks good. Could you do “strace -f -o /tmp/locale.strace locale” and upload /tmp/locale.strace to http://susepaste.org/?

This is that information from my machine: http://susepaste.org/57888716

Should I do this at a remote host?

And, may be this is important, when I login to remote machine from tty command line, I don’t obtain any error messages. The Cyrillic encoding is however broken, but at my tty it is broken independently of ssh (this is not principal for me since I usually don’t use tty, I normally use it just in case of managing video drivers).

And if I go to the remote host from mc through Shell link, the Cyrillic text is displayed properly (Midnight Commander is installed from GitHub).

Yes, that’s where it fails, right?

http://susepaste.org/75847661

Yes, that’s where it fails, right?

Yes. It fails at all remote hosts.

What OS is used on remote hosts? The obvious difference is, that on remote host there is file /usr/lib/locale/locale-archive which is not present on local host. I know that Ubuntu (actually likely Debian) is using this file in preference to “split” locale information.

That hosts have the following OSs:
Scientific Linux 6.6 (locale.strace is from there);
openSUSE 13.1;
openSUSE 42.1;
Debian 8 Jessie.

Although I don’t usually ssh into machines with a different locale setting, I thought this might be interesting enough to do a little research…

First,
the ietf description for ssh regarding locales and charactersets

Section 4.5 in the following document
Which verifies some things described by the @OP, primarily the fact the current problem started only when upgrading the one machine (ssh client) to 42.3.

https://tools.ietf.org/html/draft-ietf-secsh-architecture-22

That suggests that something likely changed in the default terminal emulator running in 42.3.
Besides running a physical tty (which is another factor verifying what is described in the ietf), you can also try running in xterm, which in openSUSE is the only console that isn’t enhanced. BTW - This also probably explains why your MC works, my guess is that since you installed it from github, it’s probably launching an xterm shell to be most compatible with all Linux systems.

If that works, then that might work for you… Just run your ssh sessions in xterm instead of whatever other console you’re running.
If this is what you decide to do, then I’d still ask you to post your issue to https://bugzilla.opensuse.org for further work.

Otherwise, I’m going to guess that the problem isn’t in your ssh configuration, it’s in the configuration of whatever console you’re running (konsole?)

For more about the various consoles installed in openSUSE
https://en.opensuse.org/Terminal

HTH,
TSU

Good evening!

Thank you all for the replies.

After some time of trials, we have found a fix with my friend. It works, if the LC_TIME variable is not sent:


> cat /etc/ssh/ssh_config | grep '.'| grep -v '# '
#       $OpenBSD: ssh_config,v 1.30 2016/02/20 23:06:23 sobrado Exp $
#KexDHMin 1024
Host *
    ForwardX11Trusted yes
    SendEnv LANG LC_CTYPE LC_NUMERIC LC_COLLATE LC_MONETARY LC_MESSAGES
    SendEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION
**#SendEnv LC_TIME**
    SendEnv LC_ALL
#

Now everything is fine:


> ssh natasha@prime 
Password: 
Last login: Fri Sep  8 20:49:59 2017 from 192.168.0.179
Have a lot of fun...
natasha@prime:~> locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
natasha@prime:~> ls task/SU2DATA/
beta_c  CC  PL  TBC  Магнитная масса  Разное

But which relation does local time have to encoding?

My best guess is that if program fails to set one locale category it leaves locale effectively undefined. What is output of “locale” on remote server when you are logged in locally in tty (on the same server you printed it earlier over ssh)? You say everything works when logged in locally? Also does locale archive on this system include ce_RU.UTF-8 locale? You can check with “localedef --list-archive”. E.g. here Ubuntu does not include it (I presume, it is generated based on enabled languages).

In my tty LC_TIME is set to en_US.UTF-8… So when I log in to a remote host from my tty via ssh (SendEnv LC_TIME is present in /etc/ssh/ssh_config), there are no errors and locale command returns that LC_TIME is set to en_US.UTF-8.

My laptop:


> localedef --list-archive
> locale -a | grep 'ce_RU'
ce_RU
> locale -a | grep 'en_US'
en_US
en_US.iso885915
en_US.utf8

Machine with Scientific Linux:


$ localedef --list-archive | grep 'ce_RU'
$ localedef --list-archive | grep 'en_US'
en_US
en_US.iso88591
en_US.iso885915
en_US.utf8
$ locale -a | grep 'ce_RU'
$ locale -a | grep 'en_US'
en_US
en_US.iso88591
en_US.iso885915
en_US.utf8

Machine with openSUSE 13.1:


> localedef --list-archive
> locale -a | grep 'ce_RU'
> locale -a | grep 'en_US'
en_US
en_US.iso885915
en_US.utf8

Yes, seems, actually there are no Russian settings at that hosts.

But when I write in the /etc/ssh/ssh_config file
SendEnv LC_TIME=“en_US.UTF-8”
and go through ssh to some host (from KDE konsole), it all the same sets LC_TIME to “ce_RU.UTF-8”…

Yes, I can trivially reproduce it with

LC_TIME=foo_bar *any_command*

then command shows similar behavior (at least it fails to display UTF-8 encoded file names). I did not expect it, but well, this is probably topic for GNU libc mailing list.

“ce” is not code for Russian language.

But when I write in the /etc/ssh/ssh_config file

Well, ~/.ssh/config takes precedence, and earlier you said you edited it. You need to verify all your configuration files.