gpg problem when using accented characters

Hi,

I had a LEAP 42.3 distro and made a smooth upgrade to LEAP 15.0 using the online method. That is making a simple update of the repo’s and zypper dup the thing as explained in the documentation and not a clean install as I used to do for a long long time.
All went well until I tried to decrypt some gpg files I had on my pc since maybe version 12 something of OpenSuSE. Inever had a problem in OpenSuSE decrypting those files.
The files were encrypted with the symmetric method, no keys needed.

There are Multiple problems I was detecting while trying to debug this very serious problem.
I will report some but those are not the subject of this thread.

– First choosing in yast language and Portuguese does not work.

since when I log into a bash session and make:

bash$ locale -a

shows the option pt_PT.utf8

but on the cli/bash the command locale shows that there is no such pt_PT.utf8, but rather pt_PT.UTF-8 … this is a potential bug for the distro.
and the error
setlocale: LC_TYPE: cannot change change Locale (en_US.UTF-8, pt_PT.UTF-8) no such file
shows up every time I log either in X mode on in the terminal text mode (ctrl+alt+F1).

For me there is no problem since my keyboard layout is correct, Portuguese, and I never had problems using en_US.UTF-8 as the cahr set anyway. I just found the problem.

Second:

I noticed that everytime I log there is error on LC_CTYPE locale line since the default locale shows that :

LC_CTYPE = en_US.UTF-8, LC_ALL=en_US.UTF-8

that line is a bug for sure!

So I had to place in my ~/.profile the change on environment variables so I added the line:

export LC_ALL="en_US.UTF-8"

and now the correct locale shows up like:

 
 :~> locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

The problem:
But the reason that brought me here was other.
I use accented characters on my passphrase. Always did. As well as some altgr characters and also of course shift+altgr and shift+accented characters.
That is, to expand a LOT the amount of characters space on symmetric gpg encrypted files.
Since if one only uses ascii words the amount of chars on that space is very limited.
(upper case, lower case, symbols on the keyboard).
I always add a lot of UTF-8 characters:
Namely using the altgr, altgr, and specially Composed characters. Meaning symbol+chars, symbol +shift+characters.
This increases very very much the amount of chars available and makes the passpharse way stronger the simply using ascii characters.

The problem is I upgraded from LEAP 42.3 to 15.0 and now none of my gpg encrypted files opens :frowning:
The result is always the dreadful bad session key … meaning …wrong passphrase.

This happened to me a LOT when moving such files and trying to open between different distros and computers since of course compose characters and altgr support varies once we change keyboard layout and specially Locale settings.
the strange thing is that I think this is solelly a problem of pinentry since I can see the characters I use when typing on the bash prompt.
I even used cli password entry did not work:
gpg -d --batch --passphrase ‘password’ encrypted_file

using a new file encrypted without algr and/or composed chars worked fine as expected with both pinentry and cli entry mode.

I also made a test that is using pinentry-curses.
Simply place a :

pinentry-program /usr/bin/pinentry-curses

on the file ~/.gnupg/gpg-agent.conf
and the ncurses version shows up with a strange behavior:

Every time I type a altgr or accented char the cursor moves two asteriscs on that prompt. As if pinentry does not even recognize or uses the system locale and does not understand a accented/altgr character!
(same for the other with shift+altg and shift+symbol+char).
Can anyone know what is going on with this strange behavior?
And don’t feel rushed to help guys :slight_smile: I’m certain that my electricity provider company at the end of this month is going to fully understand when I call the late payment department saying:
"… I have my bank access codes inside a symmetric encrypted file with gpg, but there’s an issue with Character set encoding with gpg-agent or pinentry not fully acquiring environment variables to correctly translate keyboard strings of unicode UTF-8 input characters … I wonder if you guys don’t have a decryption department with multiple peta/exa-hash brute force capability since at least you guys have the electricity to run that thing … "
I’m truly sure they’ll be delighted to hear that.

Regards.

See if your locales issues is answered in this current Forum thread, looks the same issue to me that the line you identified which specifies two locales should be broken out into two separate entries.

https://forums.opensuse.org/showthread.php/535268-YAST-MULTILINGUALISM

If your problems were the result of an upgrade, this issue should be reported as a bug to https://bugzilla.opensuse.org.
You can reference this and the other thread in your report.

Setting that in an acceptable form might also give you back support for the back ticks if you’re using them in Portuguese characters.
If you still have problems, post again…

TSU

Hi,
Thanks for the head’s up.

I also knew the problem with LC_CTYPE was clearly an issue.
But I already solved the thing.
The issue was a bug on /etc/locale.conf
my current /etc/locale.conf looks like this:

#LC_CTYPE=en_US.UTF-8,LANG=en_US.UTF-8
LANG=en_US.UTF-8

I commented the initial LC_CTYPE line that was clearly wrong on that file and that was a install update problem. I think identifying this can help a lot the debug of this issue.
I then commented and changed the LANG environment variable and all is ok.
I started by using ~/.profile to make a export of LC_CTYPE, but changing on /etc/locale.conf fixes the issue for the all OS.

But my “saga” did not end there … :slight_smile: since changing that still did not solve my gpg encryption problem not even close.
I discovered that not only the LC_CTYPE was wrong but also I had a strange value on keyboard type: microsoftpro ?!? never had such keyboard. And that was also a problem. Will be back on that issue latter.

About the problem of gpg input/passphrase with UTF-8 characters.
Has mentioned on the previous post I was indeed a bit worried. I was facing true data loss since the files are virtually “un-crakcable” under brute force attacks.
I started to wonder around and discovered several posts talking about a problem with lack of support for UTF-8 on gpg, or gpg-agent.
https://bugzilla.redhat.com/show_bug.cgi?id=704495
gpg uses pinentry as the default input agent application.

Before we go into the details I must alert for the fact that there is a Big difference between a Console and Xterm/Kterm or any variation of a X terminal.
On my console (ALt+F1) I still today can not for example display the chars I want … yet showconsolefonts shows all the correct chars possible on the keyboard! But has I focused on the Xterminal that is configurable by X11 I did not went deep into this possible problem.
I worked only on Xterminals.
So I tried to see if pinentry that was indeed the problem.

I made all necessary changes so that locale shows like the following :

 locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

and

localectl
   System Locale: LANG=en_US.UTF-8
       VC Keymap: pt
      X11 Layout: pt
       X11 Model: pc105
     X11 Options: terminate:ctrl_alt_bksp

This setup varies depending on the particular distro and on our OpenSuSE some files determine keymap and LANG:
/etc/vconsole.conf
and also
/etc/locale.conf
(btw /etc/vconsole.conf is just basically selecting KEYMAP:)

KEYMAP=pt
FONT=eurlatgr.psfu
# FONT_MAP=none
FONT_UNIMAP=

On other Distro’s it can be done simply on the graphical configuration tool. For the most part one simply chooses the keyboard at install time and language and all get done with no problem.
I remember everyone that this should be done on install with no problem. If not yast should do the trick.
So I tried:

pinentry-curses                                                                                                                                                                   
OK Pleased to meet you
setdesc Ł&Ł¢
OK
option lc-ctype=en_US.UTF-8
OK
getpin
D ƧЪæ©
OK
bye
OK closing connection

this allows us to make sure that option lc-ctype=en_US.UTF-8 is working correctly.
This combined with the setdesc was enough to determine the input was indeed using en_US.UTF-8 on pinentry. Btw editing ~/.gnupg/gpg-agent.conf
and adding at the end of the file:

pinentry-program /usr/gbin/pinentry-curses

makes gpg-agent start always on text mode with the ncurses version.
I also made sure that the Xterm or the Kterminal was running the correct setup of keyboard (pt105, not microsoftpro) and the correct environment: LANG=en_US.UTF-8 supporting UTF-8 has mentioned before.
Then I tested on the keyboard on the X terminal and the right chars were displaying.
I then proceeded to the next step : I made several VirtualBox guests, Centos7, Ubuntu, Tails, Puppy Linux and LEAP 42.1, LEAP 42.2 and LEAP 42.3 AND even LEAP 15.0 … (Yup, I made all of those) just to compare the console and Xterm environment setups and Also to test if they could recover the files encrypted between different OS’s.
I also made all of those VBox guest OS’s in order to make sure I would be testing on different OS’s with the same Xterm setup for LC_CTYPE, KEYMAP and the like. That is key (pun intended) for Portability and Verification on gpg.
That is support for UTF8 And the same keyboardmap as well as the same language.
For the sake of simplicity I used en_US.UTF-8.

All of those OS’s were set with that same parameters … and let me tell you I found amazing stuff … from getting different results on locale and localectl on some! to changing mapping and getting different results on chars that were supposed to be the same.
Anyway, after making sure all environments were the about the same, that is: pt keyboard mapping and LC_CTYPE=en_US.UTF-8 I encrypted a small file on my computer and tested to see if the VB guests would decrypted it using all types of characters previously mentioned that is :

Upper Case
Lower Case
Numbers
Symbols
AltGr
shift + AltGr
Accented characters
Shift+ Accented characters

And it worked like with no problems. All OS’s decrypted the file as long as the environment variables stayed the same. (within limits since for Example Centos7 does not have a pt keyboard …they refer to pt-latin1 but it’s the exact same type.)
So I had to realize that my files were encrypted from the get go with a different keyboard mapping… so when I updated LEAP maybe the settings change and I was inputing a wrong passphrase.
Therefore the files were only recoverable if I knew the exact settings/mappings used on the previous version of LEAP 42.3. And that was very very annoying to say the least (I would have to retry a lot of keyboards mappings/LAng combinations … a lot of work).
I forgot to mention btw all files were encrypted with Symmetric encryption like this:

gpg -c --s2k-mode 3 --s2k-count 65011712 --s2k-cipher-algo AES256 --cert-digest-algo SHA512 filename.txt

But the problem is now solved.
How did I manage to solve the issue you may wonder?
You may be thinking I got some sort of super Massive cryptocurrency asics attacking the password … nope. No brute force will work … I resorted to ingenuity … or to be precise … conscientious caution on when dealing with IT issues :slight_smile:
More precisely: -> Backups … backups … backups …
Had an old laptop with the same configuration and LEAP 42.3 and placed there the files … it decrypted the thing with no problems!
I must say I did not checked the entire configuration … I supposed the problem was the keyboard layout on the microsoftpro type of keyboard … btw I never had such keyboard, it simply was perhaps the default upon install (can not confirm that also), and has it worked and I never used the files on other computers I never noticed the problem. Until the update that is. And I suspect that was the problem all along (apart from the upgrade mis-configurations).
Using the microsoftpro layout all along and maybe something changed on those … and I found the mistake on the enctypted files.
Lessons learned for everyone:
Make Strong passpharses but remember to check the environment that was used on those passphrases … this is of paramount importance to assure Portability of gpg encryption!
Otherwise you get the same problem I did.

Regards.

Thx for posting, may be a useful guideline for someone else troubleshooting something similar.
And, congrats on figuring out what your problem was, and that the problem turns out to be something different than what you originally thought which can be tricky.

As for the different way environments are set up on different distros…

Yup.

TSU

Hi,

Oh yes … configuration of environment variables was a pain … each distro tries their best to change the location of configuration files as best as they can … and that is not a Linux problem … it is really the Other distro guys … just sayin’ … :slight_smile:

But this issue is of paramount importance for gpg usage and portability.
And I’m not even considering in here other inferior OS’s (cof cof … windoze) since anyone encrypting something of value on those said os’s does not understand the meaning of the word security … let’s just leave it there.
Increasing enormously the passphrase space improves some orders of magnitude the strength of the passphrase …and it literally grows exponentially with the increase on number of characters on the passphrase. :slight_smile:
Ascii chars+numbers+symbols it’s about 100 chars.
With altgr(utf8) chars + altgrSymbols + accented/composed characters we’re talking around 220 … more then Double …
So if you had a passphrase space of say 100 and N characters long you have :

(100)^N possible combinations.
with the altgr(utf8) and accented trick (+shift) we get 220 chars that is :
for simplcity let’s say 200
(200)^N = 2^Nx(100)^N
If N is say 16 chars long you get 2^16 more possibilities = 65536
If N is say 20 chars long (a long pass) you get 1048576.
That’s a Million more possibilities then without utf8 chars.

Now that I remember I think I had this same problem in the past, in a different way tho! Let me check the posts … I even contacted the gpg team … but it’s not a issue they can solve. They are inputing the passphrase correctly.
here! -> https://forums.opensuse.org/showthread.php/507080-Gnupg-bug-or-Desktop-Session-problem?highlight=gpg

On that post it was a change on the desktop …LXDE …
But this issue can have more consequences. Imagine you encrypt your files and then you need to open those on say a remote server, usually with a minimal install.
I work on some of those on a couple of very well known server hosting companies.
Usually servers that are online for production purposes use a minimal configuration in order to be safer and less “loaded/bloated”.
It helps both in terms of security as well as on maintenance.
But those configurations usually lack support for things we take for granted when using a desktop.
I had experiences with that on Centos6 and 7 installs.
And on those scenarios it could be problem for portability/usability of gpg.
Like the previous one I had with with LXDE.
Anyway …issue solved.

Regards.