"Cannot change locale" after installing alternative glibc

So it turns out I’m having to deal with some nasty and tricky open-heart surgery.

At work we depend on a piece of software that is “very proprietary” according to the vendor. They only provide packages for Ubuntu but I’d been repackaging it for OpenSuSE, which is the distro we use.
I was using the packages for Xenial (Ubuntu 16) to make ours for Leap but eventually ran into some internal issues and had to upgrade to a newer version, that is built for Bionic (Ubuntu 18).
The bionic package is better in many many ways, with much fewer random depedencies, but it relies on glibc 2.27 and, thus, won’t run at all on Leap. Still determined to stay away from the
nightmare that is Ubuntu, I’ve decided to do some really dodgy shenanigans and install glibc from Tumbleweed alongside. After making it work in a very CTRL-C/V way both in VMs and my own dev machine, as
well as our deployment and development docker containers, I’ve proceeded to work on a “controlled reproducible” solution in the form of an RPM package that would do and undo this mess gracefully.

However, what I thought was my final version for this package (after many many VM clones and test on a sacrificial lamb machine we have here) still had one last(?) final trick up its sleeve. Upon installing on
a fresh machine (luckily) we discovered that somehow the locale info got mangled. I’ve been trying to sort this out for a couple days now, without success and would like to know if anyone has any idea of where I
could run towards. I was half-hoping 15.1 would bump glibc but, no dice, so I need a solution to last and the vendor of the software refuses to help despite me literally offering to work for them for free to get a
release they could stick in their site, and I’m really not keen on moving to Ubuntu because <insert quite disrespectful opinion of that distro here>.

Following is a description of how I’m going about this and stuff I’ve tried:

My first step was go find out what were the glibc packages and then get the ones from Tumbleweed (which is on 2.29). They actually don’t match
the ones I’m looking at in leap are: glibc glibc-extra glibc-locale and maybe glibc-i18ndata
On Tumbleweed, they’re: glibc glibc-extra glibc-locale glibc-locale-base libcrypt1 and, again, maybe glibc-i18ndata

I’ve extracted the contents of those packages and removed duplicates and symlinks:
the /etc files all looked the same, libs with repeated names suchs libBrokenLocale.so.1 were renamed to match the others.
Folders that are present in the base install also got renamed so it’s easier to undo the mess:
/usr/lib/locale becomes /usr/lib/locale-2.29
The same is true for the binaries provided by these packages:
/usr/bin/gencat getent iconv ldd locale localedef
/usr/sbin/glibc_post_upgrade iconvconfig

after all this is done, tar everything nice for the RPM.

The SPECFILE specifies:


# make sure we keep all the "original versions"
cp -f /lib64/libSegFault.so       /lib64/libSegFault-2.26.so



# Write all the new symlinks and copies
# During these two steps I also have to backup and "mangle" libowcrypt-2.26 so ldconfig doesn't overwrite
# the symlinks I manually make for libcrypt.so.1 from TW

cp --remove-destination /sbin/ldconfig-2.29 /sbin/ldconfig

ln -snrfv /lib64/libBrokenLocale-2.29.so   /lib64/libBrokenLocale.so.1
ln -snrfv /lib64/libSegFault-2.29.so       /lib64/libSegFault.so


ln -snrfv /lib64/ld-2.29.so  /lib64/ld-linux-x86-64.so.2
LD_PRELOAD=/lib64/libc-2.29.so ln -snrfv /lib64/libc-2.29.so  /lib64/libc.so.6

# Update system config (?)



# Before we start removing stuff, link back to the old versions

# recover the "unmangled" original version of the file
cp --remove-destination /lib64/glibc2-26.back/libowcrypt-2.26.so /lib64/libowcrypt-2.26.so

ln -snrfv /lib64/libBrokenLocale-2.26.so   /lib64/libBrokenLocale.so.1
ln -snrfv /lib64/libanl-2.26.so            /lib64/libanl.so.1
ln -snrfv /lib64/libcrypt-2.26.so          /lib64/libcrypt.so.1



# Copy all remaining things. Remove extraneous copies
rm -f  /usr/lib64/gconv || true
cp -r --remove-destination /usr/lib64/gconv-2.26 /usr/lib64/gconv


rm -f /usr/sbin/glibc_post_upgrade-2.26
rm -f /usr/sbin/iconvconfig-2.26

# Update system config (?)


Which seems to make almost everything in the computer work fine. The one observed issue is that the computer will hang upon opening any YaST applet/gem/item.

I haven’t yet found a different test case, but this is pretty consistent. Upon playing around it’s looking a lot like the problem is locale-related, as my initial issue was with
missing locales and whatnot, with the “Failed to set locale. Fix your System” message, Now it just hangs.
Somehow my own dev machine, which I deliberately did the manual procedure before the package being ready, as I needed quick confirmation is working fine, which is quite baffling,

I realise this is very “can you just not bloody do this instead?” but it’s looking like the answer is “not really” because of that piece of software.It’s also possible that locale IS fixed and that what’s causing
stuff to hang is something else. And also having this package installed makes stuff we build not portable to “regular and proper” opensuse installs, so it’s still a bad solution.
But I either get that stuff running or I have to hop distro. and given the problem is precisely glibc, this problem is non trivial. So any thoughts are appreciated.

So… following me collecting my thoughts to post I started digging into yast itself, so I could try and pinpoint the problem and/or get more information.

A couple hours of digging later I found the culprit: it was** snapper
Yast preemptively calls snapper when it’s about to fire a module so you can have a “rollback point” and this snapper call is what was hanging.
I don’t really know why and I’m not sure how to trace it or whether it’s worth it. But I’ve found a workaround, which in my case is bringing yet more
packages from Tumbleweed, in this case:

**I’ve only done preliminary testing so far, but it looked like installing the new version fixed the issue (famous last words).
So I’m leaving this here in case anyone else needs to go down this nasty path

Forcibly re-installing the GlibC and Locale packages didn’t help?

And, ‘/var/log/zypp/history’ didn’t contain any hints as to what happen during the installation?
[HR][/HR]Maybe we could ask for some YaST Internals documentation …

Ever tried faking version numbers? Every now and then an install will fail because of version number conflicts. I often succeed with installing anyway and providing the requested version by linking to the installed version, such as ln /lib64/libanl-2.26.so /lib64/libanl-2.27.so

AFAIK snapper should not do anything unless you call libzypp… or boot or shutdown. Or, your machine has been running for a long time and a scheduled maintenance snapshot is made.

I would not recommend bringing in additional snapper components from TW, if anything I would recommend simply disabling snapper so it couldn’t possibly do anything while you’re compiling if you believe it’s affecting your work.

I also assume that you’re not dynamically installing anything as part of your compile script, that would be mind-blowingly risky… I assume that you are either installing your dependencies separately or if in your script any installation is kept completely separate from your compilation.