Sort from script unexpected

English Programming/Scripting
1

The following is all from 15.6, but the same behavior results in TW, Slowroll & 15.5:

# zypper --no-refresh se -s util-lin suse-mod a-dem  | grep -Ev '32bit|debug|devel|srcp|openSUSE-20' | grep -E 'x86|noarch'| sort
   | Mesa-demo            | package    | 8.3.0-1.33           | x86_64 | OSS
   | Mesa-demo            | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
   | util-linux-tty-tools | package    | 2.39.3-150600.2.1    | x86_64 | OSS
   | util-linux-tty-tools | package    | 2.39.3-150600.4.3.1  | x86_64 | UpdateSLE
   | util-linux-tty-tools | package    | 2.39.3-150600.4.6.2  | x86_64 | UpdateSLE
 l | util-linux-lang      | package    | 2.39.3-150600.2.1    | noarch | OSS
 l | util-linux-lang      | package    | 2.39.3-150600.4.3.1  | noarch | UpdateSLE
 l | util-linux-lang      | package    | 2.39.3-150600.4.6.2  | noarch | UpdateSLE
i  | util-linux           | package    | 2.39.3-150600.4.3.1  | x86_64 | UpdateSLE
i  | util-linux-systemd   | package    | 2.39.3-150600.4.3.1  | x86_64 | UpdateSLE
i+ | Mesa-demo-egl        | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
i+ | Mesa-demo-es         | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
i+ | Mesa-demo-x          | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
i+ | suse-module-tools    | package    | 15.6.9-150600.3.3.3  | x86_64 | UpdateSLE
v  | Mesa-demo-x          | package    | 8.3.0-1.33           | x86_64 | OSS
v  | suse-module-tools    | package    | 15.6.10-150600.3.6.2 | x86_64 | UpdateSLE
v  | suse-module-tools    | package    | 15.6.7-150600.1.24   | x86_64 | OSS
v  | util-linux           | package    | 2.39.3-150600.2.1    | x86_64 | OSS
v  | util-linux           | package    | 2.39.3-150600.4.6.2  | x86_64 | UpdateSLE
v  | util-linux-systemd   | package    | 2.39.3-150600.2.1    | x86_64 | OSS
v  | util-linux-systemd   | package    | 2.39.3-150600.4.6.2  | x86_64 | UpdateSLE

# cat zypse
#!/bin/sh
zypper --no-refresh se -s $*  | grep -Ev '32bit|debug|devel|srcp|openSUSE-20' | grep -E 'x86|noarch'| sort
# zypse util-lin suse-mod a-dem
i+ | Mesa-demo-egl        | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
i+ | Mesa-demo-es         | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
i+ | Mesa-demo-x          | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
i+ | suse-module-tools    | package    | 15.6.9-150600.3.3.3  | x86_64 | UpdateSLE
i  | util-linux           | package    | 2.39.3-150600.4.3.1  | x86_64 | UpdateSLE
i  | util-linux-systemd   | package    | 2.39.3-150600.4.3.1  | x86_64 | UpdateSLE
 l | util-linux-lang      | package    | 2.39.3-150600.2.1    | noarch | OSS
 l | util-linux-lang      | package    | 2.39.3-150600.4.3.1  | noarch | UpdateSLE
 l | util-linux-lang      | package    | 2.39.3-150600.4.6.2  | noarch | UpdateSLE
   | Mesa-demo            | package    | 8.3.0-1.33           | x86_64 | OSS
   | Mesa-demo            | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
   | util-linux-tty-tools | package    | 2.39.3-150600.2.1    | x86_64 | OSS
   | util-linux-tty-tools | package    | 2.39.3-150600.4.3.1  | x86_64 | UpdateSLE
   | util-linux-tty-tools | package    | 2.39.3-150600.4.6.2  | x86_64 | UpdateSLE
v  | Mesa-demo-x          | package    | 8.3.0-1.33           | x86_64 | OSS
v  | suse-module-tools    | package    | 15.6.10-150600.3.6.2 | x86_64 | UpdateSLE
v  | suse-module-tools    | package    | 15.6.7-150600.1.24   | x86_64 | OSS
v  | util-linux           | package    | 2.39.3-150600.2.1    | x86_64 | OSS
v  | util-linux           | package    | 2.39.3-150600.4.6.2  | x86_64 | UpdateSLE
v  | util-linux-systemd   | package    | 2.39.3-150600.2.1    | x86_64 | OSS
v  | util-linux-systemd   | package    | 2.39.3-150600.4.6.2  | x86_64 | UpdateSLE

# cat zypse
#!/bin/bash
zypper --no-refresh se -s $*  | grep -Ev '32bit|debug|devel|srcp|openSUSE-20' | grep -E 'x86|noarch'| sort
# zypse util-lin suse-mod a-dem
i+ | Mesa-demo-egl        | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
i+ | Mesa-demo-es         | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
i+ | Mesa-demo-x          | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
i+ | suse-module-tools    | package    | 15.6.9-150600.3.3.3  | x86_64 | UpdateSLE
i  | util-linux           | package    | 2.39.3-150600.4.3.1  | x86_64 | UpdateSLE
i  | util-linux-systemd   | package    | 2.39.3-150600.4.3.1  | x86_64 | UpdateSLE
 l | util-linux-lang      | package    | 2.39.3-150600.2.1    | noarch | OSS
 l | util-linux-lang      | package    | 2.39.3-150600.4.3.1  | noarch | UpdateSLE
 l | util-linux-lang      | package    | 2.39.3-150600.4.6.2  | noarch | UpdateSLE
   | Mesa-demo            | package    | 8.3.0-1.33           | x86_64 | OSS
   | Mesa-demo            | package    | 9.0.0-lp156.87.5     | x86_64 | X11Xorg
   | util-linux-tty-tools | package    | 2.39.3-150600.2.1    | x86_64 | OSS
   | util-linux-tty-tools | package    | 2.39.3-150600.4.3.1  | x86_64 | UpdateSLE
   | util-linux-tty-tools | package    | 2.39.3-150600.4.6.2  | x86_64 | UpdateSLE
v  | Mesa-demo-x          | package    | 8.3.0-1.33           | x86_64 | OSS
v  | suse-module-tools    | package    | 15.6.10-150600.3.6.2 | x86_64 | UpdateSLE
v  | suse-module-tools    | package    | 15.6.7-150600.1.24   | x86_64 | OSS
v  | util-linux           | package    | 2.39.3-150600.2.1    | x86_64 | OSS
v  | util-linux           | package    | 2.39.3-150600.4.6.2  | x86_64 | UpdateSLE
v  | util-linux-systemd   | package    | 2.39.3-150600.2.1    | x86_64 | OSS
v  | util-linux-systemd   | package    | 2.39.3-150600.4.6.2  | x86_64 | UpdateSLE

Both my zypse script versions used to provide the same output as typing it all out as in the first code block above. This changed several weeks ago, as shown in the second and third code blocks. Can anyone suggest what I can do to the script to make it sort as expected again? Did something in bash or sort binaries change lately?

1 Like
2 
andrei@leap155:~/src/util-linux> locale | grep LC_COLLATE
LC_COLLATE="en_US.UTF-8"
andrei@leap155:~/src/util-linux> sort /tmp/foo
i  | u
   | M
andrei@leap155:~/src/util-linux> LC_COLLATE=POSIX sort /tmp/foo
   | M
i  | u
andrei@leap155:~/src/util-linux>
3

Yes, looking at glibc locale data, collating rules for en_US.UTF-8 (and likely for many others) ignore special characters and space.

4

@arvidjaar, thank you. For now at least, I’ve amended the script to cause the sorting I expect:

  zypper --no-refresh se -s $*  | grep -Ev '32bit|debug|devel|srcp|openSUSE-20' | grep -E 'x86|noarch'| LC_COLLATE=POSIX sort

But, the reason for the recent behavior change, consistent across multiple installations, remains to be determined.

5

For posterity - good and concise explanation of collating.

Unexpected sort order in en_US.UTF-8 locale - Unix & Linux Stack Exchange

6

All I get from that is en_US.UTF-8 locale sorting is broken. No way I can understand how something can logically “sort” ahead of nothing, even if 0x20 isn’t really nothing - all alphanumerics, 0x2B, 0x2D & other printable characters are bigger than 0x09 & 0x20.

1 Like
7

Turns out this wasn’t working in 15.5, so with some experimenting I found sort -f provides a result I can live with in 15.5, 15.6 & TW, without any concern over locale:

# cat /usr/local/bin/zypse
#!/bin/bash
echo "# zypper --no-refresh se -s $* | grep -v '32bit|debug|devel|srcp|openSUSE-20' | egrep 'x86|noarch'| sort -f"
zypper --no-refresh se -s $* | egrep -v '32bit|debug|devel|srcp|openSUSE-20' | egrep 'x86|noarch'| sort -f
# zypse vulkan
# zypper --no-refresh se -s vulkan | grep -v '32bit|debug|devel|srcp|openSUSE-20' | egrep 'x86|noarch'| sort -f
   | libgstvulkan-1_0-0                        | package | 1.24.5-1.1           | x86_64 | OSS
   | libvulkan_nouveau                         | package | 24.1.3-1699.387.pm.4 | x86_64 | Packman
   | libvulkan_nouveau                         | package | 24.1.3-1699.387.pm.4 | x86_64 | PackmanE
   | libvulkan_nouveau                         | package | 24.1.3-384.1         | x86_64 | OSS
   | libvulkan_radeon                          | package | 24.1.3-1699.387.pm.4 | x86_64 | Packman
   | libvulkan_radeon                          | package | 24.1.3-1699.387.pm.4 | x86_64 | PackmanE
   | libvulkan_radeon                          | package | 24.1.3-384.1         | x86_64 | OSS
   | Mesa-vulkan-overlay                       | package | 24.1.3-1699.387.pm.4 | x86_64 | Packman
   | Mesa-vulkan-overlay                       | package | 24.1.3-1699.387.pm.4 | x86_64 | PackmanE
   | Mesa-vulkan-overlay                       | package | 24.1.3-384.1         | x86_64 | OSS
   | python310-vulkan                          | package | 1.3.275.1-1.2        | noarch | OSS
   | python311-vulkan                          | package | 1.3.275.1-1.2        | noarch | OSS
   | python312-vulkan                          | package | 1.3.275.1-1.2        | noarch | OSS
   | typelib-1_0-GstVulkan-1_0                 | package | 1.24.5-1.1           | x86_64 | OSS
   | typelib-1_0-GstVulkanWayland-1_0          | package | 1.24.5-1.1           | x86_64 | OSS
   | typelib-1_0-GstVulkanXCB-1_0              | package | 1.24.5-1.1           | x86_64 | OSS
   | vulkan-headers                            | package | 1.3.283.0-1.1        | noarch | OSS
i  | libvulkan1                                | package | 1.3.283.0-1.1        | x86_64 | OSS
i  | libVulkanLayerSettings-1_3_283_0          | package | 1.3.283.0-1.1        | x86_64 | OSS
i  | Mesa-vulkan-device-select                 | package | 24.1.3-1699.387.pm.4 | x86_64 | Packman
i  | Mesa-vulkan-device-select                 | package | 24.1.3-1699.387.pm.4 | x86_64 | PackmanE
i  | vulkan-tools                              | package | 1.3.283.0-1.1        | x86_64 | OSS
i+ | libvulkan_intel                           | package | 24.1.3-1699.387.pm.4 | x86_64 | Packman
i+ | libvulkan_intel                           | package | 24.1.3-1699.387.pm.4 | x86_64 | PackmanE
i+ | libvulkan_lvp                             | package | 24.1.3-1699.387.pm.4 | x86_64 | Packman
i+ | libvulkan_lvp                             | package | 24.1.3-1699.387.pm.4 | x86_64 | PackmanE
i+ | vulkan-validationlayers                   | package | 1.3.283.0-1.1        | x86_64 | OSS
v  | libvulkan_intel                           | package | 24.1.3-384.1         | x86_64 | OSS
v  | libvulkan_lvp                             | package | 24.1.3-384.1         | x86_64 | OSS
v  | Mesa-vulkan-device-select                 | package | 24.1.3-384.1         | x86_64 | OSS
#
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.