AWK does not respect FS '|' in a field containing (15.6)

zypper lr -d
return lignes like that :
1 | NVIDIA:repo-non-free | repo-non-free (15.6) | Yes | (r ) Yes | Yes | - | 99 | rpm-md | https://download.nvidia.com/opensuse/leap/15.6 | NVIDIA
I want to extract frield 1, 2,3,10
That should give :
1
NVIDIA:repo-non-free
repo-non-free (15.6)
https://download.nvidia.com/opensuse/leap/15.6
But I got :
1
NVIDIA:repo-non-free
repo-non-free
(15.6) https://download.nvidia.com/opensuse/leap/15.6
The piece of code is :
read NUM ALIAS NAME URI <<< $( echo ${PARAM} | awk -F "|" '{print $1 $2 $3 $10}' )

Any help is welcome.

I have done a fair bit with awk over the years.

What happens if you run:

zypper lr -d | awk -F "|" '{print $1 $2 $3 $10}'

?

Without the rest of the code (showing how these variables are being defined/extracted), it’s kind of difficult to see what the issue might be.

It would also be good to know which awk you have installed.

user_install@15-6-G815LM-JC:~> zypper lr -d | awk -F "|" '{print $1 $2 $3 $10}'
#   Alias                            Name                                                                  URI                                                                                  
---+---------------------------------+----------------------------------------------------------------------+---------+-----------+---------+------+----------+--------+--------------------------------------------------------------------------------------+---------
 1  NVIDIA:repo-non-free             repo-non-free (15.6)                                                  https://download.nvidia.com/opensuse/leap/15.6                                       
 2  openSUSE-Leap-15.6-1             openSUSE-Leap-15.6-1                                                  hd:/?device=/dev/disk/by-id/usb-JetFlash_Transcend_8GB_0793PK7KZ3DM4PI4-0:0-part2    
 3  openSUSE:repo-non-oss            repo-non-oss (15.6)                                                   http://cdn.opensuse.org/distribution/leap/15.6/repo/non-oss?mediahandler=curl2       
 4  openSUSE:repo-non-oss-debug      repo-non-oss-debug (15.6)                                             http://cdn.opensuse.org/debug/distribution/leap/15.6/repo/non-oss?mediahandler=curl2 
 5  openSUSE:repo-openh264           repo-openh264 (15.6)                                                  http://codecs.opensuse.org/openh264/openSUSE_Leap?mediahandler=curl2                 
 6  openSUSE:repo-oss                repo-oss (15.6)                                                       http://cdn.opensuse.org/distribution/leap/15.6/repo/oss?mediahandler=curl2           
 7  openSUSE:repo-oss-debug          repo-oss-debug (15.6)                                                 http://cdn.opensuse.org/debug/distribution/leap/15.6/repo/oss?mediahandler=curl2     
 8  openSUSE:repo-oss-source         repo-oss-source (15.6)                                                http://cdn.opensuse.org/source/distribution/leap/15.6/repo/oss?mediahandler=curl2    
 9  openSUSE:update-backports        update-backports (15.6)                                               http://cdn.opensuse.org/update/leap/15.6/backports?mediahandler=curl2                
10  openSUSE:update-backports-debug  update-backports-debug (15.6)                                         http://cdn.opensuse.org/update/leap/15.6/backports_debug?mediahandler=curl2          
11  openSUSE:update-non-oss          update-non-oss (15.6)                                                 http://cdn.opensuse.org/update/leap/15.6/non-oss?mediahandler=curl2                  
12  openSUSE:update-non-oss-debug    update-non-oss-debug (15.6)                                           http://cdn.opensuse.org/debug/update/leap/15.6/non-oss?mediahandler=curl2            
13  openSUSE:update-oss              update-oss (15.6)                                                     http://cdn.opensuse.org/update/leap/15.6/oss?mediahandler=curl2                      
14  openSUSE:update-oss-debug        update-oss-debug (15.6)                                               http://cdn.opensuse.org/debug/update/leap/15.6/oss?mediahandler=curl2                
15  openSUSE:update-sle              update-sle (15.6)                                                     http://cdn.opensuse.org/update/leap/15.6/sle?mediahandler=curl2                      
16  openSUSE:update-sle-debug        update-sle-debug (15.6)                                               http://cdn.opensuse.org/debug/update/leap/15.6/sle?mediahandler=curl2                
17  repo-backports-debug-update      Update repository of openSUSE Backports (Debug)                       http://download.opensuse.org/update/leap/15.6/backports_debug/                       
18  repo-backports-update            Update repository of openSUSE Backports                               http://download.opensuse.org/update/leap/15.6/backports/                             
19  repo-debug                       Debug Repository                                                      http://download.opensuse.org/debug/distribution/leap/15.6/repo/oss/                  
20  repo-debug-non-oss               Debug Repository (Non-OSS)                                            http://download.opensuse.org/debug/distribution/leap/15.6/repo/non-oss/              
21  repo-debug-update                Update Repository (Debug)                                             http://download.opensuse.org/debug/update/leap/15.6/oss/                             
22  repo-debug-update-non-oss        Update Repository (Debug, Non-OSS)                                    http://download.opensuse.org/debug/update/leap/15.6/non-oss/                         
23  repo-non-oss                     Non-OSS Repository                                                    http://download.opensuse.org/distribution/leap/15.6/repo/non-oss/                    
24  repo-openh264                    Open H.264 Codec (openSUSE Leap)                                      http://codecs.opensuse.org/openh264/openSUSE_Leap/                                   
25  repo-oss                         Main Repository                                                       http://download.opensuse.org/distribution/leap/15.6/repo/oss/                        
26  repo-sle-debug-update            Update repository with updates from SUSE Linux Enterprise 15 (Debug)  http://download.opensuse.org/debug/update/leap/15.6/sle/                             
27  repo-sle-update                  Update repository with updates from SUSE Linux Enterprise 15          http://download.opensuse.org/update/leap/15.6/sle/                                   
28  repo-source                      Source Repository                                                     http://download.opensuse.org/source/distribution/leap/15.6/repo/oss/                 
29  repo-update                      Main Update Repository                                                http://download.opensuse.org/update/leap/15.6/oss

+++++++++++++++++++++++++++++++++++++++++++++++++

START OF MY PROGRAM :

removed '/home/user_install/zypper_cmd_create_repos_from_cdn.sh'
zypper lr -d | tail -n +3 > /tmp/zypper_out.txt
cat /tmp/zypper_out.txt

 1 | NVIDIA:repo-non-free            | repo-non-free (15.6)                                                 | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | https://download.nvidia.com/opensuse/leap/15.6                                       | NVIDIA
 2 | openSUSE-Leap-15.6-1            | openSUSE-Leap-15.6-1                                                 | No      | ----      | ----    | -    |   99     | rpm-md | hd:/?device=/dev/disk/by-id/usb-JetFlash_Transcend_8GB_0793PK7KZ3DM4PI4-0:0-part2    | 
 3 | openSUSE:repo-non-oss           | repo-non-oss (15.6)                                                  | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://cdn.opensuse.org/distribution/leap/15.6/repo/non-oss?mediahandler=curl2       | openSUSE
 4 | openSUSE:repo-non-oss-debug     | repo-non-oss-debug (15.6)                                            | No      | ----      | ----    | -    |   99     | N/A    | http://cdn.opensuse.org/debug/distribution/leap/15.6/repo/non-oss?mediahandler=curl2 | openSUSE
 5 | openSUSE:repo-openh264          | repo-openh264 (15.6)                                                 | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://codecs.opensuse.org/openh264/openSUSE_Leap?mediahandler=curl2                 | openSUSE
 6 | openSUSE:repo-oss               | repo-oss (15.6)                                                      | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://cdn.opensuse.org/distribution/leap/15.6/repo/oss?mediahandler=curl2           | openSUSE
 7 | openSUSE:repo-oss-debug         | repo-oss-debug (15.6)                                                | No      | ----      | ----    | -    |   99     | N/A    | http://cdn.opensuse.org/debug/distribution/leap/15.6/repo/oss?mediahandler=curl2     | openSUSE
 8 | openSUSE:repo-oss-source        | repo-oss-source (15.6)                                               | No      | ----      | ----    | -    |   99     | N/A    | http://cdn.opensuse.org/source/distribution/leap/15.6/repo/oss?mediahandler=curl2    | openSUSE
 9 | openSUSE:update-backports       | update-backports (15.6)                                              | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://cdn.opensuse.org/update/leap/15.6/backports?mediahandler=curl2                | openSUSE
10 | openSUSE:update-backports-debug | update-backports-debug (15.6)                                        | No      | ----      | ----    | -    |   99     | N/A    | http://cdn.opensuse.org/update/leap/15.6/backports_debug?mediahandler=curl2          | openSUSE
11 | openSUSE:update-non-oss         | update-non-oss (15.6)                                                | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://cdn.opensuse.org/update/leap/15.6/non-oss?mediahandler=curl2                  | openSUSE
12 | openSUSE:update-non-oss-debug   | update-non-oss-debug (15.6)                                          | No      | ----      | ----    | -    |   99     | N/A    | http://cdn.opensuse.org/debug/update/leap/15.6/non-oss?mediahandler=curl2            | openSUSE
13 | openSUSE:update-oss             | update-oss (15.6)                                                    | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://cdn.opensuse.org/update/leap/15.6/oss?mediahandler=curl2                      | openSUSE
14 | openSUSE:update-oss-debug       | update-oss-debug (15.6)                                              | No      | ----      | ----    | -    |   99     | N/A    | http://cdn.opensuse.org/debug/update/leap/15.6/oss?mediahandler=curl2                | openSUSE
15 | openSUSE:update-sle             | update-sle (15.6)                                                    | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://cdn.opensuse.org/update/leap/15.6/sle?mediahandler=curl2                      | openSUSE
16 | openSUSE:update-sle-debug       | update-sle-debug (15.6)                                              | No      | ----      | ----    | -    |   99     | N/A    | http://cdn.opensuse.org/debug/update/leap/15.6/sle?mediahandler=curl2                | openSUSE
17 | repo-backports-debug-update     | Update repository of openSUSE Backports (Debug)                      | No      | ----      | ----    | -    |   99     | N/A    | http://download.opensuse.org/update/leap/15.6/backports_debug/                       | 
18 | repo-backports-update           | Update repository of openSUSE Backports                              | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://download.opensuse.org/update/leap/15.6/backports/                             | 
19 | repo-debug                      | Debug Repository                                                     | No      | ----      | ----    | -    |   99     | N/A    | http://download.opensuse.org/debug/distribution/leap/15.6/repo/oss/                  | 
20 | repo-debug-non-oss              | Debug Repository (Non-OSS)                                           | No      | ----      | ----    | -    |   99     | N/A    | http://download.opensuse.org/debug/distribution/leap/15.6/repo/non-oss/              | 
21 | repo-debug-update               | Update Repository (Debug)                                            | No      | ----      | ----    | -    |   99     | N/A    | http://download.opensuse.org/debug/update/leap/15.6/oss/                             | 
22 | repo-debug-update-non-oss       | Update Repository (Debug, Non-OSS)                                   | No      | ----      | ----    | -    |   99     | N/A    | http://download.opensuse.org/debug/update/leap/15.6/non-oss/                         | 
23 | repo-non-oss                    | Non-OSS Repository                                                   | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://download.opensuse.org/distribution/leap/15.6/repo/non-oss/                    | 
24 | repo-openh264                   | Open H.264 Codec (openSUSE Leap)                                     | No      | ----      | ----    | -    |   99     | rpm-md | http://codecs.opensuse.org/openh264/openSUSE_Leap/                                   | 
25 | repo-oss                        | Main Repository                                                      | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://download.opensuse.org/distribution/leap/15.6/repo/oss/                        | 
26 | repo-sle-debug-update           | Update repository with updates from SUSE Linux Enterprise 15 (Debug) | No      | ----      | ----    | -    |   99     | N/A    | http://download.opensuse.org/debug/update/leap/15.6/sle/                             | 
27 | repo-sle-update                 | Update repository with updates from SUSE Linux Enterprise 15         | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://download.opensuse.org/update/leap/15.6/sle/                                   | 
28 | repo-source                     | Source Repository                                                    | No      | ----      | ----    | -    |   99     | N/A    | http://download.opensuse.org/source/distribution/leap/15.6/repo/oss/                 | 
29 | repo-update                     | Main Update Repository                                               | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://download.opensuse.org/update/leap/15.6/oss                                    | 
30 | repo-update-non-oss             | Update Repository (Non-Oss)                                          | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | http://download.opensuse.org/update/leap/15.6/non-oss/                               |

CALLING FOO1 from FILE zypper_out.txt THROUGH XARGS OUTPUT

The parameter received in FOO1 is correct :

foo1 β€” PARAM IS : >>>1 | NVIDIA:repo-non-free | repo-non-free (15.6) | Yes | (r ) Yes | Yes | - | 99 | rpm-md | Index of /opensuse/leap/15.6 | NVIDIA<<<

but the following piece of code return bad result :

	read NUM  ALIAS  NAME  URI <<< $( echo ${PARAM} | awk -F "|" '{print $1 $2 $3 $10}' )

	echo .
 	echo "foo1 --- NUM   : $NUM"
 	echo .
 	echo "foo1 --- ALIAS : $ALIAS"
 	echo .
 	echo "foo1 --- NAME  : $NAME"
 	echo .
 	echo "foo1 --- URI   : $URI"

 	foo1 --- NUM   : 1
 	foo1 --- ALIAS : NVIDIA:repo-non-free
 	foo1 --- NAME  : repo-non-free
 	foo1 --- URI   : (15.6)  https://download.nvidia.com/opensuse/leap/15.6

(15.6) is missing in NAME
(15.6) is added in URI

Thank you for helping

user_install@15-6-G815LM-JC:~> awk --version
GNU Awk 4.2.1, API: 2.0
Copyright (C) 1989, 1991-2018 Free Software Foundation.

It has nothing to do with awk. Read how the read command (pun unintended) behaves in the bash manual (page). Count words that awk returns. You have 4 variables and 5 words.

1 Like

I concur with arvidjaar - this isn’t an awk issue, it’s the way the read command works in bash.

I would probably be inclined to write the entire parser in awk if it were me.

The other option is to look at the options for read in the bash man page, and use the -d delim argument to split it that way, rather than to use awk to do the parsing.

That does not works also when using an array

PARAM="1 | NVIDIA:repo-non-free            | repo-non-free (15.6)                                                 | Yes     | (r ) Yes  | Yes     | -    |   99     | rpm-md | https://download.nvidia.com/opensuse/leap/15.6                                       | NVIDIA"
DATA=( $( echo "$PARAM" | awk -F "|" '{printf $1"\n"  $2"\n"  $3"\n"  $10"\n" }'  ) )


echo

echo "${DATA[@]}"
return :
1 NVIDIA:repo-non-free repo-non-free (15.6) https://download.nvidia.com/opensuse/leap/15.6

and
for i in ${DATA[@]}; do echo $i; done
return :

1
NVIDIA:repo-non-free
repo-non-free
(15.6)
https://download.nvidia.com/opensuse/leap/15.6

And how using array is supposed to change the fact that shell sees five different words? What do you not understand in β€œshell splits its input in individual words separated by spaces”? How is shell supposed to guess that you want several consecutive words separated by arbitrary amount of white spaces to be one word?

Thank you very much. This explanation lights me up much better.

I had thought about it. I did a test while doing this:
read -r NUM ALIAS NAME URI < <(echo ${PARAM} | awk -F β€œ|” β€˜{print β€œ\β€β€œ$1 β€œ\”” β€œ\β€β€œ$2 β€œ\”” β€œ\β€β€œ$3 β€œ\”” β€œ\β€β€œ$10 β€œ\””}’)
But it didn’t help

I have solve my problem like this.

	INDICE=0
	while IFS="" read -r p || [ -n "$p" ] ; do
		echo .
		echo "CURRENT : $p"
		echo .
		((INDICE+=1))
		printf '%d : %s\n' "$INDICE" "$p"
		case "$INDICE" in
			"1")
			NUM="$p"
			;;

			"2")
			ALIAS="$p"
			;;

			"3")
			NAME="$p"
			;;

			"4")
			URI="$p"
			break
			;;
		esac

	done <  "$ZYPPER_OUTPUT_TO_USE"

But I would like to have a solution in one line .

If you have a direction that I can follow, I buy it.

Thank you for helping

correction of the above


echo ${PARAM} | awk -F "|" '{print $1"\n" $2"\n" $3"\n" $10"\n"}'  >  "$ZYPPER_OUTPUT_TO_USE"

INDICE=0
	while IFS="" read -r p || [ -n "$p" ] ; do
		echo .
		echo "CURRENT : $p"
		echo .
		((INDICE+=1))
		printf '%d : %s\n' "$INDICE" "$p"
		case "$INDICE" in
			"1")
			NUM="$p"
			;;

			"2")
			ALIAS="$p"
			;;

			"3")
			NAME="$p"
			;;

			"4")
			URI="$p"
			break
			;;
		esac

	done <  "$ZYPPER_OUTPUT_TO_USE"

I have read the bash manual. My level of English is insufficient to understand what is written, β€œor rather what is not written”. Moreover, there are very few examples. I solved my problem but in a somewhat inelegant way.

Thank you very much for helping.

1 Like
IFS='|' read .....

-d separates lines, not words.

Fair, I had not used read myself and seen that the behavior was different than what I expected from a cursory reading of the man page.

That seems to work if you add " |" in awk print $

#!/bin/bash
#
# ~/test.sh
#
#

trimstring_sed(){
	s="${1}"
	s="$(printf "${s}" | sed -z 's/^[[:space:]]*//')"
	s="$(printf "${s}" | sed -z 's/[[:space:]]*$//')"
	echo "${s}"
	return 0
}


OIFS="$IFS"
IFS='|'

echo
echo "foo | 1024 | bar (8.3) | /home/test" | awk -F '|' '{print $1" |", $2" |", $3" |", $4" |"}'
echo
read VAR1 VAR2 <<< $(echo "foo | 1024 | bar (8.3) | /home/test" | awk -F '|' '{print $1" |", $3" |"}')
echo "VAR1 : >>>$VAR1<<<"
echo "VAR2 : >>>$VAR2<<<"

VAR1=$(trimstring_sed "${VAR1}")
VAR2=$(trimstring_sed "${VAR2}")


echo "VAR1 : >>>$VAR1<<<"
echo "VAR2 : >>>$VAR2<<<"

IFS="$OIFS"

echo
echo "FINISHED"

Thank you very much @arvidjaar and @hendersj

1 Like

So, you take words separated by | and print them back separated by |. Oh, well … whatever works for you.

In any case - zypper output is intended for humans, not for automated processing. If you want machine parseable zypper output - use --xmlout option.

1 Like