Bash - General programming - How to send zypper output as a line by line string

Hello.
I want to do some work on output of a zypper command.
It seems that the zypper output is not simple lines terminated by /n.

I miss end of lines for what zypper has found.
And in the function, the program does not wait for user to press a key.

here is my program :

#! /bin/sh
########################################
#                                                    #
#    dnla_test_pre_requisite_dnla            #
#                                                    #
########################################

#
#ensure running as root
#
if  "$(id -u)" != "0" ]; then
  exec sudo "$0" "$@"
fi

function DO_IT () {

    local NB_VAR
    local DOLLAR_0
    local DOLLAR_ARO
    local B_VAR1

    NB_VAR="$#"
    DOLLAR_0="$0"
    DOLLAR_ARO="$@"
    B_VAR1="$1"

    echo
    echo "=================================================="
    echo
    echo
    echo "   -     NB_VAR=$NB_VAR"
    echo "   -   DOLLAR_0=$DOLLAR_0"
    echo "   - DOLLAR_ARO=$DOLLAR_ARO"
    echo
    echo
    echo "   - B_VAR1=$B_VAR1"
    echo
    echo
    echo
    echo "=================================================="
    echo
    echo

    echo
    echo "Type a key to continue"
    read MY_TEMP
    echo

}

export -f DO_IT



echo .
echo .
echo .

for A_LIB in libexif libjpeg libid3tag libFLAC libvorbis libsqlite3 libavformat ; do    # (libavformat : the ffmpeg libraries)

#    zypper se -s $A_LIB | while read -r line; do DO_IT "$line"; done  

#    zypper se -s $A_LIB | xargs -0 -n1 -I '{}' bash -c 'DO_IT "{}"' 

#    zypper se -s $A_LIB | xargs -0 -L 1 -I '{}' bash -c 'DO_IT "{}"' 

    zypper se -s $A_LIB | xargs -I '{}' bash -c 'DO_IT "{}"' 

done


I have tried different things but i did not success.

Any help is welcome

If you direct the zypper output to a file as with

sudo zypper search xxx > filename

each line will terminate with a 0a hex which you can change to what ever you want with “tr”.

As is often the case, it may help if you explain a few quick things:

  1. What is the use/business case for this?

  2. What is the exact output you are trying to manipulate? Paste it using
    the CODE/# tags using the ‘#’ button in the web UI.

  3. How would you like the output to look?

If we have the before/after we can perhaps come up with a way to script
it, or if we can understand the business case then perhaps wee can find a
better way to do what you are after.


Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.

If you want to send me a private message, please let me know in the
forum as I do not use the web interface often.

if a list of all installed packages is require try,

rpm -qa | sort -df > file

the output will be something like,

aaa_base-13.2+git20140911.61c1681-27.2.x86_64
aaa_base-extras-13.2+git20140911.61c1681-27.2.x86_64
aaa_base-malloccheck-13.2+git20140911.61c1681-27.2.x86_64
accountsservice-0.6.42-6.4.x86_64
accountsservice-lang-0.6.42-6.4.noarch
acl-2.2.52-10.15.x86_64
acpica-20170119-1.14.x86_64 
..... etc.

What is “/n”?

And in the function, the program does not wait for user to press a key.

Because “read” does not “wait for user” - it waits for a line on standard input. You do realize that pipe connects standard output of one program to standard input of your program? And that read simply gets whatever you pipe into this function?

IIRC “/n” is one of those “features” inserted at the end of the line when a Windows text editor saves the file… and usually then has to be removed before you can properly edit the file in Linux.

You might append the output each time you run your loop…

ie.
function >> outputfile

Don’t simply redirect as follows or you’ll over-write the file each time instead of appending

Don’t
function > outputfile

Similarly,
If you wish to print to stdout, then echo the function’s result as a last line in each loop.

TSU

Hi,

Like what some folks already said. If you can provide us the output or if you can try to explain what are you trying to achieve.
Also you have the sh shebang but calling bash -c in your script? Please explain what are you trying do.

Hi,

zypper can handle multiple args i.e. packages so the loop is not needed unless of course you need the loop (I hope not).

zypper se -s libexif libjpeg libid3tag libFLAC libvorbis libsqlite3 libavformat

In bash you can use an array to store the package names. Note you can change packages to anything you want since it is just an array name.

packages=(libexif libjpeg libid3tag libFLAC libvorbis libsqlite3 libavformat)
zypper se -s "${packages@]}"

In sh one can (ab)use the “$@” which the only array.

set -- "$@" libexif libjpeg libid3tag libFLAC libvorbis libsqlite3 libavformat

and print the output

printf '%s
' "$@"

using zypper

zypper se -s "$@"

Just guessing what are you trying to do here…

Without looking too hard at exactly what is happening, the core of the code appears to be the following loop which evaluates various libraries, extracts some codec information and then performs a zypper source search.

for A_LIB in libexif libjpeg libid3tag libFLAC libvorbis libsqlite3 libavformat ;do    # (libavformat : the ffmpeg libraries)
    zypper se -s $A_LIB | xargs -I '{}' bash -c 'DO_IT "{}"' 


done

I wouldn’t know if the syntax is flawed or not, but the higher level structure suggests he’s building a list so likely wants to print that list either to stdout or a file.

So, I suggested that this “For” loop should re-direct output in the ways I described to stdout or file each time the loop executes.

TSU

I will try it.

-1-
It is a very general question.
I type a command in a console.
I get a bunch of lines.
And so the first thought that came to mind was why not to use xargs to pass the output line by line to a function which can return true or false.
As I don’t use xargs very often I did not success.
Remark : The output may contains ‘CR’, ‘CRLF’, or ‘LF’ this is perhaps my problem.

-2-

zypper se -s libjpeg
Loading repository data...
Reading installed packages...

S | Name                  | Type    | Version     | Arch   | Repository            
--+-----------------------+---------+-------------+--------+-----------------------
i | libjpeg-turbo         | package | 1.3.1-36.13 | x86_64 | openSUSE-Leap-42.3-Oss
i | libjpeg62             | package | 62.1.0-36.1 | x86_64 | openSUSE-Leap-42.3-Oss
  | libjpeg62-32bit       | package | 62.1.0-36.1 | x86_64 | openSUSE-Leap-42.3-Oss
i | libjpeg62-devel       | package | 62.1.0-36.1 | x86_64 | openSUSE-Leap-42.3-Oss
  | libjpeg62-devel-32bit | package | 62.1.0-36.1 | x86_64 | openSUSE-Leap-42.3-Oss
  | libjpeg62-turbo       | package | 1.3.1-36.1  | x86_64 | openSUSE-Leap-42.3-Oss
i | libjpeg8              | package | 8.0.2-36.13 | x86_64 | openSUSE-Leap-42.3-Oss
i | libjpeg8-32bit        | package | 8.0.2-36.13 | x86_64 | openSUSE-Leap-42.3-Oss
  | libjpeg8-devel        | package | 8.0.2-36.13 | x86_64 | openSUSE-Leap-42.3-Oss
  | libjpeg8-devel-32bit  | package | 8.0.2-36.13 | x86_64 | openSUSE-Leap-42.3-Oss
  | libjpegxr0            | package | 1.1-3.14    | x86_64 | openSUSE-Leap-42.3-Oss

Remark : As it is a screen copy, you loose the raw data.

-3-
> 3. How would you like the output to look?

I don’t mind. I have raw data, and just do some job on that raw data.

**> If we have the before/after we can perhaps come up with a way to script
it, or if we can understand the business case then perhaps wee can find a
better way to do what you are after.
**
In that particular case I was just trying to learn how to use xargs.
I don’t really need a complicated script to verify that some libraries are not missing.

> What is “/n”?

I mean "
"

> Because “read” does not “wait for user” - it waits for a line on standard input. You do realize that pipe connects standard output of one program to standard input of your program? And that read simply gets whatever you pipe into this function?

No. I am got through it.
I completely missed that point.

         My first analysis was this:

1 °) I pass a command

2 °) I get a stream of data

3 °) It seems complicated to process all the data as a whole, so I will send the data line by line to a function.

4 °) Once the treatment is finished I test the result.

In my example I hope that

 .... **|** **xargs -I '{}' **bash -c 'DO_IT "{}"'

passe line by line the output command to the function ‘DO_IT’
( Or any other better syntax like xargs -0 -n1 -I ‘{}’ bash, xargs -0 -L 1 -I ‘{}’, …)

As my program does not print line by line the result of

zypper se -s libjpeg

, I just print the value of “$#”, “$0”, “$@”, “$1”
to see what happens.
I am just trying to understand why “xargs” is not sending the output of zypper command line by line.

zypper return a bunch of lines.
How to pass the whole output line by line ?
That is my question.

Did you try to start with reading xargs manual? If yes, what you do not understand in “delimited by blanks or newlines”?

Did you try to read xargs manual?

Hi,

If i remember correctly i once used xargs and have so may issues with my script, so after some hair pulling and head scratching ive found out that xargs breaks if your data/input has spaces in it. ie instead of Newdirectory you have** New Directory** or New filename instead of Newfilename. which is pretty common on raw/input data. Since you’re parsing zyppers output that might be messing with xargs. I suggest you capture first the output in a file or a stream then do your stuff, but good luck with xargs, in my experience the arrays in bash can solve what are you trying to do.

from man

-L max-lines
Use at most max-lines nonblank input lines per command line. Trailing blanks cause an input line to be logically continued on the next input line. Implies -x.

So I was expected to get one line at a time passed to the function.

I tried different things found on internet, but no chance.

If you mean that I don’t understand the use of xargs you are right. But that does not help.

This by itself implies that each result is multi-value, which is in itself an array(This aside from looping thus creating a list or array of arrays).

A general comment…

This is an example of spaghetti code, not as bad as some but still not completely modular.

Recommend…

  1. Don’t, or minimize embedding functions within functions.
  2. Properly format your code to better recognize your code structure.
  3. Simplify your tests as you build your code. So, for instance evaluate only one library, not all of the ones you list.
  4. Debug/Instrument your code by inserting ECHO statements to properly understand generated values at each point in your code. First step before figuring out how to display or use a result is to know exactly what that result looks like.
  5. May help, of course enclose all strings in quotes on the chance they have spaces in them.

I wrote the following Wiki article for first steps writing BASH code, and includes a link to SHELL CHECK which is an online code checker (verifies the integrity of your BASH code in real time).

https://en.opensuse.org/User:Tsu2/Really_Basic_Stuff_Scripting

TSU

Thanks. I will read this as soon as possible.