zypper: parsing output to human-readable form

English Programming/Scripting
#1

For automatic backup in 10.3 I use a script that calls zypper (0.8.25)
and mails me the output, like:

zypper up | $MAILER “$SENDMAILTO”

However, while working nicely on the command line zypper produces a
whole load of crap when the output is piped or redirected:

…
e[2K* Installing: timezone-2008c-0.1 [0%]e[2K* Installing:
timezone-2008c-0.1 [100%]e[2K* Installing: timezone-2008c-0.1 [0%]e[2K*
Installing: timezone-2008c-0.1 [50%]e[2K* Installing:
timezone-2008c-0.1 [0%]e[2K* Installing: timezone-2008c-0.1 [0%]e[2K*
Installing: timezone-2008c-0.1 [1%]e2K* Installing: timezone-2008c-0.1
…

Obviously a problem with this fancy and useless progress bar. With rug
and smart my script works just fine. Is there an easy way around this? I
want just simple output that tells me in a short email what has been
updated. Very much in the spirit of cron-apt on deb-based systems.

Guenther

#2

Yeah I noticed this too, but haven’t bothered to solve it. Does the -q option help? What about -t and just grep the messages at the end?

Seems to be a bug of zypper, it should test if stdout is a terminal before outputing cursor codes.

#3

ken yap wrote:
> Yeah I noticed this too, but haven’t bothered to solve it. Does the -q
> option help? What about -t and just grep the messages at the end?

The example I gave was with the --quiet option already. -t (output in
xml format) seems to be more promising:

zypper -t up | grep done
or
zypper -t up | grep message

Still have to test which xml types appear on various tasks, for example:
<progress id=“read-installed-packages” type=“done” name=""/>
<message type=“warning”>Nothing to do.</message>

> Seems to be a bug of zypper, it should test if stdout is a terminal
> before outputing cursor codes.

Indeed.

Guenther

#4

On Wed, 25 Jun 2008 12:50:25 GMT
Guenther Schwarz <strap@gmx.de> wrote:

>Obviously a problem with this fancy and useless progress bar. With rug
>and smart my script works just fine. Is there an easy way around this?
>I want just simple output that tells me in a short email what has been
>updated.

I’ve not used this, yet, but could the ‘–terse’ option be any help? If
you send me an email with your script, I’d be glad to look it over, see
what can be done with the extraneous output . . . :slight_smile:

–
Kevin Nathan (Arizona, USA)
Linux is not a destination, it’s a journey – enjoy the trip!

Linux 2.6.22.18-0.2-default
10:45am up 3 days 13:02, 21 users, load average: 0.28, 0.39, 0.43

#5

Kevin Nathan wrote:
> On Wed, 25 Jun 2008 12:50:25 GMT
> Guenther Schwarz <strap@gmx.de> wrote:
>
>> Obviously a problem with this fancy and useless progress bar. With rug
>> and smart my script works just fine. Is there an easy way around this?
>> I want just simple output that tells me in a short email what has been
>> updated.
>
> I’ve not used this, yet, but could the ‘–terse’ option be any help? If
> you send me an email with your script, I’d be glad to look it over, see
> what can be done with the extraneous output . . . :slight_smile:

Yes I switched from --quiet to --terse (xml output) since my last post.
This is what I’m testing right now (testing is cumbersome as one has to
wait for new stuff arriving on the mirrors in order to see any effect of
changes):

parsing terse ouput of zypper

cleaning and building repositories

test -e $TMPFILE && grep “done” $TMPFILE | grep “Cleaning” >> $MAILFILE
test -e $TMPFILE && grep “done” $TMPFILE | grep “Building” >> $MAILFILE

downloads

test -e $TMPFILE && grep "Downloading: " $TMPFILE >> $MAILFILE

installations

test -e $TMPFILE && grep “Install” $TMPFILE | grep “100” >> $MAILFILE

warnings

test -e $TMPFILE && grep “warning” $TMPFILE >> $MAILFILE

Note that ‘grep “done”’ gives a lot of empty lines of the form
<progress id=“download” type=“done” name=""/>
with the interesting stuff in the ‘name’ field missing. So I have to
search for the 100 from ‘value=“100”’ instead. But this does not work
for all operations. The percentage might jump to strange values instead
of going from 0 to 100.

Guenther

#6

Guenther Schwarz wrote:
> Kevin Nathan wrote:
>> On Wed, 25 Jun 2008 12:50:25 GMT
>> Guenther Schwarz <strap@gmx.de> wrote:
>>
>>> Obviously a problem with this fancy and useless progress bar. With
>>> rug and smart my script works just fine. Is there an easy way around
>>> this?
>>> I want just simple output that tells me in a short email what has
>>> been updated.
>>
>> I’ve not used this, yet, but could the ‘–terse’ option be any help? If
>> you send me an email with your script, I’d be glad to look it over, see
>> what can be done with the extraneous output . . . :slight_smile:
>
> Yes I switched from --quiet to --terse (xml output) since my last post.
> This is what I’m testing right now (testing is cumbersome as one has to
> wait for new stuff arriving on the mirrors in order to see any effect of
> changes):
>
> # parsing terse ouput of zypper
> # cleaning and building repositories
> test -e $TMPFILE && grep “done” $TMPFILE | grep “Cleaning” >> $MAILFILE
> test -e $TMPFILE && grep “done” $TMPFILE | grep “Building” >> $MAILFILE
> # downloads
> test -e $TMPFILE && grep “Downloading: " $TMPFILE >> $MAILFILE
> # installations
> test -e $TMPFILE && grep “Install” $TMPFILE | grep “100” >> $MAILFILE
> # warnings
> test -e $TMPFILE && grep “warning” $TMPFILE >> $MAILFILE
>
> Note that ‘grep “done”’ gives a lot of empty lines of the form
> <progress id=“download” type=“done” name=”"/>
> with the interesting stuff in the ‘name’ field missing. So I have to
> search for the 100 from ‘value=“100”’ instead. But this does not work
> for all operations. The percentage might jump to strange values instead
> of going from 0 to 100.

Now it looks much nicer already than without the --terse option:

<progress id=“48” type=“done” name=“Cleaning repository ‘GWDG1’ cache”/>
<progress id=“75” type=“done” name=“Cleaning repository ‘Packman
Repository’ cache”/>
<progress id=“114” type=“done” name=“Cleaning repository ‘GWDG4
(Packman)’ cache”/>
<progress id=“47” type=“done” name=“Building repository ‘GWDG1’ cache”/>
<progress id=“74” type=“done” name=“Building repository ‘Packman
Repository’ cache”/>
<progress id=“113” type=“done” name=“Building repository ‘GWDG4
(Packman)’ cache”/>
e2KDownloading: bind-libs-9.4.1.P1-12.2_12.4.i586.delta.rpm
e2KDownloading: bind-utils-9.4.1.P1-12.2_12.4.i586.delta.rpm
<progress id=“install-resolvable” type=“percentage” value=“100”
name=“Installing: bind-libs-9.4.1.P1-12.4”/>
<progress id=“install-resolvable” type=“percentage” value=“100”
name=“Installing: bind-libs-9.4.1.P1-12.4”/>
<progress id=“install-resolvable” type=“percentage” value=“100”
name=“Installing: bind-utils-9.4.1.P1-12.4”/>
<progress id=“install-resolvable” type=“percentage” value=“100”
name=“Installing: bind-utils-9.4.1.P1-12.4”/>

Now I know at least what repositories are used and which packages are
updated. Still need a list of possible error messages to look out for:
update servers not reachable, requests for clicking on license
agreements, problems with installations etc.

Guenther

#7

Well it’s XML after all so why not pass it through an XML parser and extract the elements you want? Maybe there’s a DTD somwhere.

#8

If you don’t care about the amount of progress lines you can filter out
the stupid special chars with this:

zypper -n up | tr -d ‘\033’ | sed ‘s/\2K\r/
/g’

#9

Marco Munderloh wrote:
> If you don’t care about the amount of progress lines you can filter out
> the stupid special chars with this:
>
> zypper -n up | tr -d ‘\033’ | sed ‘s/\2K\r/
/g’

Thanks, I always get confused with special characters.

Günther

#10

ken yap wrote:
> Well it’s XML after all so why not pass it through an XML parser and
> extract the elements you want? Maybe there’s a DTD somwhere.

Sure. But then why is such an utility not included in the package in the
first place? Zenworks Management? Anyway, I try to keep my scripts as
simple as ever possible. So I’m fine with some grep commands for the moment.

Günther

#11

You can write one quickly in any of the common languages available, Perl, Python, Ruby, heck even PHP in CLI mode. That’s what Linux/Unix is good for, creating simple tools quickly. Here’s one using Perl:

.#!/usr/bin/perl -w

use XML::Simple;
use Data::Dumper;

my $xs = XML::Simple->new;
my $config = $xs->XMLin($ARGV[0]);
print Dumper($config);

When run like this:

perl /tmp/parsexml.pl /etc/splashy/config.xml

It gives:

$VAR1 = {
          'current_theme' => 'openSUSE',
          'themes' => '/usr/share/splashy/themes',
          'pid' => '/etc/splashy/splashy.pid',
          'default_theme' => '/usr/share/splashy/themes/default'
        };

Of course in real use you would print out the desired element of the data structure. See the man page for XML::Simple.

#12

ken yap wrote:
> You can write one quickly in any of the common languages available,
> Perl, Python, Ruby, heck even PHP in CLI mode. That’s what Linux/Unix
> is good for, creating simple tools quickly. Here’s one using Perl:
>
>
> Code:
> --------------------
> .#!/usr/bin/perl -w
>
> use XML::Simple;
> use Data::Dumper;
>
> my $xs = XML::Simple->new;
> my $config = $xs->XMLin($ARGV[0]);
> print Dumper($config);
> --------------------

> Of course in real use you would print out the desired element of the
> data structure. See the man page for XML::Simple.

Very nice and handy, thanks for posting this script. I can’t say that
I’m familiar with perl, for me this is more like for Keith Bostic:

“Perl is the only language that looks the same before and after RSA
encryption.”

But nevertheless it works nicely and solves my problem.

Guenther