List of installed packages since OS installation

Hi

I have an opensuse 11.3 32bit desktop installation which has been improved by many package installations.
I am about to buy a laptop and want to put the same installation on it (or at least install the packages to the same level - I might use 11.4).

How can I easily get a list of all the packages I have installed on my desktop system since the OS was installed? Is rpm -qa the only way or is there an installation history stored somewhere which I can easily get at?

Thanks

awk -F "|" '!/^#/ { if ( $2 == "install" ) print $1, ":", $3"-"$4}' /var/log/zypp/history | more
yast2 --install

File → Export…

Thanks for your suggestions.

The ‘awk’ code looks good - I would like to distinguish between what I’ve manually installed and what has been installed automatically (either as updates or as dependencies) but I suppose this is very difficult.

I’m presuming that the obvious ‘yast2 --install’ option allows me somehow to filter only those packages which have been installed and to export the list (in xml format?) so that I could then import the same list on my laptop and execute an ‘install’ against all those listed packages.

As I’ve said I’d really like to get a list of the packages I’ve manually installed, as yast would then take care of the dependent packages. Any more suggestions?

its not clear to me that you also looked at:


rpm -qa --last

in order to provide a chronological list.

Yes I did look at rpm -qa --last. As I have said what I really want is a list of packages I have manually installed ideally not including automatic updates and dependencies.

Does the rpm database or any zypper history distinguish between manually installed packages and others (in particular dependencies)?

The output of the ‘awk’ script or the output of ‘rpm -qa --last’ is fine and would be perfect if it told me which packages had been manually installed by me and which had been installed as the result of dependency checks.

Does anyone know if this is possible?

> Does anyone know if this is possible?

i understand you seek an easy way to have the machine keep a list of
what you have installed…there probably is a way to do that, and i
follow this thread hoping someone will tell how…

until then, i’ve been trying to manually keep a list of the software i
install…its not perfect (because i sometimes forget to update it, of
course) but it sure helps when the time comes to build a new
system…you can see my list in my sig… you might notice it doesn’t
include all dependencies (i agree that would be too ‘messy’)…


DD
Caveat-Hardware-Software
openSUSE®, the “German Engineered Automobile” of operating systems!

#! /bin/bash

# display the packages included in the installed patterns. 
# You'll only have to install the patterns - not the packages! 
for pat in $(zypper pt -i | awk -F "|" '/^i/ { print $2 }' | sort -u | sed 's/^ //;s/ *$//;s/ /#/g') ; do
	pkg=$(zypper info -t pattern "${pat//#/ }" | awk 'BEGIN { ORS=" "} ; /^i/&&/| package |/{ print $3 }')
	 "$pkg" ] || continue
	printf "* %s
%s

" "${pat//#/ }" "$pkg"
	pkgs="$pkgs $pkg"
done


echo $pkgs | tr " " "
" | sort -u > /tmp/pkg0.lst
zypper se -i -t package | awk '/^i/ { print $3}' | sort -u > /tmp/pkg1.lst
diff /tmp/pkg{0,1}.lst | awk '/^>/ { print $2 }' > /tmp/pkg2.lst

# display dependencies installed by the remaining packages.
# You'll only have to install these packages. 
for pkg in $(cat /tmp/pkg2.lst) ; do
	dep=$(rpm -q --requires $pkg | sed '/(/d;/^\//d;s/ .*//;/\.so.[0-9]*/d' | sort -u | tr "
" " ")
	printf "* %s
%s

" $pkg "$dep" 		
	deps="$deps $dep"
done

echo $deps | tr " " "
" | sort -u > /tmp/pkg3.lst

# display remaining packages to install
echo "* remaining packages: "
diff /tmp/pkg{2,3}.lst | awk '/^>/ { print $2 }' | tr "
" " "

Hope it helps.

On 08/27/2011 08:16 PM, please try again wrote:
> Hope it helps.

well, not as i can see…what kind of output do you get with that?

i got a stream of text to the terminal that was (maybe 1000 lines long,
far too fast to count…maybe 2000…took about five or ten minutes…(i
didn’t know i needed to time it)…

and, the list seemed to include the name of everything on the
machine…for sure i saw stuff go by that i didn’t elect to install
after the initial install (like page after page after page of Xorg stuff…

so, somewhere i think you missed something…

so i run it again with an [executable]>>output.txt


DD
openSUSE®, the “German Engineered Automobile” of operating systems!

Well, it was obvious. I don’t say people how they should redirect, so they can add pipes to filter the output. :wink:

All right, if you replace “*” with “#” in the second printf to make it easier to parse, then

[exectutable] | awk '/^*/&&!/packages/ { print $2 }'

will show the patterns to install
and

[exectutable] | sed '1,/* remaining/d'

the remaining packages. Remaining packages means they were either not installed as dependencies or not found anywhere. But I’m sure I missed things - it was quick and dirty.

Btw, I’m doing like you do. I have a list of patterns and packages that get parsed by my installation script.

On 08/27/2011 10:06 PM, please try again wrote:
>[executable] awk ‘/^*/&&!/packages/ { print $2 }’

well, maybe there is a language problem: i thought, after reading this
“How can I easily get a list of all the packages I have installed on my
desktop system since the OS was installed?” it was the OP’s desire to
construct a list of all the packages he had installed since the initial
install, like:

Inkscape
bluefish
Chrome
Google Earth
kate
mc
etc
etc

but instead, the awk quoted above gives this


base
devel_basis
devel_C_C++
enhanced_base
fonts
games
imaging
kde4
kde4_basis
etc
etc

and, i am certain KDE4 was part of the initial install–so, it is not
excluding the default installed bits…

and, the list generated doesn’t include (for example) Google Earth which
i am sure i did install after the initial install…

and, the other ([exectutable] | awk ‘/^*/&&!/packages/ { print $2 }’)
was even less helpful in tracking what i’d have to install after the
initial install to rebuild my system…

so, unless i totally missed what the OP is looking for, your script is
not very helpful…but, thanks for your effort and i freely admit that
perhaps i don’t yet understand what the op is looking for…


DD Software
openSUSE®, the “German Engineered Automobile” of operating systems!

Google earth and some other packages don’t go through RPM so would never show up. Things like installing installing video drivers the hard way Google earth anything you have hand compiled etc bypass the RPM.

That’s right, it doesn’t show the packages installed since the OS installation. It just shows the list of patterns and packages not belonging to any pattern or part of dependencies that you would install on another machine to get the same system - although, as I said, that’s not how I’m doing since I know the list of what I’m going to install in advance. I would say it’s too late to get that list, but you could write a “zypper wrapper” which would keep track of all packages you install manually and install packages with such a script.

After a new installation, you can compare the output of

 [exectutable] | awk '/^*/&&!/packages/ { print $2 }'

with the list of patterns that are already installed and install the missing patterns. Or you check for each pattern if it is installed and if not, install it. It comes to the same but takes significantly longer.

Actually you should rather use this command because some patterns “stupidily” have blanks in their name:

[executable] | awk '/^*/&&!/packages/ { $1="" ; sub (/ /,"",$0) ; print $0 }'

for pat in $([executable] | awk '/^*/&&!/packages/ { $1="" ; sub(/ /,"",$0) ; gsub(/ /,"#",$0) ; print $0 }') ; do 
	zypper info -t pattern "${pat//#/ }" | grep -q -i 'installed: yes' || zypper --non-interactive install --auto-agree-with-licenses -t pattern "${pat//#/ }"
done

Notice the tricks I have to apply because of the blanks! (Those patterns were suggested by MS lol!).

Then you take the output of

[exectutable] | sed '1,/* remaining/d'

and do (more or less) the same for packages.

At the end, you’ll just have to type “[some other executable]” to install the packages you are missing. I guess this was the intention. It’s not the perfect method but “reverse installing” can not be.

  • You’ll first have to add the needed repositories!

Well, that’s why I avoid to compile/install stuff that way and always try to create rpms. When an app I need is missing for openSUSE, I build it in OBS and install it from my home repo. In the rare cases where it doesn’t work, I build it locally with rpmbuid. In the worst (very rare) case where it doesn’t compile, I aliened it from a .deb or .tgz archive. If it doesn’t work after all, I just forget about it. So my list of packages to install is divided in 3 parts: patterns first, rpm from repos and some other locally built rpms.

My Google earth is an rpm btw. So is the latest ATI proprietary driver (installed with atiupgrade)

rpm -qa | grep -e google-earth -e fglrx
google-earth-stable-6.0.3.2197-0.x86_64
fglrx64_xpic_SUSE114-8.881-1.x86_64

I never had the need to compile the nvidia driver (unlike the ATI one), but if I had too, I’ll probably create an rpm. I’m surprised that the nvidia installer doesn’t offer this possibility (atiupgrade actually uses a feature of the ATI installer ; it doesn’t create a rpm on its own).

This one (in red) has been in the top 5 of the list on all my Linuxes and Unixes for … a long time!
But the one below seems to have another meaning on my system. lol!

Here’s my list for 11.4, if it might inspire anyone. I always start with a Gnome desktop (so don’t have to install Gnome). The ones starting with “!” are patterns ; the ones starting with “_” are local rpms (that I don’t install from repo). These characters are stripped by the script that parses this list. 'Lines starting with #" are ignored as usual.


#--------------------------------
!console
!kde4
!xfce
!lxde
!devel_web
!devel_kernel
!devel_python
!devel_gnome
!devel_kde
!devel_ruby
!devel_perl
!devel_ide
!network_admin
#!kde4_ide
#!devel_java
#--------------------------------
#!dhcp_dns_server
#--------------------------------
patterns-openSUSE-kde4_admin
patterns-openSUSE-kde4_edutainment
amor
kalarm
kteatime
ktimetracker
ktux
bovo
katomic
kbattleship
kbounce
kbreakout
kdiamond
kiriki
kjumpingcube
klines
knetwalk
kolf
ktron
kgoldrunner
kollision
konquest
kshisen
ksirk
kspaceduel
ksquares
ktuberling
kubrick
lskat
kjots
juk
kblackbox
kfilereplace
kfourinline
kimagemapeditor
klinkstatus
kmouth
kolourpaint
krecord
ktimer
kturtle
kuiviewer
kuser
kdf
kwikdisk
superkaramba
kppp
kfloppy
kblocks
kmtrace
kdeartwork4-emoticons
kdeartwork4-icons
kdeartwork4-sounds
kdeartwork4-wallpapers
kdebase4-wallpapers
kdesvn
kde4-printer-applet
ruby-kde4
dragonplayer
krename
krusader
speedcrunch
kdirstat
quassel-mono
yakuake
ksystemlog
knode
rekonq
#--------------------------------
bluefish
#--------------------------------
compizconfig-settings-manager
compiz-plugins-extra
fusion-icon
#--------------------------------
nfs-kernel-server
pure-ftpd
#--------------------------------
koffice2-karbon
koffice2-kformula
koffice2-kplato
koffice2-kpresenter
koffice2-krita
koffice2-kspread
koffice2-kthesaurus
koffice2-kword
#--------------------------------
xfce4-panel-plugin-clipman
xfce4-panel-plugin-cpufreq
xfce4-panel-plugin-cpugraph
xfce4-panel-plugin-datetime
xfce4-panel-plugin-diskperf
xfce4-panel-plugin-fsguard
xfce4-panel-plugin-genmon
xfce4-panel-plugin-mailwatch
xfce4-panel-plugin-mount
xfce4-panel-plugin-mpc
xfce4-panel-plugin-netload
xfce4-panel-plugin-notes
xfce4-panel-plugin-places
xfce4-panel-plugin-quicklauncher
xfce4-panel-plugin-radio
xfce4-panel-plugin-screenshooter
xfce4-panel-plugin-sensors
xfce4-panel-plugin-systemload
xfce4-panel-plugin-timeout
xfce4-panel-plugin-timer
xfce4-panel-plugin-verve
xfce4-panel-plugin-wavelan
xfce4-panel-plugin-weather
xfce4-panel-plugin-xfapplet
xfce4-panel-plugin-xkb
xfce4-panel-plugin-xmms
#--------------------------------
xfce4-dev-tools
xfce4-dict
xfce4-icon-theme
xfwm4-themes
audacious
audacious-plugins
leafpad
metacity-themes
gnome-themes-extras
gnome-color-chooser
#--------------------------------
#festival
#festival-hi
#alsa-firmware
#alsa-tools
#alsa-driver-kmp-default 
padevchooser
paman
pavumeter
#--------------------------------
gnome-games
gtk2-themes
gtk2-metatheme-gilouche
pan
rhythmbox
ffmpeg
w32codec-all
pidgin
gftp
dia
ekiga
libxine1-32bit
#--------------------------------
gstreamer-0_10-32bit
gstreamer-0_10-plugins-base-32bit
gstreamer-0_10-ffmpeg
gstreamer-0_10-fluendo-mpegmux
#gstreamer-0_10-plugins-good-extra
#gstreamer-0_10-fluendo-mpegdemux
#--------------------------------
fvwm2
rxvt-unicode
aterm
WindowMaker-applets
WindowMaker
sawfish
asclock
#--------------------------------
acroread
opera
links
lynx
mutt
tin
dog
MozillaThunderbird
#--------------------------------
checkinstall
alien
zenmap
snack
webalizer
tcptraceroute
netcat-openbsd
bridge-utils
iperf
uml-utilities
rdate
xcolors
xless
xfishtank
xpenguins
#--------------------------------
php5-pgsql
php5-gd
php5-curl
php5-devel
php5-ldap
php5-mcrypt
php5-odbc
php5-pspell
php5-xmlrpc
php5-xsl
graphviz-php
php5-bz2
php5-bcmath
php5-calendar
php5-dba
php5-exif
php5-fastcgi
php5-ftp
php5-gettext
php5-gmp
php5-imap
php5-pcntl
php5-posix
php5-readline
php5-shmop
php5-soap
php5-sockets
php5-sysvmsg
php5-sysvsem
php5-sysvshm
php5-wddx
php5-zip
php-doc
phpPgAdmin
php5-tidy
#php5-debuginfo
#php5-debugsource
#--------------------------------
python-mysql
postgresql-server
mysql-query-browser
xmlstarlet
#--------------------------------
yast2-mail-plugins
clamav
pam_ldap
nss_ldap
amavisd-new
#--------------------------------
vlc
gmplayer
#mplayerplug-in
#xmms
#xmms-agent
#xmms-alarm
#xmms-bs2b
#xmms-crossfade
#xmms-eq
#xmms-freeverb3
#xmms-mac
#xmms-mp3cue
#xmms-osd
#xmms-plugins-normalize
#xmms-real-random
#xmms-scrobbler
#xmms-shn
#xmms-tta
#xmms-volnorm
#xmms-wavpack
#xmms-wma
#xmms-xmp
#--------------------------------
ungifsicle
xfig
nasm
gv
fping
par
rar
fetchmail
fetchmailconf
agrep
mmv
bchunk
mcrypt
icmpinfo
rxp
xosview
mtr-gtk
dictd
sshfs
tree
htop
makedev
nano
i2c-tools
sensors
xbill
#J7zip
#unace
#--------------------------------
xaw3d-devel
osc
java-1_6_0-openjdk-devel
#--------------------------------
gqview
dosemu
openmotif
scribus
gparted
audacity
apcupsd-gui
gvim
bitstream-vera
free-ttf-fonts
xv
xpdf-poppler
expect
stellarium
#--------------------------------
VirtualBox-4.0
virt-manager
virt-viewer
qemu
kvm
chromium
python-eyeD3
macchanger
google-earth-stable
#--------------------------------
# needed for Skype
#libqt4-x11-32bit
#libpng12-0-32bit
##libsigc++2-32bit
#--------------------------------
iw
#ralink-firmware
#--------------------------------
agave
#--------------------------------
# PTA packages
#--------------------------------
jmk-x11-fonts
elvis
ctwm
sylpheed
wterm
par2
libpar2-0
eterm
wmfsm
wmcpuload
wmmemmon
wmmemload
wmping
wmCalClock
wmclockmon
wmMoonClock
wmcalc
libdockapp2
wmnetload
lshw-gui
dillo
xplanet
xcolorsel
xsnow
gcolor2
vagalume
gossip
FreeNX
updategrub
atiupgrade
conkyconf
etherape-0.9.12
#--------------------------------
#eclipse
#--------------------------------
# RPM packages installed manually
#--------------------------------
_xpostit
_wmbutton
_xmix
_nxclient
_nxnode
_nxserver
_nxplugin
_gpar2
#_skype
#
#--------------------------------
#video driver
#ATI HD2000 series and later
# x11-video-fglrxG02
#ATI Radeon 9500 - X1900
# x11-video-fglrxG01
#
#NVIDIA GeForce FX 5500
#x11-video-nvidiaG01
#NVIDIA Geforce 7x
#x11-video-nvidiaG02

  • mc is included in the pattern “console” at the top of the list above (and at the bottom of the one below).
# zypper info -t pattern console
Information for pattern console:

Repository: openSUSE-11.4-Oss
Name: console
Version: 11.4-6.9.1
Arch: x86_64
Vendor: openSUSE
Installed: Yes
Summary: Console Tools
Description: 
Applications useful for those using the console and no graphical desktop environment.
Contents:

S | Name                      | Type    | Dependency
--+---------------------------+---------+-----------
  | lftp                      | package |           
i | emacs-nox                 | package |           
i | vorbis-tools              | package |           
i | ivman                     | package |           
  | convert                   | package |           
  | units                     | package |           
i | w3m                       | package |           
  | pico                      | package |           
  | irssi                     | package |           
i | mtools                    | package |           
  | ncftp                     | package |           
i | patterns-openSUSE-console | package |           
  | cryptconfig               | package |           
i | links                     | package |           
  | slrn                      | package |           
  | minicom                   | package |           
  | findutils-locate          | package |           
  | pinfo                     | package |           
  | ding                      | package |           
  | dar                       | package |           
  | bsd-games                 | package |           
  | alpine                    | package |           
i | mutt                      | package |           
  | gcal                      | package |           
i | par                       | package |           
i | makedev                   | package |           
i | wodim                     | package |           
i | sox                       | package |           
  | cnetworkmanager           | package |           
i | nano                      | package |           
  | grepmail                  | package |           
i | mc                        | package |           
  | vlock                     | package |           

This is getting far too complicated!!

I was hoping that, for instance, the rpm database at /var/lib/rpm/Packages, or else one of the yast logs would contain information which allowed me to distinguish between packages that had been manually installed by me using ‘yast2 --install’ or ‘rpm -i’, and those which had been installed either as automatic updates or as dependencies. I can deal with the few additions made through .tgz builds - it’s the rpms I’m interested in.

It sounds from what has been said that this is not possible. DenverD is right in his thoughts about what I’m trying to get at. I haven’t tried the script (my first rule is ‘never run a script supplied by someone else until you understand how it works and what it’s going to do’) as I’ve only just seen it. I will have a closer look and then when I’m happy, I’ll try it out, but from the comments made it doesn’t sound like it’s what I’m after. Thanks for your efforts all the same.
Is there a parser for the /var/lib/rpm/Packages database which will simply display each entry and ALL its associated information? That might help - or are we back to using ‘rpm -qa’?

Thanks.

I am not a shell script expert so will suggest an algorithm for a solution!!

  1. Produce a list of installed packages using ‘rpm -qa --last’
  2. Manually remove those packages from the list, installed at OS installation time, leaving a list of all packages installed post-OS installation
  3. Sort the list alphabetically (may help to search the list in the following code)
  4. For each package in the list, get a list of its dependencies using ‘rpm -q --requires’
    for each package in the dependency list
    if the dependent package appears in the original list, remove it from the original list

What you end up with is a list of installed packages with all the dependencies removed.
This would work for me. Are there any shell script experts who could code this?
Thanks

Would it not violate your first rule?

Ha Ha !!

I would always check it first!! I once imported a script from a forum which was supposed to convert .m4a audio files to .mp3’s retaining all the tag info. It had a good go at doing a conversion (but failed) and then deleted the .m4a files.

I had copied a sample batch of .m4a’s to a test directory so I didn’t lose anything luckily but you can never be too careful!!

Anyhow I went through your previous script and got a good understanding of how it worked so that was useful. Unfortunately it wasn’t what I was after. Maybe I should use my limited shell scripting skills to try to find a solution!!

This would lists all packages installed after OS installation and their dependencies … well not exactly - it depends at what minute of the hour you performed the installation… but it could easily be changed to exclude all packages installed on that day (makes the command a bit longer). Maybe you can do it once you understand this command. Good luck! (as my wife always says, my style is “write only” )

eval `rpm -qa --last | grep -v "$(rpm -qa --last | sed -n '$s/^ ]* *\(^:]*\).*/\1/p')" | awk '{ print "echo -n \"" $1, "-> \" ; rpm -q --requires", $1, "| sed \"/(/d;/^\\\\//d;s/ .*//;/\\\\.so.[0-9]*/d\" | tr \"\\\
\" \" \" ; echo ;"}'` | sort

Be patient (it takes some time) … and I promise it won’t delete your music.

You’re only halfway there with the output of this script … I leave it to you to you figure out the rest or post the problem in the development forum.