Need volunteer verification change to "SDB System Upgrade" does not work.

English Install/Boot/Login
#1

A day or so ago a major change was contributed to one of our most import SDB articles which is “The” Authority for doing an online upgrade

https://en.opensuse.org/SDB:System_upgrade

Testing the change, I have not been successful in getting the change to work, so am asking anyone with a little time to test this change in a virtual machine or install they don’t mind risking (although to date I haven’t yet destroyed a machine).

The Big Change…
Replacing the tried and true sed statement that converts all existing repo URLs and labels,
A new $releasever zypper option is used to supposedly write/change the version.
Documentation is sparse, says that this can be used as part of the upgrading process.
Bottom line, the following is supposed to replace the sed statement and change all release version numbers in the repo configuration files to 15.1 no matter what the current openSUSE version is

zypper --releasever 15.1 --gpg-auto-import-keys ref

But,
When I ran it in a 15.1 trying to change to 15.2 and when that failed to 15.0, in both cases the command failed and displayed the zypper help.
When I ran it in a 42.3 trying to upgrade to 15.1, the command ran successfully but when I did a “zypper lr -d” nothing had changed and when I checked the repo.d files directly, they also had not changed. Thinking that despite what was described in the MAN pages perhaps the modified strings were only in memory and not written to disk, I went ahead and tried a “zypper dup” but that resulted in the 42.3 suggesting it would re-install 42.3.

So, wondering what others might experience before I go ahead and revert the change back to how the SDB System Upgrade has worked for the past 6 or so openSUSE versions…

As always,
The first 3/4 of the SDB System Upgrade page can be skipped, those parts apply to people who are upgrading from 13.1 or earlier.
Start about 3/4 down the page with “Running the Upgrade” in big, bold headline type.

TIA,
TSU

#2

I’m cloning a virtual machine with 15.0 right now. I’ll test when that is done.

I don’t want to try going to 15.2, because at present “zypper ref” on 15.2 gives errors on the update repos, which might interfere with the proper operation.

#3

The command seemed to run okay. This was on a Leap 15.0 system.


# zypper --releasever 15.1 --gpg-auto-import-keys ref
Warning: Enforced setting: $releasever=15.1
Retrieving repository 'packman' metadata .................................[done]
Building repository 'packman' cache ......................................[done]
Repository 'openSUSE-Leap-15.0-Non-Oss' is up to date.                          
Repository 'openSUSE-Leap-15.0-Oss' is up to date.                              
Retrieving repository 'openSUSE-Leap-15.0-Update' metadata ...............[done]
Building repository 'openSUSE-Leap-15.0-Update' cache ....................[done]
Repository 'openSUSE-Leap-15.0-Update-Non-Oss' is up to date.                   
All repositories have been refreshed.

It looks as if it has only refreshed the 15.0 repos, and ignored the “–releasever” part. But it is only listing repo names there, not urls.

When I check “/etc/zypp/repos.d”, none of the repos have changed.

When I look in “/var/cache/zypp/raw/repo-update/repodata/”, I see updated files (metadata). But those files seem to be references to Leap 15.0 packages (the string “-lp150.” is part of the package versions).

#4

Thx for your confirmation what I’m seeing, I hope we both aren’t somehow doing something incorrect but it looks pretty simple so it either works or it doesn’t.

If anyone else can verify, it’d be useful…
I’m reluctant to make such a drastic change in documentation as to undo a contribution completely unless I’m pretty darn sure what I’m doing… and I’m trying to avoid some kind of personal oversight/error.

And,
Some might find it interesting what this is <supposed> to do,
If it’s currently faulty then I’m sure it’ll be fixed and this might come in handy to know in the future.

TSU

#5

Perhaps a bug report would get a response from the “zypper” maintainers.

#6

I’ll file that, too.

BTW -
This incident made me re-read zypper help and MAN pages, and I don’t know when they were added but I see plenty of new options I don’t remember seeing before. One in particular is pattern-info which although isn’t exactly what I want does provide a partial (why not complete?) list of packages in the pattern.

TSU

#7

I decided to go ahead and

When the zypper bug has been fixed,
The SDB can be reviewed and the proposed replacement can be considered.

TSU

#8

Do your repository URLs include $releasever variable instead of actual version number?

#9

In my installation of Leap 15.0 they do not.

So the change in the documentation in
https://en.opensuse.org/SDB:System_upgrade
that TSU was looking after, may have been premature.

The second comment on bugzilla.opensuse.org, and the change of status there to RESOLVED INVALID, came after TSU reported, as far as I can see.

#10

Or is the output of

zypper lr -d

affected by the value of

$releasever
#11

When I look into the files in /etc/zypp/repos.d in Leap 15.0 I find

15.0

instead of

$releasever
#12

I suppose it depends on what is in the code,
But if the “code behind” zypper --releasever writes the variable value, then it shouldn’t be necessary for the variable to be embedded in the file.
I’d also guess that not fixing the value and embedding a variable in the repo definition file might be dangerous since it would direct to an unintended (or maliciously intended?) location very easily.
An example of extracting a version number and using it to write a repo URL is in the “zypper” section of my personal Wiki… making use of the standard “add repo” (can possibly be changed to modify repo) command
https://en.opensuse.org/User:Tsu2/BASH_zypper

Additionally,
It should be noted that I remarked in my bug report that it can be important to set the new value in more than just the URL… If the name/label of the repo contains the version, it should be modified, too… Something the sed command does but I’m guessing would not with the “–releasever” method.

TSU

#13

Additionally,
I don’t know if $releasever even should exist by default on my openSUSE
The following returns null (empty space)

echo $releasever

And, although is not necessarily any kind of evidence, I wonder how $releasever is even supposed to be created and set by the system… certainly not by any related file, it can’t be found using the “locate” utility. I’m just starting to suspect it may be something created on the fly and only within a zypper controlled namespace.

TSU

#14

I just ran a test replacing the “15.1” in my system’s Update repo with “$releasever” as follows and it throws and error “repository is invalid”

  1. View my existing 15.1 update repo details (is repo 3 on my system), the following is edited to remove irrelevant info
zypper lr -d 3Alias          : http-download.opensuse.org-49125803                       
Name           : openSUSE:Leap:15.1:Update                                 
URI            : http://download.opensuse.org/update/leap/15.1/oss/        
Enabled        : Yes                                                                                                           
Repo Info Path : /etc/zypp/repos.d/http-download.opensuse.org-49125803.repo
MD Cache Path  : /var/cache/zypp/raw/http-download.opensuse.org-49125803
  1. Create a new repo based on the above, but replace “15.1” with “$releasever”
zypper ar -f http://download.opensuse/leap/$releasever/oss/ openSUSE:Leap:15.1:Update_releasever
  1. Disable original Update Repo, leaving only the modified enabled
zypper mr --disable 3
  1. Test, setting $releasever to 15.1, returns an invalid URL
zypper --releasever 15.1 --gpg-auto-import-keys ref
  1. And indeed when I open the repo file I see that nothing has been inserted into the URL so the path is incorrect.
    I then edited the file manually to insert $releasever so that the URL is as follows

baseurl=http://download.opensuse/leap/$releasever/oss/

But, unfortunately the URL is still invalid.
So, is broken no matter what the bugzilla says.
I will recommend the bug be re-opened

TSU

#15

Finally got this to work, but it seems that you can’t add the repo using the standard “zypper addrepo” command, you’ll have to edit the repo file directly by hand or create the file some other way.

My test procedure:

  1. View my existing 15.1 update repo details (is repo 3 on my system), the following is edited to remove irrelevant info
zypper lr -d 3Alias          : http-download.opensuse.org-49125803                       
Name           : openSUSE:Leap:15.1:Update                                 
URI            : http://download.opensuse.org/update/leap/15.1/oss/        
Enabled        : Yes                                                                                                           
Repo Info Path : /etc/zypp/repos.d/http-download.opensuse.org-49125803.repo
MD Cache Path  : /var/cache/zypp/raw/http-download.opensuse.org-49125803
  1. Create a new repo based on the above, but replace “15.1” with “$releasever”
zypper ar -f http://download.opensuse.org/update/leap/$releasever/oss/ openSUSE:Leap:15.1:Update_releasever
  1. Disable original Update Repo, leaving only the modified enabled
zypper mr --disable 3
  1. Test, setting $releasever to 15.1, but it returns an invalid URL
zypper --releasever 15.1 --gpg-auto-import-keys ref
  1. And indeed when I open the repo file I see that $releasever has not been inserted into the URL so the path is incorrect.
    I then edited the file manually to insert $releasever so that the URL is as follows
baseurl=http://download.opensuse.org/update/leap/$releasever/oss/

Finally, re-running the step 4 command works for 15.1 but for some reason does not work for any other version number(tested 15.0, 42.3).
Am looking at that issue, but at least it works for 15.1.
And, tested the $releasever variable in the repo name… Doesn’t work. “$releasever” is read literally and not evaluated as a variable in the repo name.

If openSUSE moves forward distributing the modified repo files to support $releasever, I wonder if it’s for the better, the sed command works better because it will modify repos names as well as the repo URL to avoid confusion which repos belong to which version of openSUSE. And, of course I’m still unsure at the moment why only 15.1 works in my current test.

TSU

#16

A further update…
I re-created the repo using $releasever using the “zypper addrepo” command again…

And, it works.
Can’t figure out why it was throwing an error earlier and now isn’t.
Inspected the repo file and as before, the $releasever variable is non-existent so the URL should fail.
But it doesn’t.
And, setting $releasever to 15.0 and 42.3 both work, too.

This is too weird and defies logic…

TSU

#17

In Comment 1 of
https://bugzilla.opensuse.org/show_bug.cgi?id=1149511
it is described that

If you want to make use of releasever all URLs in your repositories need to be in the form of ( replace the release nr with the variable ) :

https://download.opensuse.org/distribution/leap/$releasever/repo/oss/

#18

I saw that, but
If i were to not just blindly take that statement literally,

I can’t think it’s likely that any system would fail everything if every repo wasn’t configured to use $releasever
The reasons for following that statement would be

  • You want to use a consistent method of naming for all repos, else you’ll end up with a confusing mess
  • When you use a variable like $releasever which is intended to make mass changes, you don’t want to later find that it works only in some cases and not others you’ve forgotten about.

In other words,
If you don’t make everything work one way or another the same way, be warned! You’ll have a weird mess that at worse could install packages intended for another system!

In a small impromptu test though,
I’d be very surprised if a single repo can’t be used for testing by itself… I have to just be careful to leave the system in a consistent state that won’t cause problems in the future.

TSU

#19

OK,
For the brave souls out there,
Here is a script that upgrades 42.3 to 15.1
and
converts the repo files to use $releasever in the repo config files.

Until a number of people test this script and verify no problems, I recommend people upgrade only virtual machines running 42.3, and take a snapshot before upgrading.

Follow other preparatory steps described in the SDB:System Upgrade like disabling unnecessary repos, backing up repo configuration files if desired and ensuring you have enough free space (I recommend for most people >2GB)

It should be noted that the script is not written to be unattended, it’s intended only to ensure that once started each step required for the upgrade is done properly but will still require User input at various points.

Also,
Be aware that since this is not an official method created by Maintainers, a future “zypper dup” may be required to incorporate whatever they do, but from what I’ve seen I’d find it hard to believe they might do something very different than what this script does.

INSTRUCTIONS:
Prep your 42.3 machine as described in the SDB:System Upgrade “Running the Upgrade” - Skip over, and ignore everything in the SDB before this starting point.
Continue until after you optionally create backup copies of your repo configuration files, execute the below script.
That’s all!

See the comments in the file to understand what it does.


#!/bin/bash
#
# This script supports only the following scenario
# Upgrade LEAP 42.3 to 15.1
#
# This script incorporates the currently relatively untested method of modifying
# the repo URLs to use $releasever to designate the target version
#
# This script fixes as needed for this scenario, beware that several of these steps
# would not apply to other scenarios, for instance a beginning version 15.0
#
# Modify repo config files to use $releasever instead of fixed numbers
sed -i 's/42.3/$releasever/g' /etc/zypp/repos.d/*
# The yast2 repository type has been deprecated, the target will be rpm-md
sed -i 's/yast2/rpm-md/g' /etc/zypp/repos.d/*
#
# The following command executes the upgrade.
# The original recommended command to simply refresh is insufficient since it 
# appears that setting $releasever to a non-default value lasts only for the command and is not persistent 
zypper --releasever 15.1 --gpg-auto-import-keys dup
sleep 2
# Reboot
systemctl reboot

Some additional caveats…
Some additional “features” were discovered during my hacking…

  • The $releaseserv variable does work in the repo “name” field, but does not in the “Alias” field
  • Typically the “Big 4 repos” that are generally minimally necessary should be the only repos enabled during an upgrade, and by default both their names and alias don’t specify a version number…
    but…
  • Other repos can have an openSUSE version number. I myself specify the openSUSE version number as my SOP to remind myself what that repo is supposed to support. If any of your repos have an openSUSE version number, the name will be fixed along with the URL by the sed command in my script but be beware that this may be the last time you may run the sed command, the new method to use $releasever is likely to deprecate the sed command.

Note that by using the sed command to edit the repo configuration files, the problem with strange and broken strings when zypper commands are used to make the same modifications is avoided.

If people find this script is helpful,
I can prepare a similar script for a 15.0 > 15.1 upgrade which is less demanding than 42.3 > 15.1.

Have fun!
TSU