Over the last month or so, I’ve noticed a relatively high frequency of times when a “zypper ref” from my local Tumblweed mirror fails due to a message along the lines of
File './repodata/183928edbce08b88415fe1be2705f4a3cbd9484c3b465b6d86985f94ff4f871f-primary.xml.gz' not found on medium 'http://mymirrorhost/opensuse/tumbleweed/repo/oss'
I look, and sure enough, that file is mentioned in /srv/susemirror/opensuse/tumbleweed/repo/oss/repodata/repomd.xml, and doesn’t actually exist. This renders updates non-functional, and makes reinstalls a pain. As I intentionally reinstall on a couple of dev machines pretty regularly (right now I’m testing automated deployments), this is inconvenient.
I manage the mirror by running this daily, which doesn’t seem wrong:
Often, when the rsync repo is in this state, the main download.opensuse.org site is fine. Which would be a fine substitute, but my bandwidth is horrible - which is why I have a local mirror to begin with.
Am I doing something incorrectly / unexpected? If not, is there someone I can poke or otherwise help to make this happen less?
I’m not running a public mirror, so almost none of that applies. But the relevant parts - “rsync some stuff and point a web server at the directory” - are equivalent, yes. I added the cache-control headers, since that’s a good idea, but the issue appears that the, umm, “metadata metadata” is not consistent with the repo metadata it’s describing.
I’m not running a public mirror, so almost none of that applies. But
the relevant parts - “rsync some stuff and point a web server at the
directory” - are equivalent, yes. I added the cache-control headers,
since that’s a good idea, but the issue appears that the, umm, “metadata
metadata” is not consistent with the repo metadata it’s describing.
Hi
From memory (Factory Mailing List) it can take some time to create the
metadata and for it to filter though to the mirrors. I would suggest
heading there to ask (or search the archive)…
–
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
SLES 15 | GNOME Shell 3.26.2 | 4.12.14-25.28-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!
I would think that rsync.opensuse.org would essentially always be in-sync; not spend periods of >24 hours in an unusable state. But that answer is what I was looking for. I guess I’ll sign up for another mailing list. Thanks.
Off topic, but re: your signature, Malcom, I was sad to recently actually look at my cron logs (people look at logs?) and learn of the Linux Counter Project’s demise. #108731 here. :disapointed:
On Tue 12 Feb 2019 08:46:03 PM CST, dannysauer wrote:
Off topic, but re: your signature, Malcom, I was sad to recently
actually look at my cron logs (people look at logs?) and learn of the
Linux Counter Project’s demise. #108731 here. :disapointed:
Hi
Oh…didn’t realize that, was only in December last year it would seem,
guess I need to update my signature…
–
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
SLES 15 | GNOME Shell 3.26.2 | 4.12.14-25.28-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!
Yes, but, as you indicate, the synchronisation was falling over and therefore the rsync bits and pieces are the needed change to “what’s been working for years” …
And, as discussed below, in general, the mirroring process can take a considerable amount of time – meaning “days” rather than “hours” …
Agreed, specially after the complete rebuild of all packages in the TW repos. So, rsync is still busy on job X, where in the meantime the source gets updated by the next snapshot.
In your opening post, you mention that, you have a TelCo bandwidth issue …
The rsync module “opensuse-full-with-factory” (needed for Tumbleweed) implies a high frequency of updates – due to Tumbleweed …
Just a thought: is it possible that, reducing the frequency – currently daily – of the mirror synchronisation will possibly reduce, or even eliminate, the metadata issues you’re experiencing?
Good thought, but no. I’m on a 20Mbps connection; it’s not a dial-up - it’s just not as fast as a gigabit connection on my internal network.
My mirror system is in sync with the rsync server; that’s all working just fine. It does get updated package files daily; takes about 10-14 minutes to verify sync if there aren’t any changes. The issue is that the actual upstream server I’m rsyncing from has a repomod file which references metadata files which are not present, and which do not become present for a matter of several days. Right now, the repomod.xml file on rsync.o.o references other files which have been in place on the download.o.o server (or whatever mirror it redirects to) for over eight days. I didn’t want to come right out and blame infrastructure up front, but that’s really only explained by a config error somewhere in either the rsync server or what pushes to it.
Thanks.