script to maintain local copies of resositories

Of course to do this i need to be able to download the relevant
repositories in the first place.

I have some current hangups:

Wine, i have:

http://download.opensuse.org/repositories/Emulators%3A/Wine/openSUSE_11.1/

but i cannot download it.

ATI, for my video card, same issue:
http://www2.ati.com/suse/11.1

Science, (for gEDA)
http://download.opensuse.org/repositories/science/openSUSE_11.1/

Java
http://download.opensuse.org/repositories/Java:/packages/openSUSE_11.1/

The repositories work in Yast but i have not found a way do download
them, which is necessary to maintain them with rsync.

rsync, ftp, uucp, and even http can dl repositories fine the first time
but rsync has huge advantages for keeping them up to date.

JosephKK wrote:
> but i cannot download it.

you can make a local mirror and keep it updated with wget…
have a look at man wget, and info wget…

pay attention to the --mirror switch, and the others to get it just
right…

once you have the correct command line magic you can load it into a cron…


Enjoy Packman’s benefits? Show your appreciation by donating at
http://se.unixheads.org/?donate

DenverD
CAVEAT: http://is.gd/bpoMD [posted via NNTP w/openSUSE 10.3]

On Sun, 11 Jul 2010 22:07:28 GMT, DenverD
<DenverD@no-mx.forums.opensuse.org> wrote:

>JosephKK wrote:
>> but i cannot download it.
>
>you can make a local mirror and keep it updated with wget…
>have a look at man wget, and info wget…
>
>pay attention to the --mirror switch, and the others to get it just
>right…
>
>once you have the correct command line magic you can load it into a cron…

Does wget have the same time saving properties in maintaining copies that
rsync does? Or is it just another bulk download tool?

JosephKK wrote:
> Does wget have the same time saving properties in maintaining copies that
> rsync does? Or is it just another bulk download tool?

you began your Q by asking to download first so you could rsync later…

first, i am not aware if the repos you target have an active rsync
daemon (which is probably why you can’t use rsync for your ‘initial’
download)

for the task you have outlined i would use wget, others might not…it
is a very powerful way to download and maintain current a local copy
of anything it can touch…(including things available via ftp or http
on servers not running rsyncd

see:
man wget
info wget
man rsync
info rsync

be sure and look at the --mirror switch for wget (second and last
hint, but if you wanna use rsync, do that)


Enjoy Packman’s benefits? Show your appreciation by donating at
http://se.unixheads.org/?donate

DenverD
CAVEAT: http://is.gd/bpoMD [posted via NNTP w/openSUSE 10.3]

On 2010-07-14 07:16 GMT DenverD wrote:

> first, i am not aware if the repos you target have an active rsync
> daemon (which is probably why you can’t use rsync for your ‘initial’
> download)

I think that there is a list of download mirrors allowing rsync.
Something with rsync in the opensuse.download link.

>
> for the task you have outlined i would use wget, others might not…it
> is a very powerful way to download and maintain current a local copy
> of anything it can touch…(including things available via ftp or http
> on servers not running rsyncd

I did try wget with --mirror, but what I got was several directories
with the actual names of the mirrors it used; the download name is a
redirector; wget put the names it god redirected to instead of merging
the directories. Not something anyone would want. In the end, I used
fileezilla. I was in a hurry, couldn’t take hours to study man and
test. :slight_smile:


Cheers / Saludos,

Carlos E. R.
(from 11.2 x86_64 “Emerald” GM (Elessar))

Carlos E. R. wrote:
> I did try wget with --mirror, but what I got was several directories
> with the actual names of the mirrors it used; the download name is a
> redirector; wget put the names it god redirected to instead of merging
> the directories. Not something anyone would want. In the end, I used
> fileezilla. I was in a hurry, couldn’t take hours to study man and
> test. :slight_smile:

i didn’t say to use --miror only (and, i wasn’t gonna do the hours
of man study for him/her)…

but, i hadn’t thought about the redirector sending wget to one place
one time and another place the next…and then having (maybe) several
differently named sub-directories of the SAME packages…

hmmmm…maybe best to go around the redirector and pick a specific,
local, dependable (quickly updated and maintained) fast mirror…

fileezilla i don’t know, but i guess if it has a GUI it will be a hit
with the Ship Jumpers…


Enjoy Packman’s benefits? Show your appreciation by donating at
http://se.unixheads.org/?donate

DenverD
CAVEAT: http://is.gd/bpoMD [posted via NNTP w/openSUSE 10.3]

On Sat, 17 Jul 2010 10:25:37 GMT, “Carlos E. R.”
<robin.listas@localhost> wrote:

>On 2010-07-14 07:16 GMT DenverD wrote:
>
>> first, i am not aware if the repos you target have an active rsync
>> daemon (which is probably why you can’t use rsync for your ‘initial’
>> download)
>
>I think that there is a list of download mirrors allowing rsync.
>Something with rsync in the opensuse.download link.
>
>>
>> for the task you have outlined i would use wget, others might not…it
>> is a very powerful way to download and maintain current a local copy
>> of anything it can touch…(including things available via ftp or http
>> on servers not running rsyncd
>
>I did try wget with --mirror, but what I got was several directories
>with the actual names of the mirrors it used; the download name is a
>redirector; wget put the names it god redirected to instead of merging
>the directories. Not something anyone would want. In the end, I used
>fileezilla. I was in a hurry, couldn’t take hours to study man and
>test. :slight_smile:

I experienced similar results. 7.7 GiB for directories and indexes
strikes me a more than a bit spurious. The documentation on the
–mirror option is less clear than i could wish for.

On Sat, 17 Jul 2010 10:42:48 GMT, DenverD
<DenverD@no-mx.forums.opensuse.org> wrote:

>Carlos E. R. wrote:
>> I did try wget with --mirror, but what I got was several directories
>> with the actual names of the mirrors it used; the download name is a
>> redirector; wget put the names it god redirected to instead of merging
>> the directories. Not something anyone would want. In the end, I used
>> fileezilla. I was in a hurry, couldn’t take hours to study man and
>> test. :slight_smile:
>
>i didn’t say to use --miror only (and, i wasn’t gonna do the hours
>of man study for him/her)…
>
>but, i hadn’t thought about the redirector sending wget to one place
>one time and another place the next…and then having (maybe) several
>differently named sub-directories of the SAME packages…
>
>hmmmm…maybe best to go around the redirector and pick a specific,
>local, dependable (quickly updated and maintained) fast mirror…
>
>fileezilla i don’t know, but i guess if it has a GUI it will be a hit
>with the Ship Jumpers…

The disk i had all my repositories on went toes up. Controller board
died as far as i can tell. Wasn’t even 1 year since date of
manufacture. There may be something about grub/boot that makes TB
class disk drives breakable. I adjusted by making my volumes readily
user mountable after boot and not mounting at boot.

Any way i have to do it all over again. At least i have a few
succesful rsync commands to use.

On Sat, 24 Jul 2010 23:44:58 +0000, JosephKK wrote:

> The disk i had all my repositories on went toes up. Controller board
> died as far as i can tell. Wasn’t even 1 year since date of
> manufacture. There may be something about grub/boot that makes TB class
> disk drives breakable. I adjusted by making my volumes readily user
> mountable after boot and not mounting at boot.

I doubt it. I think instead that the TB class is still too new and
unreliable.

Maybe if you buy enterprise class :-?


Cheers / Saludos,

Carlos E. R.
(from 11.2 x86_64 “Emerald” GM (Elessar))

On Sun, 25 Jul 2010 01:01:47 GMT, “Carlos E. R.”
<robin.listas@nopost.localhost> wrote:

>On Sat, 24 Jul 2010 23:44:58 +0000, JosephKK wrote:
>
>
>> The disk i had all my repositories on went toes up. Controller board
>> died as far as i can tell. Wasn’t even 1 year since date of
>> manufacture. There may be something about grub/boot that makes TB class
>> disk drives breakable. I adjusted by making my volumes readily user
>> mountable after boot and not mounting at boot.
>
>I doubt it. I think instead that the TB class is still too new and
>unreliable.
>
>Maybe if you buy enterprise class :-?

Where did you find enterprise class TB drives?

JosephKK wrote:

> On Sun, 25 Jul 2010 01:01:47 GMT, “Carlos E. R.”
> <robin.listas@nopost.localhost> wrote:
>
>>On Sat, 24 Jul 2010 23:44:58 +0000, JosephKK wrote:
>>
>>
>>> The disk i had all my repositories on went toes up. Controller
>>> board
>>> died as far as i can tell. Wasn’t even 1 year since date of
>>> manufacture. There may be something about grub/boot that makes TB
>>> class
>>> disk drives breakable. I adjusted by making my volumes readily user
>>> mountable after boot and not mounting at boot.
>>
>>I doubt it. I think instead that the TB class is still too new and
>>unreliable.
>>
>>Maybe if you buy enterprise class :-?
>
> Where did you find enterprise class TB drives?

Easy -
Seagate does SAS drives up to 2Tb - HP does SAS to 1TB, SATA to 2Tb.


Per Jessen, Zürich (15.4°C)
http://en.opensuse.org/User:pjessen