Page 2 of 2 FirstFirst 12
Results 11 to 13 of 13

Thread: Parallel download of rpm packages

  1. #11
    Join Date
    Jan 2014
    Location
    Erlangen
    Posts
    2,299
    Blog Entries
    1

    Default Re: Parallel download of rpm packages

    Quote Originally Posted by Knurpht View Post
    Another thing to consider: If all packages are cached, it can still happen that new ones are added and repos need refreshing. What i've ran into, is that cached packages did not meet the checksums provided from the repo refresh, and would generate errors.

    That said, I can see the idea of paralel downloading to the max bandwidth.

    And that said, I'm in NL with a very stable 300/30 Mbit internet connection and apparently a couple of good mirrors.
    Well, no need for mirrors, as openSUSE resides round the corner:

    Code:
    erlangen:~ # ping -c 1 pontifex.opensuse.org
    PING pontifex.opensuse.org(pontifex.opensuse.org (2620:113:80c0:8::13)) 56 data bytes
    64 bytes from pontifex.opensuse.org (2620:113:80c0:8::13): icmp_seq=1 ttl=59 time=16.0 ms
    
    --- pontifex.opensuse.org ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 16.002/16.002/16.002/0.000 ms
    erlangen:~ #
    I am paying for 25/5 Mbit only, but updating works like a charm. Nice to hear mirroring works well 20,000 km away.
    AMD Athlon 4850e (2009), openSUSE 13.1, KDE 4, Intel i3-4130 (2014), i7-6700K (2016), i5-8250U (2018), AMD Ryzen 5 3400G (2020), openSUSE Tumbleweed, KDE Plasma 5

  2. #12
    Join Date
    Mar 2020
    Location
    São Leopoldo, RS, Brazil
    Posts
    281

    Default Re: Parallel download of rpm packages

    Quote Originally Posted by Knurpht View Post
    Another thing to consider: If all packages are cached, it can still happen that new ones are added and repos need refreshing. What i've ran into, is that cached packages did not meet the checksums provided from the repo refresh, and would generate errors.
    Good catch! I have disabled auto refreshing, but I should either find a way to download new packages (maybe parsing zypper dup output or somehow calling libzypp directly), or follow with a zypper dup giving up on parallel download for those.

    Quote Originally Posted by karlmistelberger View Post
    Well, no need for mirrors, as openSUSE resides round the corner:

    Code:
    erlangen:~ # ping -c 1 pontifex.opensuse.org
    PING pontifex.opensuse.org(pontifex.opensuse.org (2620:113:80c0:8::13)) 56 data bytes
    64 bytes from pontifex.opensuse.org (2620:113:80c0:8::13): icmp_seq=1 ttl=59 time=16.0 ms
    
    --- pontifex.opensuse.org ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 16.002/16.002/16.002/0.000 ms
    erlangen:~ #
    I am paying for 25/5 Mbit only, but updating works like a charm. Nice to hear mirroring works well 20,000 km away.
    I can see why you're not worried about that

    Consider the server closest to me (756km):

    Code:
    PING opensuse.c3sl.ufpr.br(opensuse.c3sl.ufpr.br (2801:82:80ff:8000::b)) 56 data bytes
    64 bytes from opensuse.c3sl.ufpr.br (2801:82:80ff:8000::b): icmp_seq=1 ttl=43 time=579 ms
    
    --- opensuse.c3sl.ufpr.br ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 578.850/578.850/578.850/0.000 ms
    Repeat a thousand times... Serial throughtput is 300-1000 kB/s. There's a faster server (1228 km) but unfortunately it only serves Leap packages. I reached out to them but the email returned. Maybe I'll switch to a farther away server.
    openSUSE Tumbleweed

  3. #13
    Join Date
    Mar 2020
    Location
    São Leopoldo, RS, Brazil
    Posts
    281

    Default Re: Parallel download of rpm packages

    So I looked into ftp (no quality servers nearby), curl --parallel (unreliable), curl url-list... (okay), pick local mirror directly (better), and then I settled with aria2. aria2 can take advantage of global mirror infrastructure to split resources into parts. From 20m down to 3:30m. I think I can stop squeezing the last bit of speed by now.

    Thanks everyone for your input!
    openSUSE Tumbleweed

Page 2 of 2 FirstFirst 12

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •