Results 1 to 9 of 9

Thread: curl transfer issue

  1. #1

    Exclamation curl transfer issue

    Hi All,
    I'm trying to upload a file with 350Mb using curl but it can only transfer ~40% of the total size.
    OpenSUSE 12.2, no firewall or such enabled

    # curl -k -T image.tar -u root:ass sftp://192.168.5.213:/var/
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed
    41 351M 0 0 41 146M 0 6580k 0:00:54 0:00:22 0:00:32 2855k

    curl stops at 41% and upload speed decrease to 0k, the connection keeps stablished

    # curl --version
    curl 7.25.0 (i586-suse-linux-gnu) libcurl/7.25.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.25 libssh2/1.4.0
    Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
    Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

    # uname -a
    Linux linux-yyug 3.4.6-2.10.1-pae #1 SMP Thu Jul 26 06:36:26 BRT 2012 (641c197) i686 i686 i386 GNU/Linux


    scp copy works

    # scp image.tar 192.168.5.213:/var/
    The authenticity of host '192.168.5.213 (192.168.5.213)' can't be established.
    RSA key fingerprint is xxx.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added '192.168.5.213' (RSA) to the list of known hosts.
    Password:
    image.tar 100% 351MB 22.0MB/s 00:16


    running the curl command with strace, it shows poll timeout

    recv(4, 0x9de5e58, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily unavailable)
    send(4, "\3707\3679x/|\r\203\352\207\306\344u\1\346C^9\264k\to\210\242TeWg\227\325\206"..., 16452, M SG_NOSIGNAL) = 16452
    recv(4, 0x9de5e58, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily unavailable)
    clock_gettime(CLOCK_MONOTONIC, {4318, 499701085}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4318, 499719386}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4318, 499738099}) = 0
    poll([{fd=4, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 1000) = 1 ([{fd=4, revents=POLLIN|POL LRDNORM}])
    poll([{fd=4, events=POLLOUT|POLLWRNORM}], 1, 0) = 1 ([{fd=4, revents=POLLOUT|POLLWRNORM}])
    time(NULL) = 1353937011
    recv(4, "\306\v6\"\276\3470;\35\5\2620\257\204\251\214jl\330\321F~\27_\356\262\343\5\367(\211\v"..., 16384, MSG_NOSIGNAL) = 68
    recv(4, 0x9de5e58, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily unavailable)
    recv(4, 0x9de5e58, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily unavailable)
    recv(4, 0x9de5e58, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily unavailable)
    clock_gettime(CLOCK_MONOTONIC, {4318, 501271622}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4318, 501289383}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4318, 501308415}) = 0
    poll([{fd=4, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 1000) = 0 (Timeout)
    poll([{fd=4, events=POLLOUT|POLLWRNORM}], 1, 0) = 1 ([{fd=4, revents=POLLOUT|POLLWRNORM}])
    time(NULL) = 1353937012
    recv(4, 0x9de5e58, 16384, MSG_NOSIGNAL) = -1 EAGAIN (Resource temporarily unavailable)
    clock_gettime(CLOCK_MONOTONIC, {4319, 503479166}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4319, 503515745}) = 0
    ) = 1
    write(2, " ", 1 ) = 1
    write(2, "4", 14) = 1
    write(2, "1", 11) = 1
    write(2, " ", 1 ) = 1
    write(2, " ", 1 ) = 1
    ...

  2. #2
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    25,158

    Default Re: curl transfer issue

    Hello and welcome here.

    First a bit outside your problem (but inside the forums way of posting).
    You post several pices of copied/pasted computer text. That is fine, because computer facts often say more then long stories. But please, next time, post such pieces between CODE tags. For that you can use the # button in the toobar of the pst composer.

    Now to your problem.

    It is strange that it stops at about the same precentage (not exactly?).

    Did you try another file (of about same size) from the same and/or another source to see if it is definitely connected to curl, or to the combunation of curl with that file (and source system) only?
    Henk van Velden

  3. #3

    Default Re: curl transfer issue

    Hi,
    Yep, sorry for the embedded text.

    Note that curl only transfers 146Mb, no matter the file size.
    If the file size is less than 146Mb, it works.
    Also note I'm now using localhost.

    ~300Mb file
    Code:
    # dd if=/dev/zero of=file bs=1024 count=300000
    300000+0 records in
    300000+0 records out
    307200000 bytes (307 MB) copied
    
    # curl -k -T /var/file -u root:pass sftp://localhost:/tmp/
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
     49  292M    0     0   49  146M      0  12.1M  0:00:24  0:00:12  0:00:12
    ~160Mb file
    Code:
    dd if=/dev/zero of=file bs=1024 count=160000
    160000+0 records in
    160000+0 records out
    163840000 bytes (164 MB) copied
    
    # curl -k -T /var/file -u root:pass sftp://localhost:/tmp/
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
     93  156M    0     0   93  146M      0  9196k  0:00:17  0:00:16  0:00:01
    Then I tried on OpenSUSE 11.3, and it works with any size
    Code:
    # dd if=/dev/zero of=file bs=1024 count=300000
    300000+0 records in
    300000+0 records out
    307200000 bytes (307 MB) copied
    
    # curl -k -T file -u root:pass sftp://localhost
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  292M    0     0  100  292M      0  16.1M  0:00:18  0:00:18 --:--:-- 16.1M
    
    # curl --version
    curl 7.20.1 (i686-pc-linux-gnu) libcurl/7.20.1 OpenSSL/1.0.0 zlib/1.2.3 libidn/1.15 libssh2/1.2.2_DEV
    Protocols: dict file ftp ftps http https imap imaps ldap pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
    Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz
    So, can you guys try:
    Code:
    dd if=/dev/zero of=file bs=1024 count=160000
    curl -k -T file -u root:pass sftp://localhost

  4. #4
    ab NNTP User

    Default Re: curl transfer issue

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Duped. At the same time, sftp worked perfectly for the transfer. Weird....

    openSUSE 12.2 x86_64

    curl 7.25.0 (x86_64-suse-linux-gnu) libcurl/7.25.0 OpenSSL/1.0.1c
    zlib/1.2.7 libidn/1.25 libssh2/1.4.0
    Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps
    pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
    Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL
    libz TLS-SRP

    Good luck.
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.19 (GNU/Linux)
    Comment: Using GnuPG with undefined - http://www.enigmail.net/

    iQIcBAEBAgAGBQJQs6nUAAoJEF+XTK08PnB5UZoQALUBPqPPmBveSTb1zW7kmq3W
    LA/VEA9KrXw2zaaIJEHpw+vRX9g9v4G/7n/6LecgfBYb8qdLEAJYbf+1+QRncc3B
    OXLrKwCt8lzzf7ly1kWn05V/Z8VhSEfhnrqkU20ZAngdXolQGCG8cb7KZg/pgiNB
    hVgfu6V1duYyGVByXQBQ2fncBOO0jvLBjf2MswWJ8g2tcR6kj6FPo5hDFCmmeHZA
    X9Rr6wMkoIjqxZVvjwQhS2E6nXiq5g9RqilnHsoLfjwOfgwZkOXwLWr1gAeuhtwE
    Xee1ljWP2NSt+xs0HBIQAHQOFgTS/j7qhgt0LJGBvL7Tpbi5tDi3feoL2/Ko8pTJ
    UgStZNj2pXSHCIvV0lgoYkciTU1X4yPE8AR1pJWGF7Pzd4EteaGwdrmfAxNDCQHf
    d8WyLuTRCE/GCJWZbHjDHuDI5hYQ4tA+ay//WTQFpBz1Rh9sEdp5hmCmswAvx0x+
    YaS67lsLuKC9e1Not5eRZ2E1O7F4IEQBZiNC4E5W/+qQ7lzDp+KXgtoMhdY+dJzK
    GKDeCloTV9/Y4gNlLeBBs080yNyVP3vFMu0MuXTx+UdLBNy1fflYdcU2WvzScDIG
    zaLIDvJSvxLclTWTTW0tTHNZrZU+a7B2qptKX3GoSLezlwW6fBT9Aasvde6FGvuY
    lsFbKbmAOwdfHjFF03oP
    =ZMgi
    -----END PGP SIGNATURE-----

  5. #5
    dd NNTP User

    Default Re: curl transfer issue

    On 11/26/2012 06:41 PM, ab wrote:
    > Duped. At the same time, sftp worked perfectly for the transfer. Weird...


    to me it sounds like a perfect time to drop into bugzilla and log it.
    here, http://tinyurl.com/nzhq7j

    --
    dd

  6. #6

    Default Re: curl transfer issue

    Quote Originally Posted by ab View Post
    Duped. At the same time, sftp worked perfectly for the transfer. Weird....
    openSUSE 12.2 x86_64
    try with 32bits please

  7. #7

    Default Re: curl transfer issue

    Quote Originally Posted by dd View Post
    On 11/26/2012 06:41 PM, ab wrote:
    to me it sounds like a perfect time to drop into bugzilla and log it.
    https://bugzilla.novell.com/show_bug.cgi?id=791288

  8. #8

    Default Re: curl transfer issue

    I tried again with factory curl, but the problem is still there

    Code:
    # curl --version
    curl 7.28.1 (i586-suse-linux-gnu) libcurl/7.25.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.25 libssh2/1.4.0
    Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
    Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

  9. #9
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: curl transfer issue

    On 2012-12-12 14:16, real doug wrote:
    >
    > I tried again with factory curl, but the problem is still there


    Then I suggest you mention the issue in the factory mail list (with link
    to the bugzilla), specially if another person can duplicate the problem.
    I don't have 12.2 installed.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 12.1 x86_64 "Asparagus" at Telcontar)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •