Results 1 to 2 of 2

Thread: The fast & secure way to transfer large data

  1. #1
    Join Date
    Jun 2008
    Albuquerque, NM, USA

    Lightbulb The fast & secure way to transfer large data

    You might think, scp is easy, secure, and usually fast enough. But the truth is that for large files scp is slow.

    Thankfully tunneling through ssh is fast. You might also think that scp is tunneling through ssh. But scp actually uses these nifty internal buffers that really slow things down.

    Using tar and lzma compression in openSUSE 11.1 or gzip compression in less than openSUSE 11.1 you can significantly reduce the time to securely copy large files from one machine to another.

    tar --lzma -cf - <source> | ssh <host> tar --lzma -xf - -C <destination>
    For example:
    tar --lzma -cf - /home/mdb/NeverwinterNights/NWN/ | ssh tar --lzma -xf - -C /home/xemiu/
    It's really that much fun!
    Geeko Samurai (cc)
    My Recent Tracks

  2. #2
    Join Date
    Jun 2008
    Blog Entries

    Default Re: The fast & secure way to transfer large data

    If this on your LAN and you are not concerned about interlopers, you can do this:

    At the receiving end run:

    netcat -l -p 12345 | tar xvf -

    Then go to the sending end and run:

    tar cf - . | netcat recv 12345

    where recv is the name or IP address of the other end.

    This assumes you are in the right directories already, so adjust as necessary.

    I've used a variant of this to clone a machine onto a disk on another, and as a last step I install the bootloader.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts