Whether I use SFTP or Samba, transferring files over the network is extremely slow (~1 MB/s). I know I can physically transfer data much faster than that as X-tunneling through SSH is ~10 MB/s, which is what should be expected (100 Mb/s = 12.5 MB/s). I don’t want to use NFS (no way to access on Windows without installing something). Any ideas on getting a better transfer speed? It should not take half a day to copy 50 GB.
Then try rsync.
I suppose that a Windows standalone executable is available. Otherwise it should be available with Cygwin.
rsync is about the same speed as SFTP. I suppose it could be faster if I ran an rsync daemon, but something about the storing passwords in a human-readable plain text file screams “bad idea.”
We need more details on the setup in question. What distro are you using for the server at the other end? How do you know ssh -X is “~10 MB/s;” how did you measure it? It sounds to me like you have a network issue – perhaps even hardware; maybe you have a “smart” switch and it has defaulted to half-duplex, for example.
Name the OS’s at each end, name your hardware, describe the setup in more detail and someone should be able to help you. Is the server remote? Or is it local, on the same 100 Mbit Ethernet? Are you using a hub, a switch, a router, etc., etc. … need more info.
Speaking from experience, a lot of network hardware will default to slower speeds and/or half-duplex if it senses any problems with the link.
Hmm, that’s quite odd because I get the opposite happening. Anything copied through an ssh connection is extremely slow at about 20% of the fastest available speed!
This happens on both my laptop (wireless) and desktop (100 lan), my server runs openSUSE 11.1 32bit, and my 2 other machines run openSUSE 11.2 64bit, and I’m on my 2nd modem/router now (both different brands).
@david
I have seen a similar problem and in my case the solution was to use a different driver module for the NIC. If you know how to use wireshark you can take a look at what happens during the transmission.
Also a good suggestion. I’ve had that happen, too – change the driver, she starts working.
The OP hasn’t replied with more details yet, so I guess we’re just talking to each other, though. So … how y’ doin??? rotfl!
Sorry about the delay at getting back.
I know ssh -X is getting me ~10 MB/s from ksysguard.
The network driver in use is r8169, which I am pretty sure is the correct driver (the mobo specs say it is a Realtek 8111C). Here’s what lspci tells me:
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
I don’t think it’s in the other network hardware, because performance is about the same whether on my local switch or cross-campus. ethtool tells me the connection is 100 Mb/s full duplex. Setting up a direct link to my second ethernet port is faster, but that’s because then I can get a gigabit connection.
Just for my general information (and my apologies to the OP for this thread hijack) what would I expect to see with wireshark if the link was running, but running sub-optimally due to the wrong drivers?
This is the right driver.
You can use iperf to perform speedtest.
Software.openSUSE.org
Example :
on the server :
root@server: iperf -s
on the client :
root@client: iperf -i 1 -t 30 -c server
- where ‘server’ describes the listening machine.
I don’t have another computer with a wired connection that I can install iperf on, but over WiFi, I’m pretty consistently getting ~40 Mb/s, which is only slightly under the physical link speed.
Another thought, is there any file server protocol that operates over UDP, rather than TCP? I know UDP is generally faster and substantially more aggressive; so, if such a thing exists, it would probably be faster.
NFS used to run over UDP and can still be made to do so. However it ends up doing its own retransmits anyway. You’ll find that any protocol that cares about content integrity uses TCP.
There’s something badly wrong with your network, can’t say what, but you’ll have to do some more fact finding, like maybe packet tracing, to work it out.
Yeah, I know TCP is for integrity, but I’m pretty sure it also cares about order, and won’t just keep sending packets without waiting for acknowledgment.
I know there’s something badly wrong with the network. It’s a university network complete with one of the world’s worst evils—traffic shaping, but that’s only supposed to affect internet traffic, not local. That’s why I asked about UDP. UDP connections on high-numbered ports seem to evade traffic shaping.
TCP has a window and also SACK.
Maybe you should run a UDP based VPN on top of the network. But then this is a LAN, no? Your traffic is not going through their routers is it?
According to the “official” network architecture, everything I have should be on one side of a Juniper M10i router and then just switches from there, but who knows what it’s really doing.