Is there any way to do it? All I can find in a search is a suggestion to use scp, which is what I’m using. It gives me a consistent 1.1 Mbyte/sec transfer rate, when I should be seeing 100x that. The machines are all on my own internal net, with Gbit ethernet connections (wired) through a router that’s supposed to handle Gbit speeds.
I don’t think it’s a hardware problem, as applications like OpenMPI seem to run at fulll speed.
Thanks,
James
How many do you want 
?
sftp, ftp could work fine.
If you want parts of the other machines available all the time, you can use NFS, it’s bleeding fast.
In fact, it would require a more detailed description of what you want to achieve. Is it just you copying stuff? Are we talking about GUI or CLI? Is sharing folders across machines what you want?
You might find this interesting:
If security is not important (eg secure LAN environment), then there are options available to weaken or disable the encryption with scp, which should lessen the CPU overhead considerably. Other possibilities are to use ftp, netcat, or rsync to do the file transfer faster.
Thanks for the responses. Sorry for not being perfectly clear the first time - I was pretty frustrated watching scp copy at 1.1 Mbyte/sec. Which raises a question: I know scp is supposedly not that fast due to encryption, but this is a new, pretty much top of the line quad-core I7 machine I’m running on. Encryption can’t be that compute-intensive, can it? And in fact the pigs/nc link above quotes ~50 MB/sec rates, which would be acceptable, and way better than the 1.1 MByte/sec I get from scp between any of my four machines.
To clear up the questions, these are occasional file transfers - things like moving the ~700 Mbytes of the CUDA installer on to the new machine, copying back a tarred backup of my home directory on the new machine 'cause I have to downgrade to SuSE 12.1 for CUDA to work, etc. I do use NFS on other machines (and eventually will set it up on this one), and it’s fast while scp is slow. Everything is done from command line. Security isn’t an issue: everything’s inside my local network.
I did try ftp, but it didn’t work at all, just hangs, maybe waiting for some input that I don’t know how to give it.
James
I assume the quad-core does not help much here. The calculation is done on one of them (leaving he others free for other things, but that does not speed up your copy action).
On 2012-11-18 04:36, jamesqf wrote:
> Which raises a question: I know scp is supposedly not that fast due to
> encryption, but this is a new, pretty much top of the line quad-core I7
> machine I’m running on.
And what about the other machine? Both are involved.
> I did try ftp, but it didn’t work at all, just hangs, maybe waiting for
> some input that I don’t know how to give it.
Maybe you’d like to find out why? It is usually a firewalling issue, ftp
uses two ports and one is random.
Another method is http, via apache.
–
Cheers / Saludos,
Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))
If ftp doesn’t work, that’s not good. Like Carlos says, could be a firewall problem.
Are you aware of the fact that Yast is available on a headless server, in it’s ncurses version? Install the yast2 ftp and sshd modules, configure the services through Yast and try again.
It’s a Core2. But since it’s on the receiving end, the decrypt speed shouldn’t be an issue, as it should just buffer to disk, no?
> I did try ftp, but it didn’t work at all, just hangs, maybe waiting for
> some input that I don’t know how to give it.
Maybe you’d like to find out why? It is usually a firewalling issue, ftp
uses two ports and one is random.
Would I like to find out why? Yes. Do I know enough about about firewalling and other network stuff to stand a reasonable chance of finding out why without devoting days to it? No. After all, if it was simple, I wouldn’t need to ask for help 
PS: And I did do some Google searching prior to asking here, with no luck. Lots of hits about slow scp, but their idea of “slow” seems to be in the 50 MByte/sec range over Gbit ethernet, which I’d LOVE to get.
you could also consider SFTP, it requires no more than the SSH daemon running on all machines. If your router does not forward port 22 to a machine, SSH will only work inside the network.
On each machine
su -c 'zypper in yast2-sshd'
Next, run
su -c yast
Go Network Services - SSHD configuration - set it to start at boot and, if running, to open a port in the firewall.
Now, to connect to machine-1 with user X:
sftp REMOTE_USERNAME@REMOTE_IPADRES
You’ll be asked for the remote user’s password, and see a “Connected to …” message. Entering “help” will give you some info, “man sftp” on any openSUSE machine will tell you which commands and options you have. Should be fast, though I think that in your “calculations” you forget the overhead of network traffic.
On 2012-11-18 21:16, Knurpht wrote:
>
> you could also consider SFTP, it requires no more than the SSH daemon
> running on all machines.
It also uses encryption, which may be slow in his combo of machines.
–
Cheers / Saludos,
Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))
On 2012-11-18 19:56, jamesqf wrote:
>
> robin_listas;2504589 Wrote:
>> And what about the other machine? Both are involved.
>
> It’s a Core2. But since it’s on the receiving end, the decrypt speed
> shouldn’t be an issue, as it should just buffer to disk, no?
It has to be decrypted first. It should be faster than encryption, but I
have not tested that.
>>> I did try ftp, but it didn’t work at all, just hangs, maybe waiting
>>> some input that I don’t know how to give it.
>>
>> Maybe you’d like to find out why? It is usually a firewalling issue,
>> ftp uses two ports and one is random.
>
> Would I like to find out why? Yes. Do I know enough about about
> firewalling and other network stuff to stand a reasonable chance of
> finding out why without devoting days to it? No. After all, if it was
> simple, I wouldn’t need to ask for help 
FTP is a bit more complex than other protocols, because it uses two
connections: one for control, another for data. And depends, you may
need to open other ports on the server or the client. It is doable. If
you feel like trying, ask.
Another way, as I said, is http, via apache. It can be used to serve
entire directories, not only for web pages, and it is very simple on the
firewall. Of course, it goes in one direction only, unless both machines
have it. Same as ftp, now that I think.
> PS: And I did do some Google searching prior to asking here, with no
> luck. Lots of hits about slow scp, but their idea of “slow” seems to be
> in the 50 MByte/sec range over Gbit ethernet, which I’d LOVE to get.
–
Cheers / Saludos,
Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))
I’m wondering if it is not something more fundamental, because I just did an net install of openSUSE 12.2 on the new machine, and as packages were downloading, the download speed was almost always something that would round to 1.1 MBytes/sec, never (that I saw) higher than that.
I’ve no clue what it could be. The network configuration is 2 running machines connected (hardwired) to a Trendnet TEW639-GR router, which is supposed to handle Gbit speeds. That in turn is connected to a Motorola Surfboard SB6121 cable modem, but it shouldn’t affect machine-machine speeds. Any thoughts?
On 2012-11-19 05:06, jamesqf wrote:
>
> I’m wondering if it is not something more fundamental, because I just
> did an net install of openSUSE 12.2 on the new machine, and as packages
> were downloading, the download speed was almost always something that
> would round to 1.1 MBytes/sec, never (that I saw) higher than that.
It could be that the cable is bad and speed switches down to 10 mbit/S.
I have seen it happen.
eth-tools might say. Or ifconfig, would show TX/RX errors.
>
> I’ve no clue what it could be. The network configuration is 2 running
> machines connected (hardwired) to a Trendnet TEW639-GR router, which is
> supposed to handle Gbit speeds. That in turn is connected to a Motorola
> Surfboard SB6121 cable modem, but it shouldn’t affect machine-machine
> speeds. Any thoughts?
It should not, unless the router is cheap and switches all ports to the
lower common speed :-? Test by unplugging the modem.
–
Cheers / Saludos,
Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))
Well, I think I have found the problem, but I simply don’t understand it. As I said, I have four machines connected to the router via cable, though usually only one or two are powered on at any particular time. So I unplugged the cable to the other desktop machine (which was not on) and scp xfer speed jumped up to 40 MB/sec.
That perked my curiousity. I do not know why that could happen. … Could it be that the desktop machine that was switched OFF, has a BIOS that allows the PC to be switched ON via network command ? And while that BIOS feature is likely not enabled, the router (or switch/hub) is still picking up an indication that the desktop (which is switched OFF) has an alive Ethernet, and its constantly polling it ? And is polling the switched OFF machine excessively in trying to make a connection, reducing the router (or switch/hub) transfer performance ? It makes me wonder if there is a router setting to stop this. …
What is the behaviour like when that switched OFF desktop is also switched ON ? The same slow speed ?
If you figure out why this behaviour takes place, I would be most interested to learn from you.
You could ping a machine from another one, while plugging other ethernet cables in and out, switching ports on the router, and turning other computers on and off, and see if you can find the culprit. You might also replace the cables one by one - are you mixing cat5, cat5E and cat6 cables? - or put a (good) switch between the router and the lan.
Along the lines of what please_try_again has mentioned, this could be an auto-negotiation issue, causing the affected link to flap.
On 2012-11-21 09:26, please try again wrote:
> cables? - or put a (good) switch between the router and the lan.
The router might not have a switch, but a hub :-?
–
Cheers / Saludos,
Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))
Same here. This reply got me thinking. Reading the whole story again, wouldn’t a (partly) damaged port on the router be an option. Meanwhile I’d love to see what happens if the router was replaced by an other one.