Speed cap?

When I transfer files through Konqueror via fish:// (SFTP right?)… There seems to be a cap of ~1.2MB/s it can burst to almost 2.

Now here’s the weird part, if I start a second transfer, it gets the same speed of ~1.2MB/s… If there was a limit of transfer speed, limited by CPU or network, then that 1.2MB should drop down to ~600Kb, right?

So my question is this, how can I make my transfer faster? I have another 50 gigs to backup to my server so I can finally delete the old vista partition I have on my box… I don’t feel like waiting.

P.S. I’ve seen speeds of 6MB/s on a an XP box in safe mode using WinSCP, so I know my server is capable of a faster transfer.

On Fri, 22 Aug 2008 08:16:02 GMT
Crazy PotHead <Crazy_PotHead@no-mx.forums.opensuse.org> wrote:

>
> When I transfer files through Konqueror via fish:// (SFTP right?)…
> There seems to be a cap of ~1.2MB/s it can burst to almost 2.
>
> Now here’s the weird part, if I start a second transfer, it gets the
> same speed of ~1.2MB/s… If there was a limit of transfer speed,
> limited by CPU or network, then that 1.2MB should drop down to ~600Kb,
> right?
>
> So my question is this, how can I make my transfer faster? I have
> another 50 gigs to backup to my server so I can finally delete the old
> vista partition I have on my box… I don’t feel like waiting.
>
> P.S. I’ve seen speeds of 6MB/s on a an XP box in safe mode using
> WinSCP, so I know my server is capable of a faster transfer.
>
>

Ya, have seen that myself. I tend to use ‘rsync’ if possible, rather than
‘scp’, as it will compress and uncompress on the fly to reduce the amount of
traffic actually sent across the wires.

There’s no cap that I know of, although the need to encrypt/decrypt data
could be a factor too.

I routinely push data at ~10MB/s on my internal lan. Are you using all
100BaseT devices (or 1000BaseT/Gigabit)? My big machine is the only gigabit
nic here, so it tends to saturate the poor 100bT systems.

Any other traffic on the wire to account for?

if you’re using (for example)

scp -r * me@machine:/path/

rsync -avzP * me@machine:/path/

will do the same, and compress data to move more through the wire. (It’s got
many more features, read the man pages {Grin})

Are you using Windows/WinSCP on either end?

Is your nic using full duplex? The ‘ethtool eth0’ tool will tell you.

The cipher and compression used for the ssh connection does have an impact on
transfer speeds, using a ‘lighter’ cipher method would of course increase
transmission speeds.

Compression is handled automatically by Version 2 ssh connections, so that’s
not really an option to play with.

The cipher used to encrypt the session can be changed with the ‘-c xxx’
option to ssh, blowfish seemingly the fastest cipher overall: (again, for v2
ssh connections)

scp -c blowfish-cbc a b:c

You CAN set a transfer speed limit with the ‘-l’ option, units are Kbit/s
(doubt you’d want that)

Basically, it seems that overall speed is somewhat limited by the buffering
methods used by the individual implementations of ssh/scp. Some research has
been done to develop ‘high-performance’ versions of ssh/scp, as seen here:

http://www.psc.edu/networking/projects/hpn-ssh/

Good reading if you’re interested.

I believe using your current scp command along with the above ‘-c
blowfish-cbc’ should produce some good speed improvements. You may also try
rsync, although to supply the ‘-c blowfish-cbc’ option to ssh, you’ll need to
add:

Host *
cipher blowfish-cbc

to your .ssh/config file on the sending machine.

You might also consider moving the machines closer together and using shorter
cabling between them… every foot of distance the electrical signal must
travel works out to about 1 picosecond of propagation delay. this delay CAN
build up to nanosecond levels in some cases. {Grin} (sorry, couldn’t help it)

Loni


L R Nix
lornix@lornix.com
They’re all tubes!

Lol… I was debating on posting some of that info… it’s all 100BaseT/Full… Everything would be gigabit, but I haven’t really seen a need to upgrade my switches

I guess I’ll just use plain old FTP, I don’t have to worry about security… I’ve never used Rsync aside from “emerge” back in my gentoo days

But on that note, which protocol do you think would give me the best performance(speed wise)… And again it’s over a wired LAN so security is of no concern to me… Plus most of it is just legally downloaded TV and music;)

Also… I may have to transfer some files to a windows box because it has a lot more storage… different protocol suggestion for that?

On Fri, 22 Aug 2008 11:06:02 GMT
Crazy PotHead <Crazy_PotHead@no-mx.forums.opensuse.org> wrote:

>
> Lol… I was debating on posting some of that info… it’s all
> 100BaseT/Full… Everything would be gigabit, but I haven’t really seen
> a need to upgrade my switches
>
> I guess I’ll just use plain old FTP, I don’t have to worry about
> security… I’ve never used Rsync aside from “emerge” back in my gentoo
> days
>
> But on that note, which protocol do you think would give me the best
> performance… And again it’s over a wired LAN so security is of no
> concern to me… Plus most of it is just legally downloaded TV and
> music;)
>
> Also… I may have to transfer some files to a windows box because it
> has a lot more storage… different protocol suggestion for that?
>
>

Hmmm, myself, I’d probably just CTRL-ALT-F1/F2/F3 to a console screen, log in
and setup/start transfers from there… that way it’d be running all the
time, and I’d not be bothered by it on the desktop where I would probably
click the little ‘X’ and close it accidently. Oh, and that’s why I use
rsync… it won’t copy stuff that’s already been copied. So if you stop it
accidently or otherwise, just reissue the same command, and it’ll pick up
where it left off. BIG help there.

FTP’s ok, too easy to forget BIN mode and screw things up though, not too
many ‘automated transfer’ type applications come to mind though.

NFS works well between linux boxes, considering that’s what it was designed
for… I get good transfer rates between systems that way.

As for the windows side… I wouldn’t trust windows to transfer gigabytes of
anything through the samba filesystem, it always seems to burp and barf all
over whenever I ask it to transfer ISO’s. (stupid windows!)

Probably WinSCP again. Although, if you’re really looking at DAYS of file
transfer, you might peek at the NFS services for windows, you could use that
to connect windows and linux through a central NFS server on the linux box.
Works quite well and is pretty zippy, relatively simple install in windows
too.

Download for Windows SFU:
http://tinyurl.com/6fptu5

http://www.microsoft.com/downloads/details.aspx?familyid=896C9688-601B-44F1-81A4-02878FF11778&displaylang=en

Tech Support for SFU:
http://tinyurl.com/38us3k

http://technet.microsoft.com/en-us/interopmigration/bb380242.aspx

(I always see that as STFU… which isn’t the same!)

Could always put files to be transferred into a web server, then use IE or
FF3 to drag-drop files from linux to windows. (see! no samba!)

Loni


L R Nix
lornix@lornix.com
Moving Moving Moving, Keep that Data Moving!