FTP file corruption from 32 bit Windows clients

English Network/Internet
1

We have a very odd issue we are trying to solve. We have several FTP servers that serve up FTP, Samba and NFS. Some run SLES 10 SP1, the later ones run SLES 11 SP1. The 10GbE NICs installed in the machines are identical. We need to move FTP clients to the servers running SLES 11 SP1.

During this we noticed that if you MD5 a file that has been transferred via FTP to a Windows 32 bit client from a SLES 11 SP1 server it does not come back as expected. The file has become corrupted. The same process from a SLES 10 SP1 server yields the correct MD5.

If you do the same transfer using Samba the MD5s are correct from both servers. We have tried building pure-ftp from source and running the same versions on both SLES 10 and SLES 11. SLES 11 still has the corruption issue.

To add confusion, performing an FTP transfer from a linux client or Windows 64 bit client, the FTP transfers are fine from both the SLES 10 or SLES 11 servers…

We have tried upgrading the network drivers on the windows 32 bit clients and the Linux servers, but made no difference. Also tried disabling the TCP offload checksum on the windows 32 bit clients - again, no change.

Any ideas would be much appreciated!

2

On 09/11/2013 07:16 AM, dom era wrote:
>
> We have a very odd issue we are trying to solve. We have several FTP
> servers that serve up FTP, Samba and NFS. Some run SLES 10 SP1, the
> later ones run SLES 11 SP1. The 10GbE NICs installed in the machines are
> identical. We need to move FTP clients to the servers running SLES 11
> SP1.

Good initial description.

> During this we noticed that if you MD5 a file that has been transferred
> via FTP to a Windows 32 bit client from a SLES 11 SP1 server it does not
> come back as expected. The file has become corrupted. The same process
> from a SLES 10 SP1 server yields the correct MD5.

Stop using windows? I (mostly) kid.

> If you do the same transfer using Samba the MD5s are correct from both
> servers. We have tried building pure-ftp from source and running the
> same versions on both SLES 10 and SLES 11. SLES 11 still has the
> corruption issue.

Different protocol, so the difference appears to be related to FTP at this
point.

> To add confusion, performing an FTP transfer from a linux client or
> Windows 64 bit client, the FTP transfers are fine from both the SLES 10
> or SLES 11 servers…

And since FTP is both a protocol as well as a command, apparently the
implementation differs between proper OS’s and deficient alternatives.

> We have tried upgrading the network drivers on the windows 32 bit
> clients and the Linux servers, but made no difference. Also tried
> disabling the TCP offload checksum on the windows 32 bit clients -
> again, no change.

Um… wow. Network drivers and Linux (which works) itself? Disabling TCP
checksum offloading? Those are some interesting steps, but why do them
when other layer seven protocols (SMB via Samba) work?

The problem is almost certainly that you’re using FTP (on windows) in
ASCII (vs. BINARY) mode. If you are using the command line just type
‘binary’ once you have authenticated to the FTP server. Linux does this
by default because it’s not stupid and ASCII mode is stupid.

Good luck.

3

Are you aware of the fact that these are the openSUSE forums?
The SLES/SLED forums are at: https://forums.suse.com/

People here might be willing to help you, but as they have another system then you have, trying to recreate the problem will be difficult (even if they have Windows systems available for such a test).

4

On 2013-09-11 15:16, dom era wrote:
>
> We have a very odd issue we are trying to solve. We have several FTP
> servers that serve up FTP, Samba and NFS. Some run SLES 10 SP1, the
> later ones run SLES 11 SP1. The 10GbE NICs installed in the machines are
> identical. We need to move FTP clients to the servers running SLES 11
> SP1.

Sorry, but these are the openSUSE forums, not the SLES/SLED forums.
Please post here instead: SLES/SLED
forums (same login/pass).

–
Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

5

Sorry I should have mentioned that we confirmed that transfers were all being completed in binary mode, that was our first conclusion, but we shall recheck.

Also agree that driver updates and disabling TCP offloading was unlikely to change things when other protocols function correctly, but these seemed like sensible things to try.

I shall move this to SLES forums.

Thanks for the comments so far.

Please close this post.

6

On 09/11/2013 08:46 AM, dom era wrote:
>
> ab;2584100 Wrote:
>> On 09/11/2013 07:16 AM, dom era wrote:
>> Um… wow. Network drivers and Linux (which works) itself? Disabling
>> TCP
>> checksum offloading? Those are some interesting steps, but why do them
>> when other layer seven protocols (SMB via Samba) work?
>>
>> The problem is almost certainly that you’re using FTP (on windows) in
>> ASCII (vs. BINARY) mode. If you are using the command line just type
>> ‘binary’ once you have authenticated to the FTP server. Linux does this
>> by default because it’s not stupid and ASCII mode is stupid.
>>
>> Good luck.
>
> Sorry I should have mentioned that we confirmed that transfers were all
> being completed in binary mode, that was our first conclusion, but we
> shall recheck.

Good to know. Yes, this is usually the issue.

Other possible issues could arise around file size. For example, if the
‘ftp’ client on windows does not handle large files, for some varying
definition of “large”, then that would explain things. To rule this
in/out test with something else, like Firefox as the FTP client or some
other tool like FileZilla.

> Also agree that driver updates and disabling TCP offloading was unlikely
> to change things when other protocols function correctly, but these
> seemed like sensible things to try.

Fair enough. I’m glad they didn’t pan out as I would have been profoundly
confused had those been related to any significant degree. For now I’d
try ruling out the client utility by using other clients. Obviously your
filesystem can handle the file (windows filesystems often have problems
with larger files depending on the one used, such as fat32) which lead to
your exact symptom, and there have been issues for years with “larger”
files not working when downloaded via tools like microsoft’s internet
explorer… yet another reason to use an alternative browser or tool.

Good luck.

7

The description and analysis of the Win32 clients is severely lacking… most critically omitting the name of the client app used.

In fact, I would test several FTP client apps, it’s quite common with all protocols that some clients work better than others.

Common clients I would test on a Winbox…

IE (of course)
Filezilla (a popular third party)
Command line (rarely used but it’s nearly as plain and simple as on Linux)
CuteFTP (another popular client)
SmartFTP

In my Windows days when I was regularly transferring enormous numbers of files regularly, I’ve used all of the above and naturally preferred one over another at various times… for quick and easy (Command line), Features (any/all of the 3rd party apps), practicality particularly if some non-technical user needed FTP access (IE).

Remember also that by default most FTP transfers will be according to default which means no verification checks. This is common and usually fine when transferring files only a few kb in size but risky the larger the files are. If you need a verified transfer, then you should be using one of the 3rd party apps which typically do verification because they allow pause/resume, multi-stream downloading and maybe even multi-protocol downloading… and verification is needed to support all of these additional features (and isn’t supported in plain FTP).

HTH,
TSU