USB flash drive corrupts large files?

I have a 16 Gn flash drive that I use to transfer data between systems. However, when I try large files (100 Mb or so) in .tgz format, I get messages like “gzip: not in gzip format” when I try to unpack the file on the receiving system. Can someone give me an idea of what might be wrong, or at least where to start looking?

Both systems are SUSE 11.0. One’s a Thinkpad T61, the other a Lenovo desktop. My older flash drive seems to work ok on the small files that are all its 128 Mb can handle.

Thanks,
James

Are you clicking on Safely Remove Drive before you unplug it? If not all the data has been flushed to the drive, then you would lose data.

No, because there is nothing of that sort on my system. I do a “umount /media/FlashDisk” before removing, which should flush everything, and the files on the flash disk are the same size as the ones on the source system.

I don’t suppose there’s any sort of utility that would check to see if there’s a problem with the flash disk itself, is there?

If you are using KDE, then you can Safely Remove using the device notifier in the system tray or from the device icon in dolphin. I’m sure there is something similar for GNOME. Although the umount should also do it.

You can check a flash drive just like any other drive.

fsck /dev/sdc1

or whatever the device is. It mustn’t be mounted for the check though.

BTW file size is not a reliable indicator that the data has been copied correctly. Depending on the details of the copy, the directory data may have been first flushed to disk, giving the right size, but the data blocks haven’t.

But I don’t use either KDE or Gnome. If I wanted a Windows-like user interface, I’d just run Windows :slight_smile: Since I want something that meets my needs better, I use FVWM.

You can check a flash drive just like any other drive.

fsck /dev/sdc1

or whatever the device is. It mustn’t be mounted for the check though.

I’ll give that a try, though I though that fsck would just check the filesystem structure. I was thinking of something that could check the physical hardware, if there is such a thing?

Thanks,
James

On 2009-02-01, jamesqf <jamesqf@no-mx.forums.opensuse.org> wrote:

> But I don’t use either KDE or Gnome. If I wanted a Windows-like user
> interface, I’d just run Windows :slight_smile: Since I want something that meets
> my needs better, I use FVWM.

Don’t be rude. :slight_smile:

To flush the data from the write-cache, I use ‘sync’.

The only other problem that using an USB could bring you is the 2GB per file
limit that a FAT formatting brings. But since you’re talking about 100MB…


Elevators smell different to midgets

No problem, whatever rocks you. Just use the CLI to umount it then.

Maybe someday there will be a Fubuntu for you. lol!

I wasn’t intending to be, just stating a fact. I don’t get on well with Windoze-like GUIs (to the point where I’m almost completely non-productive), so the freedom to have the kind of interface I can work well with is a major factor in my use of Linux.

To flush the data from the write-cache, I use ‘sync’.

I don’t think the problem could be caused by the write-cache not being flushed, unless it’s actually some sort of system problem. I have two systems. Normally only one of them will be running at a time. They share a common keyboard and external monitor, and the flash drive is connected to a USB port on the monitor. So if I intend to switch systems, I’ll copy the files to the flash drive, then do other things, maybe for an hour or more, then shut down the system (which should include syncing the flash drive, no?), switch power off at the single surge protector/power strip, swap a couple of cables, and power back on.

I tried fsck on the drive. It marked some bad clusters, and the few small files I’ve had time to try seem to have copied correctly, so I’m keeping fingers crossed that that fixed the problem. I still wonder why the sectors were bad, though, and if there’s some way to check the underlying hardware.

Thanks,
James

I have the same problem transmitting a bunch of small files over at the same time. I frequently transfer mp3s to a flash drive. They come over corrupted as well.

I am using KDE4, and this happens when I transfer files over, and then click the Device Manager and eject not long afterward. There is something in the action of forcing the flushing of the cache that is corrupting the files.

If I leave the USB plugged in for a very long time after transfering the files, and then unmount the usb drive, the files are fine.

Does this help? I wouldn’t mind an answer to this one…

I am using KDE4, and this happens when I transfer files over, and then click the Device Manager and eject not long afterward. There is something in the action of forcing the flushing of the cache that is corrupting the files.

Have you tried unmounting the device first via ‘umount’ CLI? What version of KDE 4 are you running? The CLI ‘sync’ command can also be used to force disk write completion. See how that goes.

It’s the 64bit OpenSuse 11.1, so it’s KDE 4.1.xx I think.

I have tried unmounting via CLI, with the same results. I haven’t tried the ‘sync’ command though. I will try that when I get home and have access to the machine again.

It does seem to me though that the ‘sync’ command would already be called when I choose eject from the Device Manager - isn’t it?

The whole file is there, and in the case of mp3s they are playable - the file is corrupted though and the mp3s have skips and jumps in them. If I checked the md5 on the original and the copied version, there is no way they’d match.

Update: sync does NOT work. It still corrupts the mp3’s I am moving. I know it isn’t the flash drive, because I’ve tried multiple flash drives, and I’ve tried the flash drives on a Fedora machine.

So, now we know that sync isn’t working… What do you guys think it is? it really is truly a maddening experience! >:(

Did you try other USB ports? If on a desktop try one in the pack panel, as it’s wired directly to your motherboard, without cables. Cheap long USB cables/front-panel connectors are unreliable at best.

I presume you already checked your bios settings/version. In an intel P4 MoBo I could never r/w reliably when bios was set to USB 2.0 support and/or legacy enabled. Only worked after a bios upgrade, both USB 2.0 and legacy (needed to boot with an USB kbd, I think).

That’s something I didn’t think about, only because I’ve had this machine for quite a while and I’ve never had an issue. I have been using only the front panel usb slots, so I will check the back panel ones that are directly wired to the motherboard.

I don’t think it’s the bios, because I’ve never had an issue with this before, either with OpenSuse 11.1 or Debian 5.0; which have been the two primary OS’s on this particular machine. The problem had only started occuring within the last month, and I’ve had 11.1 on it for the last 3 months or so.

The front usb connections could have degraded though, it’s not an expensive case - but not a real cheap on either. I’ll check the back usb’s soon and report back.

Huh, I checked the back usb ports, and I have the same problem. I really thought we had it that time.

I can’t find any error messages anywhere. I’m not sure where to turn at this point, as long as I’m using USB sticks, I get corrupted data when moving from teh computer to the usb stick.

Turns out, it wasn’t my USB hubs or OpenSuse itself. Mashpodder (Chess Griffin’s version of Bashpodder) was corrupting my podcasts during the downloads.

I’m a little shocked, honestly. Does anyone know of any problems with mashpodder, bashpodder, or wget???

Turns out, it wasn’t my USB hubs or OpenSuse itself.

No surprises here. It works reliably for 99.999999% of users.

Mashpodder (Chess Griffin’s version of Bashpodder) was corrupting my podcasts during the downloads.

Never heard of it, but if you had mentioned this from the outset, we would not have been left guessing. Normal file system copying works perfectly.

Mashpodder is a bash and perl script that uses wget. Nothing more. If something were to go wrong - that’d be the last place you’d look. It also works for 99.99999% of users.

So, at the outset of a problem, you clearly see every single variable?