Copying from CF card extremely slow (USB3 card reader)

Hi,

Today I tried to download a few images from my camera card which is compact flash SandDisk Extreme UDMA.

However Dolphin reports extremely low download speed:

https://snag.gy/WOZnQ3.jpg

The card reader is USB3 and so far it has always worked at quite high speed (usually around 70mb/s). This is the first time I see such slowness. I reconnected, ejected the card and tried again - still slow.

What is the reason/fix for this?

Nobody knows?

Or perhaps those who might know haven’t been around here to read and answer your post yet? Anyway, given that you’ve noticed a recent ‘slow down’ with data transfer for this card, perhaps it is beginning to fail? Do you have another CF card available to compare with?

That’s why I ask again :slight_smile:

Anyway, given that you’ve noticed a recent ‘slow down’ with data transfer for this card, perhaps it is beginning to fail?

I don’t think so. The same card works fine and very fast in Windows, on the same computer, same card reader. Something more which I just noticed: Linux seems to damage the file system because after copying the images from the card - Windows sees no files in the directory on the card and also after inserting the card back in the camera: the camera cannot see any file. Yet both Windows and the camera detect that there is some space occupied on the card.

Any idea what might be going on?

For what it’s worth I’ve very occasionally seen the same thing using a Lexar Professional UDMA 7 CF card. (I’m using Tumbleweed not Leap). In my case it’s not just the external card reader as I’ve seen it also with a (Corsair Voyager USB3 GT) USB stick. Once it “happens” all transfers are painfully slow. A reboot “solves” the problem. Happens very seldom and I’ve not bothered to investigate.

Edit: I’ve never had any file system corruption when it happens.

Not in my case.

The same card works fine and very fast in Windows, on the same computer, same card reader.

Something more which I just noticed: Linux seems to damage the file system because after copying the images from the card - Windows sees no files in the directory on the card and also after inserting the card back in the camera: the camera cannot see any file. Yet both Windows and the camera detect that there is some space occupied on the card.

So, which file system is in use here?

Windows says FAT32.

Linux says:

/dev/sde1 on /run/media/*username*/EOS_DIGITAL type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=100,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)
/dev/sde1 on /var/run/media/*username*/EOS_DIGITAL type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=100,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro)

How is the device seen by the system? Open a terminal window and run

dmesg -w

then observe/capture output when the device is first connected.

 7028.910652] usb 6-2: new SuperSpeed USB device number 3 using xhci_hcd
 7028.925782] usb 6-2: New USB device found, idVendor=8564, idProduct=4000
 7028.925786] usb 6-2: New USB device strings: Mfr=3, Product=4, SerialNumber=5
 7028.925788] usb 6-2: Product: Transcend   
 7028.925789] usb 6-2: Manufacturer: TS-RDF8 
 7028.925790] usb 6-2: SerialNumber: 000000080
 7028.926894] usb-storage 6-2:1.0: USB Mass Storage device detected
 7028.926993] scsi host9: usb-storage 6-2:1.0
 7029.927784] scsi 9:0:0:0: Direct-Access     Generic  STORAGE DEVICE   TS22 PQ: 0 ANSI: 6
 7029.928088] sd 9:0:0:0: Attached scsi generic sg5 type 0
 7029.928867] scsi 9:0:0:1: Direct-Access     Generic  STORAGE DEVICE   TS22 PQ: 0 ANSI: 6
 7029.929160] sd 9:0:0:1: Attached scsi generic sg6 type 0
 7029.930352] scsi 9:0:0:2: Direct-Access     Generic  STORAGE DEVICE   TS22 PQ: 0 ANSI: 6
 7029.930596] sd 9:0:0:2: Attached scsi generic sg7 type 0
 7029.930940] sd 9:0:0:0: [sde] Attached SCSI removable disk
 7029.931780] sd 9:0:0:1: [sdf] Attached SCSI removable disk
 7029.936922] sd 9:0:0:2: [sdg] Attached SCSI removable disk

and upon inserting of the card:


 7121.241680] sd 9:0:0:0: [sde] 125001728 512-byte logical blocks: (64.0 GB/59.6 GiB)
 7121.271862]  sde: sde1

and upon accessing the EOS_DIGITAL device in Dolphin:


 7177.818888] usb 6-2: reset SuperSpeed USB device number 3 using xhci_hcd
 7177.837168] sd 9:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
 7177.837173] sd 9:0:0:0: [sde] tag#0 CDB: Read(10) 28 00 00 00 02 23 00 00 3c 00
 7177.837176] blk_update_request: I/O error, dev sde, sector 547
 7178.551688] sd 9:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
 7178.551692] sd 9:0:0:0: [sde] tag#0 Sense Key : Unit Attention [current] 
 7178.551695] sd 9:0:0:0: [sde] tag#0 Add. Sense: Not ready to ready change, medium may have changed
 7178.551698] sd 9:0:0:0: [sde] tag#0 CDB: Read(10) 28 00 00 00 02 24 00 00 3b 00
 7178.551701] blk_update_request: I/O error, dev sde, sector 548
 7178.580286] FAT-fs (sde1): Directory bread(block 30613) failed
 7178.580296] FAT-fs (sde1): Directory bread(block 30614) failed
 7178.580302] FAT-fs (sde1): Directory bread(block 30615) failed
 7178.580308] FAT-fs (sde1): Directory bread(block 30616) failed
 7178.580313] FAT-fs (sde1): Directory bread(block 30617) failed
 7178.580318] FAT-fs (sde1): Directory bread(block 30618) failed
 7178.580326] FAT-fs (sde1): Directory bread(block 30619) failed
 7178.580332] FAT-fs (sde1): Directory bread(block 30620) failed
 7178.580337] FAT-fs (sde1): Directory bread(block 30621) failed
 7178.580342] FAT-fs (sde1): Directory bread(block 30622) failed
 7178.589055] FAT-fs (sde1): FAT read failed (blocknr 484)
 7178.589844] FAT-fs (sde1): FAT read failed (blocknr 484)
 7178.616871] FAT-fs (sde1): FAT read failed (blocknr 484)
 7178.617666] FAT-fs (sde1): FAT read failed (blocknr 484)
 7178.634020] FAT-fs (sde1): FAT read failed (blocknr 484)
 7187.556530] sd 9:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
 7187.556535] sd 9:0:0:0: [sde] tag#0 Sense Key : Aborted Command [current] 
 7187.556538] sd 9:0:0:0: [sde] tag#0 Add. Sense: Incompatible medium installed
 7187.556541] sd 9:0:0:0: [sde] tag#0 CDB: Read(10) 28 00 00 00 0f c9 00 00 78 00
 7187.556543] blk_update_request: I/O error, dev sde, sector 4041
 7188.423163] usb 6-2: reset SuperSpeed USB device number 3 using xhci_hcd
 7188.441515] sd 9:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
 7188.441522] sd 9:0:0:0: [sde] tag#0 CDB: Read(10) 28 00 00 00 11 de 00 00 64 00
 7188.441525] blk_update_request: I/O error, dev sde, sector 4574
 7189.155869] sd 9:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
 7189.155872] sd 9:0:0:0: [sde] tag#0 Sense Key : Unit Attention [current] 
 7189.155874] sd 9:0:0:0: [sde] tag#0 Add. Sense: Not ready to ready change, medium may have changed
 7189.155876] sd 9:0:0:0: [sde] tag#0 CDB: Read(10) 28 00 00 00 11 df 00 00 63 00
 7189.155879] blk_update_request: I/O error, dev sde, sector 4575
 7189.155923] FAT-fs (sde1): FAT read failed (blocknr 4511)
 7193.351257] usb 6-2: reset SuperSpeed USB device number 3 using xhci_hcd
 7193.369896] sd 9:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
 7193.369901] sd 9:0:0:0: [sde] tag#0 CDB: Read(10) 28 00 00 00 1f d4 00 00 76 00
 7193.369904] blk_update_request: I/O error, dev sde, sector 8148
 7194.084289] sd 9:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
 7194.084295] sd 9:0:0:0: [sde] tag#0 Sense Key : Unit Attention [current] 
 7194.084298] sd 9:0:0:0: [sde] tag#0 Add. Sense: Not ready to ready change, medium may have changed
 7194.084301] sd 9:0:0:0: [sde] tag#0 CDB: Read(10) 28 00 00 00 1f d5 00 00 75 00
 7194.084306] blk_update_request: I/O error, dev sde, sector 8149
 7194.085532] FAT-fs (sde1): FAT read failed (blocknr 8085)
 7203.359539] usb 6-2: reset SuperSpeed USB device number 3 using xhci_hcd
 7203.377983] sd 9:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
 7203.377989] sd 9:0:0:0: [sde] tag#0 CDB: Read(10) 28 00 00 00 2d c8 00 00 84 00
 7203.377994] blk_update_request: I/O error, dev sde, sector 11720
 7204.092435] sd 9:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
 7204.092440] sd 9:0:0:0: [sde] tag#0 Sense Key : Unit Attention [current] 
 7204.092445] sd 9:0:0:0: [sde] tag#0 Add. Sense: Not ready to ready change, medium may have changed
 7204.092449] sd 9:0:0:0: [sde] tag#0 CDB: Read(10) 28 00 00 00 2d c9 00 00 83 00
 7204.092469] blk_update_request: I/O error, dev sde, sector 11721
 7204.092668] FAT-fs (sde1): FAT read failed (blocknr 11657)

and upon “Safely removing” in Dolphin:


 7301.004365]  sde: sde1
 7301.106410] sde: detected capacity change from 64000884736 to 0

and upon disconnecting the card reader from the USB cable:


 7350.114878] xhci_hcd 0000:07:00.0: Cannot set link state.
 7350.114885] usb usb6-port2: cannot disable (err = -32)
 7350.114888] usb 6-2: USB disconnect, device number 3

Hi
It’s because of the flush mount option which AFAIK was a recent addition…

What do you mean by “It’s because”? What is “it”? I was simply answering the previous question but please clarify if you have some info.
Thanks.

I tested with an SD card from the same camera today - download speed is high, no issues.

Also tested again with CF - still slow. But today I don’t notice the error messages in ‘dmesg -w’ which appeared yesterday when accessing the card through Dolphin.

I wonder what’s going on.

Tested with Hama USB 2.0 Card reader (35 in 1) - the download speed with the same CF card is 17mb/s, which is fine.

So from all tests so far it seems there is an issue with the Transcend card reader which seems to happen only in Linux, and only with CF cards.

Hi
Look at the man page for mount and the flush option, the filesystem will try to flush to the disk more earlier than normal. This is not set by default but is now. I would guess it’s for the end user since it can be deceptive when writting to external devices and ensuring everything is synced/flushed to the device.

If you mount via the command line and write files to it, does it write quicker? Don’t forget to run sync command before unmounting.

And yes it’s also more than likely a flash device thing. Just because it appears quicker in different operating systems does not equate to faster of slower, it’s how it’s presented to the end user.

So say in Windows I copy a file and it’s done in a few seconds, then go to unmount (eject) it says please wait as the device is busy and that takes 10 seconds, so 12 seconds. In linux I copy the same file and it takes 12 seconds and then select unmount and it’s unmounted (ejected) immediately which OS copied quicker? Neither they both took 12 seconds… So it can be subjective as well…

Before doing anything, let me please clarify and ask:

Right now when the card is inserted in the card reader, it is mounted automatically, obviously using the options which I pasted. This happens for both cards - for the CF (which works slowly) and for the SD (which works fine).

Considering that: What mount options do you suggest? And if they turn out to work better and are to be made persistent (so they can work with the automatic mounting) - how to do that?

And yes it’s also more than likely a flash device thing. Just because it appears quicker in different operating systems does not equate to faster of slower, it’s how it’s presented to the end user.

So say in Windows I copy a file and it’s done in a few seconds, then go to unmount (eject) it says please wait as the device is busy and that takes 10 seconds, so 12 seconds. In linux I copy the same file and it takes 12 seconds and then select unmount and it’s unmounted (ejected) immediately which OS copied quicker? Neither they both took 12 seconds… So it can be subjective as well…

I know that but that’s not the case. Remember that the issue is about reading from the card, not about writing to it (I haven’t tested writing as normally I want the camera to write to the card). In Windows I copy 200-300mb from the card in 3-4 seconds and I can read them instantly on the target drive (and yes, I can disconnect the card reader after these 3-4 seconds). In Linux I have to wait several minutes until those are copied from the CF card. If it is a device thing (i.e. a hardware issue) - why does it happen only in Linux?

Hi
Ahh ok, wonder if it needs a quirk option set for the reader driver (xhci_hcd), but also there are some errors in the output. Maybe a scan/repair of the device under windows may help.

Again, have you tested from the command line, just to see if they are similar. Need to look at eliminating what may be the cause.

How do I do all that?

Maybe a scan/repair of the device under windows may help.

When I tested in Windows (after the Linux test) and when I noticed that the fs is corrupt, Windows suggested automatically to fix fs errors and I did. But anyway I have formatted the card a few times since then.

Again, have you tested from the command line, just to see if they are similar. Need to look at eliminating what may be the cause.

I just did that:

  1. Insert the CF card
  2. Copy files from card using command line - very fast
  3. (without ejecting card) Open Dolphin - copy files to another dir - very fast
  4. Eject the card
  5. Insert the card again
  6. Open Dolphin
  7. Copy the same files - very slow again

Hi
So from the command line after you mount the device, what are the options invoked? Are they different when it’s mounted by Dolphin?

I don’t do any special manual mounting. The card is mounted automatically, just as before. Then I simply run:

cp /run/media/username/EOS_DIGITAL/DCIM/100EOS5D/* .

Are they different when it’s mounted by Dolphin?

No idea. I just drag and drop.