RSync issue

English Applications
1

I would like to backup important files (totaling about 400GB) on my ext 4 RAID 5 array to an ext4 external hard drive over USB (external drive is mounted to /mnt. In the future I’d like to automate the process using rsync and cron so for now I’m using rsync to transfer the files. My problem is that using the rsync command like this:

rsync -Pr “/dir1” “/dir2” “/dir3” “/dir4” /mnt

rsync shows me the checks and transfers for awhile and then throws up an i/o error (wish I had a screenshot to show but I don’t). When I ls /mnt I get a similar i/o error. I then check /dev for the drive and find that it no longer shows up. Originally the partition was /dev/sdc1. I tried unplugging the USB at this point, plugging it back in and mounting the drive back to /mnt, however it has now assigned it to (you guessed it) /dev/sdd1. I get the drive mounted and try the original rsync command again, hoping the first error was a fluke or some kind of one-time drive fart. This time it makes it quite a bit further and then throws up the exact same problem. Am I doing something terribly wrong here? As I said, I’m very new to bash so I’m not making some absolutely moronic, newbie mistake. Any advice would be much appreciated!

2

Hello,

First to the point of showing what your computer says here in the forums. When you work in a terminal (as you aperently do) there is no need to show screenshots. We only need the copied/pasted text of the text inside the terminal emulator (and do not forget to put it between CODE tags: Posting in Code Tags - A Guide). This is the correct way to show us what you typed and what the system answered.

Second, you forgot completely to tell us which level of openSUSE you use. And also if you use a desktop environment and when yes, which one (KDE, Gnome, …). We can not guess all these things!

Third, to see that your device is allways mounted in the same place in /media when you go for an automatic mount, give the file system on the device (I guess there is only one) a Volume Label. When for example he partition ATM is known as /dev/sdc1 and you want to give the file system on it the label bigbackup:

tune2fs -L bigbackup /dev/sdc1

A often called “automatic mount” will then allways be on /media/bigbackup.

And when you want to mount it on another place (like you used /mnt above) then create an entry for it in */etc/fstab *

/dev/disk/by-label/bigbackup /mnt                    ext4       noauto,acl,user_xattr        0 0

You can then mount easily with

mount /mnt

But your main concern is about the unwanted unmounting of the device. Are you sure you have a good connection to the device, no looose contacts, try another USB port. etc.

3

Thank you for your quick response and your tips on code etiquette and mounting. Yes, the unwanted unmounting is my big problem and I’m really not sure why it’s doing it. I did indeed change USB plugs after it happened the first time but I don’t think it has to do with the plug since I use them all fairly regularly and they all have worked without a hitch so far. I double checked that the cable is securely connected to the HDD and the desktop. It’s in the process of rsyncing again right now and I’m making sure to stay clear of that room so as not to unintentionally bump the drive. Does it make any difference that the RAID array is running off an Adaptec card? Could it possible be a problem with the HDD settings since it’s run from an external device?

Sorry I didn’t list my OS info. In other forums I’m used to having a signature with that information but I guess I haven’t achieved “signature” status in this forum yet. I’m running openSUSE 11.4 Gnome. Thanks again for your help!

4

I thought everybody can have a signature here, but I can not test that because I am allready too long a member :frowning:
Some use their signature for this, but as many people have more systems with different levels it is not bad to start any request with that info, thus people do not have to search for it (and after all, at least theoretical, it is good to make it easy for others, then you get more/better answers :wink: ).

When it isn’t the USB connection, it may be the disk itself. I do not know, I am not that good in hardware (either it works or it is lousy). Did you look in

dmesg

immediatly after your device is gone? May be it has some usefull log about the event. Dmesg is very long, thus you should go to the end for a recent event.

5

On 08/14/2011 03:56 PM, hcvv wrote:
>
> I thought everybody can have a signature here

i read somewhere there is an anti-spam thing which keeps new posters
from establishing a ‘sig’ until after some pre-set minimum number of
posts…(sounds like a good idea to me, but then it is a pain to a
person who has been around the block, like apparently has the OP)

> When it isn’t the USB connection, it may be the disk itself.

those are my first thought (cables/connections, disk and if it is a disk
in an enclosure maybe it is overheating and clicking offline, or don’t
some enclosures have a controller of their own, maybe it is garbage

finally (and i nowhere near sure about this) but maybe the problem is in
the command line switches and the format you are using…you might just
need to spend some more time in man rsync (or the thousands of blogs and
guides for rsync–but be careful and always check those against your
rsync manual…because switches DO change around from time to
time…and, the blog you read might be 15 years old!!)

for example, i make backups with something like this:


rsync -avzr * /mnt/[somewhere]

will make an exact copy of the current directory (where rsync is run)
and everything below it into /mount/[somewhere]

and if i were gonna wanna move a lot of directories (but not all) i
would sub in place of the * this


--files-from=FILE

where i would replace FILE with a fully qualified path to a file name
with an internal listing of the directories i wanted backed up…that
way you could put the rsync in cron and then if/when you might wanna
change the list of stuff to backup, you could easy edit (say)
/home/[you]/stuff/directories_to_backup (in which case the full rsync
command would be


rsync -avzr --files-from=/home/[you]/stuff/directories_to_backup
/mnt/[somewhere]

now, rsync is more complicated than i am…so, maybe i made a mistake
and we can both hope someone corrects it…until then read my sig caveat
and move carefully…

oh, and by the way: thank you for your service to humanity.

–
DD Caveat
openSUSE®, the “German Engineered Automobile” of operating systems!

6

My guess is that you can (mis)use rsync how much you want, but after all, rsync is a normal end-user program only doing reads and writes like many programs and no unmount is done by whichever option you use with it.

7

I’m absolutely positive there are more efficient ways of using rsync, and after having poured over the manual I know simply that I’m skimming the surface. Hopefully after more time and practice (and attempting later on to use it as a scheduled daemon) I’ll be a lot better. Thank you DD for getting me thinking about better ways to use the command. I’m particularly interested in specifying a file with an internal listing of the files I want transferred. I will definitely have to look into that later. I think both of you are right in saying that it’s most likely hardware related. I’m very doubtful the rsync command is supposed to unmount drives. I’m going to attempt this procedure on a different drive just to check. I took the earlier advice to check

dmesg

but didn’t see anything that stood out. Looked like the drive just unmounted itself for some strange reason. I’ve had other problems with this drive (although nothing like this). The housing has an eSATA port which worked fine back when I was using it on Windows but for some reason it refuses to mount from it in openSUSE. Anyway, I’ll just try on a different one to ensure it’s a hardware issue.

Just read over my original post and I wish to correct a rather egregious mistake. What I meant to say is that: “I’m very new to bash so I hope I’m not making some absolutely moronic, newbie mistake.” Sorry if I came off as a jerk.

Oh, and DD - Right back at you brother.

8

Just read over my original post and I wish to correct a rather egregious mistake. What I meant to say is that: “I’m very new to bash so I hope I’m not making some absolutely moronic, newbie mistake.” Sorry if I came off as a jerk.

Not at all. But being a newby at bash does not explain why your disk us unmounted (as long as you do not type umount statements :wink: )
In fact I gave not more then a quick glance at your rsync statement, because, as said above, whatever program you run which whatever options, it would normaly only do user read/write calls, no matter if they are usefull to your goal or efficient or what.

9

Hi there,

If a USB drive is mysteriously detaching, I have nearly always found it to be due to either a bad disk or crappy controller. The following tips might help.

First, identify the UBS to ATA/SATA controller being used. You can typically see this by running lspci - usually you can find the chipset of the USB controller reported. (Another option: run: udevadm monitor (as root) prior to plugging in the device, then plug it in an examine the messages.)

Next, see if the controller identified is listed at: Supported_USB-Devices – smartmontools

If so, you may be able to use smartmontools to the drive smart data through the USB controller, such as:

#smartctrl -d sat -a /dev/sdc1

(Or some variation thereof.) If successful, look at the reallocated sectors, and pending sectors. You will likely get a very good idea of the health of the drive from examining these attributes.

If you can’t read the smart data through the USB interface, I would take the drive out and connect it directly. Until you know for certain if the drive is good, you might just find yourself going in circles.

Lews Therin

10

On 08/14/2011 09:16 PM, nicktlloyd wrote:

> What I meant to say is that: “I’m very new to bash
> so -I hope- I’m not making some absolutely moronic, newbie mistake.”
> Sorry if I came off as a jerk.

amazing how the mind/eye works! i accidentally misread the original to
mean exactly what you meant it to convey (as above)…

therefor: jerkless.

–
DD
openSUSE®, the “German Engineered Automobile” of operating systems!

11

Hi,

I’ve also had a few issues with “long” file operations (around a TB or so) on USB disks which have resulted in the drive “disappearing”. The issue has usually been down to power and quite often happened when I’ve installed a new drive in the case but also with 2.5" drives using the USB port for power. No easy fix unfortunately…

You say your drive is esata capable as well as USB so it could be worth trying that again. If you run dmesg from your bash prompt and make a note of the last couple of lines. Then connect the drive to the esata port on your PC and then run “dmesg” from your bash prompt again you should see the drive detected. If you do see some new messages but don’t get a mounted drive then its probably worth posting the messages on the forum here and someone may be able to help you get that working.

If you’re lucky then if you can get the drive mounted using esata then you may find it’s more reliable - its certainly worked for me on occasion.

Good luck!

Steve

12

Thank you all for the help in diagnosing this issue. I performed the same operation on a different drive which worked just fine. This leads me to believe that it’s a hardware issue. I’m going to scan the drive later for bad sectors and check it on a different machine. It very well may have had something to do with the drives power. If I can get the drive working properly again I’ll definitely take earlier advice to attempt mounting via eSATA again and diagnosing that particular problem. Thanks again for everyone’s help!

13

You are welcome. I think @audiocomp’s suggestion about the power consumption is indeed worthwhile to take into account.