Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: RSync issue

  1. #1
    Join Date
    May 2010
    Location
    Vicenza, Italy
    Posts
    23

    Question RSync issue

    I would like to backup important files (totaling about 400GB) on my ext 4 RAID 5 array to an ext4 external hard drive over USB (external drive is mounted to /mnt. In the future I'd like to automate the process using rsync and cron so for now I'm using rsync to transfer the files. My problem is that using the rsync command like this:

    # rsync -Pr "/dir1" "/dir2" "/dir3" "/dir4" /mnt

    rsync shows me the checks and transfers for awhile and then throws up an i/o error (wish I had a screenshot to show but I don't). When I ls /mnt I get a similar i/o error. I then check /dev for the drive and find that it no longer shows up. Originally the partition was /dev/sdc1. I tried unplugging the USB at this point, plugging it back in and mounting the drive back to /mnt, however it has now assigned it to (you guessed it) /dev/sdd1. I get the drive mounted and try the original rsync command again, hoping the first error was a fluke or some kind of one-time drive fart. This time it makes it quite a bit further and then throws up the exact same problem. Am I doing something terribly wrong here? As I said, I'm very new to bash so I'm not making some absolutely moronic, newbie mistake. Any advice would be much appreciated!

  2. #2
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    24,853

    Default Re: RSync issue

    Hello,

    First to the point of showing what your computer says here in the forums. When you work in a terminal (as you aperently do) there is no need to show screenshots. We only need the copied/pasted text of the text inside the terminal emulator (and do not forget to put it between CODE tags: Posting in Code Tags - A Guide). This is the correct way to show us what you typed and what the system answered.

    Second, you forgot completely to tell us which level of openSUSE you use. And also if you use a desktop environment and when yes, which one (KDE, Gnome, ...). We can not guess all these things!

    Third, to see that your device is allways mounted in the same place in /media when you go for an automatic mount, give the file system on the device (I guess there is only one) a Volume Label. When for example he partition ATM is known as /dev/sdc1 and you want to give the file system on it the label bigbackup:
    Code:
    tune2fs -L bigbackup /dev/sdc1
    A often called "automatic mount" will then allways be on /media/bigbackup.

    And when you want to mount it on another place (like you used /mnt above) then create an entry for it in /etc/fstab
    Code:
    /dev/disk/by-label/bigbackup /mnt                    ext4       noauto,acl,user_xattr        0 0
    You can then mount easily with
    Code:
    mount /mnt
    But your main concern is about the unwanted unmounting of the device. Are you sure you have a good connection to the device, no looose contacts, try another USB port. etc.
    Henk van Velden

  3. #3
    Join Date
    May 2010
    Location
    Vicenza, Italy
    Posts
    23

    Default Re: RSync issue

    Thank you for your quick response and your tips on code etiquette and mounting. Yes, the unwanted unmounting is my big problem and I'm really not sure why it's doing it. I did indeed change USB plugs after it happened the first time but I don't think it has to do with the plug since I use them all fairly regularly and they all have worked without a hitch so far. I double checked that the cable is securely connected to the HDD and the desktop. It's in the process of rsyncing again right now and I'm making sure to stay clear of that room so as not to unintentionally bump the drive. Does it make any difference that the RAID array is running off an Adaptec card? Could it possible be a problem with the HDD settings since it's run from an external device?

    Sorry I didn't list my OS info. In other forums I'm used to having a signature with that information but I guess I haven't achieved "signature" status in this forum yet. I'm running openSUSE 11.4 Gnome. Thanks again for your help!

  4. #4
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    24,853

    Default Re: RSync issue

    I thought everybody can have a signature here, but I can not test that because I am allready too long a member
    Some use their signature for this, but as many people have more systems with different levels it is not bad to start any request with that info, thus people do not have to search for it (and after all, at least theoretical, it is good to make it easy for others, then you get more/better answers ).

    When it isn't the USB connection, it may be the disk itself. I do not know, I am not that good in hardware (either it works or it is lousy). Did you look in
    Code:
    dmesg
    immediatly after your device is gone? May be it has some usefull log about the event. Dmesg is very long, thus you should go to the end for a recent event.
    Henk van Velden

  5. #5
    Join Date
    Jun 2008
    Location
    Earth - Denmark
    Posts
    10,730

    Default Re: RSync issue

    On 08/14/2011 03:56 PM, hcvv wrote:
    >
    > I thought everybody can have a signature here


    i read somewhere there is an anti-spam thing which keeps new posters
    from establishing a 'sig' until after some pre-set minimum number of
    posts...(sounds like a good idea to me, but then it is a pain to a
    person who has been around the block, like apparently has the OP)


    > When it isn't the USB connection, it may be the disk itself.


    those are my first thought (cables/connections, disk and if it is a disk
    in an enclosure maybe it is overheating and clicking offline, or don't
    some enclosures have a controller of their own, maybe it is garbage

    finally (and i nowhere near sure about this) but maybe the problem is in
    the command line switches and the format you are using...you might just
    need to spend some more time in man rsync (or the thousands of blogs and
    guides for rsync--but be careful and always check those against _your_
    rsync manual....because switches DO change around from time to
    time...and, the blog you read might be 15 years old!!)

    for example, i make backups with something like this:

    Code:
    rsync -avzr * /mnt/[somewhere]
    will make an exact copy of the current directory (where rsync is run)
    and everything below it into /mount/[somewhere]

    and if i were gonna wanna move a lot of directories (but not all) i
    would sub in place of the * this

    [/code]
    --files-from=FILE
    [/code]

    where i would replace FILE with a fully qualified path to a file name
    with an internal listing of the directories i wanted backed up...that
    way you could put the rsync in cron and then if/when you might wanna
    change the list of stuff to backup, you could easy edit (say)
    /home/[you]/stuff/directories_to_backup (in which case the full rsync
    command would be

    Code:
    rsync -avzr --files-from=/home/[you]/stuff/directories_to_backup
    /mnt/[somewhere]
    now, rsync is more complicated than i am...so, maybe i made a mistake
    and we can both hope someone corrects it...until then read my sig caveat
    and move carefully..

    oh, and by the way: thank you for your service to humanity.

    --
    DD Caveat
    openSUSE®, the "German Engineered Automobile" of operating systems!

  6. #6
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    24,853

    Default Re: RSync issue

    My guess is that you can (mis)use rsync how much you want, but after all, rsync is a normal end-user program only doing reads and writes like many programs and no unmount is done by whichever option you use with it.
    Henk van Velden

  7. #7
    Join Date
    May 2010
    Location
    Vicenza, Italy
    Posts
    23

    Default Re: RSync issue

    I'm absolutely positive there are more efficient ways of using rsync, and after having poured over the manual I know simply that I'm skimming the surface. Hopefully after more time and practice (and attempting later on to use it as a scheduled daemon) I'll be a lot better. Thank you DD for getting me thinking about better ways to use the command. I'm particularly interested in specifying a file with an internal listing of the files I want transferred. I will definitely have to look into that later. I think both of you are right in saying that it's most likely hardware related. I'm very doubtful the rsync command is supposed to unmount drives. I'm going to attempt this procedure on a different drive just to check. I took the earlier advice to check
    Code:
    dmesg
    but didn't see anything that stood out. Looked like the drive just unmounted itself for some strange reason. I've had other problems with this drive (although nothing like this). The housing has an eSATA port which worked fine back when I was using it on Windows but for some reason it refuses to mount from it in openSUSE. Anyway, I'll just try on a different one to ensure it's a hardware issue.

    Just read over my original post and I wish to correct a rather egregious mistake. What I meant to say is that: "I'm very new to bash so I hope I'm not making some absolutely moronic, newbie mistake." Sorry if I came off as a jerk.

    Oh, and DD - Right back at you brother.

  8. #8
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    24,853

    Default Re: RSync issue

    Just read over my original post and I wish to correct a rather egregious mistake. What I meant to say is that: "I'm very new to bash so I hope I'm not making some absolutely moronic, newbie mistake." Sorry if I came off as a jerk.
    Not at all. But being a newby at bash does not explain why your disk us unmounted (as long as you do not type umount statements )
    In fact I gave not more then a quick glance at your rsync statement, because, as said above, whatever program you run which whatever options, it would normaly only do user read/write calls, no matter if they are usefull to your goal or efficient or what.
    Henk van Velden

  9. #9

    Default Re: RSync issue

    Hi there,

    If a USB drive is mysteriously detaching, I have nearly always found it to be due to either a bad disk or crappy controller. The following tips might help.

    First, identify the UBS to ATA/SATA controller being used. You can typically see this by running lspci - usually you can find the chipset of the USB controller reported. (Another option: run: udevadm monitor (as root) prior to plugging in the device, then plug it in an examine the messages.)

    Next, see if the controller identified is listed at: Supported_USB-Devices – smartmontools

    If so, you may be able to use smartmontools to the drive smart data through the USB controller, such as:

    Code:
    #smartctrl -d sat -a /dev/sdc1

    (Or some variation thereof.) If successful, look at the reallocated sectors, and pending sectors. You will likely get a very good idea of the health of the drive from examining these attributes.

    If you can't read the smart data through the USB interface, I would take the drive out and connect it directly. Until you know for certain if the drive is good, you might just find yourself going in circles.

    Lews Therin

  10. #10
    Join Date
    Jun 2008
    Location
    Earth - Denmark
    Posts
    10,730

    Default Re: RSync issue

    On 08/14/2011 09:16 PM, nicktlloyd wrote:

    > What I meant to say is that: "I'm very new to bash
    > so *-I hope-* I'm not making some absolutely moronic, newbie mistake."
    > Sorry if I came off as a jerk.


    amazing how the mind/eye works! i accidentally misread the original to
    mean exactly what you meant it to convey (as above)..

    therefor: jerkless.

    --
    DD
    openSUSE®, the "German Engineered Automobile" of operating systems!

Page 1 of 2 12 LastLast

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •