Results 1 to 7 of 7

Thread: Using rsync to an vfat device

  1. #1
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Using rsync to an vfat device


    I want to sync some data files over to a vfat device (an usb stick). I'm
    not worried about keeping linux file attributes. However, every time I
    run the process all the files are transferred again, instead of skipping
    all existing files.

    Either I do something wrong or I have wrong expectations.

    My command is this:

    Code:
    rsync --stats --human-readable  --exclude incomplete/* \
    SOURCE_DIRECTORY/* \
    /media/STICK/DIRECTORY/Jazz

    And the output is like this:

    Code:
    skipping directory incomplete
    
    Number of files: 262
    Number of files transferred: 262
    Total file size: 559.81M bytes
    Total transferred file size: 559.81M bytes
    Literal data: 559.81M bytes
    Matched data: 0 bytes
    File list size: 20.21K
    File list generation time: 0.001 seconds
    File list transfer time: 0.000 seconds
    Total bytes sent: 559.91M
    Total bytes received: 4.99K
    
    sent 559.91M bytes  received 4.99K bytes  3.96M bytes/sec
    total size is 559.81M  speedup is 1.00

    I have found that if I use "--size-only" it does what I want, but that
    would skip a file with different time stamp (ie, changed).


    --
    Cheers / Saludos,

    Carlos E. R.
    (from 12.1 x86_64 "Asparagus" at Telcontar)

  2. #2
    Join Date
    Mar 2010
    Location
    Austin - Texas
    Posts
    10,140
    Blog Entries
    48

    Smile Re: Using rsync to an vfat device

    I found this example, perhaps it might help?

    rsync and vfat

    It took several tries, and a lot of poking around, but I finally have my music collection mirrored to a disk I can take around (most notably to work). The hard part was getting rsync to work right. Finally I got it working after finding a helpful article on the topic. To summarize (in less than 3 pages), I used to following 2 commands:

    Code:
    mount -t vfat -o shortname=mixed,iocharset=utf8 /dev/sda1 /mnt
    
    rsync --modify-window=1 -rtv --delete /data/mp3/ /mnt/mp3
    Now I won’t lose them, and maybe they’ll help you. The only reason for having problems is that I was using the vfat filesystem under FC3 Linux (where my custom-built audio archive exists) to make a disk I could plug in to my work laptop. Windows filesystems aren’t so great, they have problems doing mixed case and being very accurate with times. So this makes it work!
    Thank You,
    My Blog: https://forums.opensuse.org/blogs/jdmcdaniel3/

    Software efficiency halves every 18 months, thus compensating for Moore's Law

    Its James again from Austin, Texas

  3. #3
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Using rsync to an vfat device

    On 2013-02-02 03:46, jdmcdaniel3 wrote:
    >
    > I found this example, perhaps it might help?
    >
    >> RSYNC AND
    >> VFAT (\"HTTP://WWW.KYLEV.COM/2005/03/29/RSYNC-AND-VFAT/\")
    >> It took several tries, and a lot of poking around, but I finally have
    >> my music collection mirrored to a disk I can take around (most notably
    >> to work). The hard part was getting rsync to work right. Finally I got
    >> it working after 'finding a helpful article on the topic'
    >> (http://www.osnews.com/story.php?news_id=9681&page=1). To summarize (in
    >> less than 3 pages), I used to following 2 commands:
    >>
    >>>

    > Code:
    > --------------------
    > > > mount -t vfat -o shortname=mixed,iocharset=utf8 /dev/sda1 /mnt

    > >
    > > rsync --modify-window=1 -rtv --delete /data/mp3/ /mnt/mp3

    > --------------------


    --modify-window
    When comparing two timestamps, rsync treats the timestamps as
    being equal if they differ by no more than
    the modify-window value. This is normally 0 (for an exact
    match), but you may find it useful to set
    this to a larger value in some situations. In particular,
    when transferring to or from an MS Windows
    FAT filesystem (which represents times with a 2-second
    resolution), --modify-window=1 is useful (allow-
    ing times to differ by up to 1 second).


    Interesting! That will be it.

    -r, --recursive recurse into directories
    -t, --times preserve modification times
    -v, --verbose increase verbosity


    I'll try it.

    ....

    Ok, first run was slow, I thought it failed. But a second run just after
    did run in seconds, so it appears to work. I then umounted the device,
    mounted it again (automatic mount under xfce), run the copy again, and
    it was away in seconds. So yes, that is the trick, thankyou.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 12.1 x86_64 "Asparagus" at Telcontar)

  4. #4
    Join Date
    Mar 2010
    Location
    Austin - Texas
    Posts
    10,140
    Blog Entries
    48

    Smile Re: Using rsync to an vfat device

    Happy to help and to find a good example of what you wanted to do.

    Thank You,
    My Blog: https://forums.opensuse.org/blogs/jdmcdaniel3/

    Software efficiency halves every 18 months, thus compensating for Moore's Law

    Its James again from Austin, Texas

  5. #5
    josephkk NNTP User

    Default Re: Using rsync to an vfat device

    On Sat, 02 Feb 2013 01:28:06 GMT, "Carlos E. R."
    <robin_listas@no-mx.forums.opensuse.org> wrote:

    >
    >I want to sync some data files over to a vfat device (an usb stick). I'm
    >not worried about keeping linux file attributes. However, every time I
    >run the process all the files are transferred again, instead of skipping
    >all existing files.
    >
    >Either I do something wrong or I have wrong expectations.
    >
    >My command is this:
    >
    >
    Code:
    >rsync --stats --human-readable  --exclude incomplete/* \
    >        SOURCE_DIRECTORY/* \
    >        /media/STICK/DIRECTORY/Jazz
    >
    >
    >
    >And the output is like this:
    >
    >
    Code:
    >skipping directory incomplete
    >
    >Number of files: 262
    >Number of files transferred: 262
    >Total file size: 559.81M bytes
    >Total transferred file size: 559.81M bytes
    >Literal data: 559.81M bytes
    >Matched data: 0 bytes
    >File list size: 20.21K
    >File list generation time: 0.001 seconds
    >File list transfer time: 0.000 seconds
    >Total bytes sent: 559.91M
    >Total bytes received: 4.99K
    >
    >sent 559.91M bytes  received 4.99K bytes  3.96M bytes/sec
    >total size is 559.81M  speedup is 1.00
    >
    >
    >
    >I have found that if I use "--size-only" it does what I want, but that
    >would skip a file with different time stamp (ie, changed).


    I suspect that when you rsync to (or from) vfat all the file timestamps
    are set to current time. I suggest: writing to a tar, copying that to the
    memory stick, copying from memory stick to a temp directory, untaring, and
    rsyncing from that. A lot of steps but it may save some time.
    Alternatively reformat a memory stick as ext3 and not have the vfat issue.

    ?-)

  6. #6
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Using rsync to an vfat device

    On 2013-02-09 03:53, josephkk wrote:

    > I suspect that when you rsync to (or from) vfat all the file timestamps
    > are set to current time. I suggest: writing to a tar, copying that to the
    > memory stick, copying from memory stick to a temp directory, untaring, and
    > rsyncing from that. A lot of steps but it may save some time.
    > Alternatively reformat a memory stick as ext3 and not have the vfat issue.


    If you read the thread you will see that the solution was found, using
    "--modify-window=1".

    vfat is mandated by the system that will use the stick later.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 12.1 x86_64 "Asparagus" at Telcontar)

  7. #7
    josephkk NNTP User

    Default Re: Using rsync to an vfat device

    On Sat, 09 Feb 2013 13:13:05 GMT, "Carlos E. R."
    <robin_listas@no-mx.forums.opensuse.org> wrote:

    >On 2013-02-09 03:53, josephkk wrote:
    >
    >> I suspect that when you rsync to (or from) vfat all the file timestamps
    >> are set to current time. I suggest: writing to a tar, copying that tothe
    >> memory stick, copying from memory stick to a temp directory, untaring,and
    >> rsyncing from that. A lot of steps but it may save some time.
    >> Alternatively reformat a memory stick as ext3 and not have the vfat issue.

    >
    >If you read the thread you will see that the solution was found, using
    >"--modify-window=1".
    >
    >vfat is mandated by the system that will use the stick later.


    Thank you. I found that later. Far better advice than i gave as well.

    ?-)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •