Results 1 to 2 of 2

Thread: rsync and data transfer integrity

  1. #1
    Join Date
    Feb 2010
    Location
    nh,usa
    Posts
    156

    Default rsync and data transfer integrity

    Leap 15 and KDE

    For this linux machine I use 3 disks, 1 for the OS, 1 for my-data and 1 for a copy-of-my-data.

    As super user, I run an rsync script to copy my data from my-data-disk to the copy-of-my-data-disk.

    The script is here, comments and suggestions are welcome.

    Code:
      
    
     rsync --archive --checksum  --checksum-choice='md5,md5'  --verbose --delete   \
          --progress  --stats  --whole-file   --include='*.*'                     \
          --log-file='/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/Eight_Disk_Copy_Log.txt'    \
          /run/media/david/776ef86b--2c8023c2581c/david/___David    \
          /run/media/david/98b9e487--edf39e211b93/david

    After the script runs, I do a right click on the top directory of each disk and select properties.

    The number of directories and number of files is exact, but the amount of disk space is different.

    Each disk is 1.7 Tb, with 1.2 Tb of data, the copy disk has 50 Mb less data.

    Next I run an ls + diff script which is here, again comments and suggestions are welcome.

    Code:
      
    
    
    DIR1=/run/media/david/776ef86b--2c8023c2581c/david
    DIR2=/run/media/david/98b9e487--edf39e211b93/david
    DIR3=/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/
    
    FILE1=/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/dir1_ls.txt
    FILE2=/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/dir2_ls.txt
    FILE3=/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/dir_ls_diff.txt
    
    cd $DIR1
    ls  --almost-all  -Xl  --recursive  ___David  &>$FILE1
    cd $DIR2
    ls  --almost-all  -Xl  --recursive  ___David  &>$FILE2
    cd $DIR3
    diff  $FILE1  $FILE2  &>$FILE3
    The ls text files are ~70Mb, and the diff text file is ~ 60Kb.

    Looking at all this, I suspect some of the files are of different size.

    In the rsync script, I request an MD5sum check of the files.

    Q1) Should I request a different checksum?

    Q2) Am I doing this right, is my strategy OK?

    As a side note, the rsync script always fails if the copy disk is empty. It just stops along the way. Rerunning might get a little further.

    Instead, I first copy and paste using the mouse and dolphin, then I run the rsync script, which runs smoothly. After the first copy, every rsync script runs fines.

    Q3) How can I be 100% confident of the data transfer integrity.

    Thank you.
    Regards,

    RIH5342

  2. #2
    Join Date
    Sep 2012
    Posts
    5,201

    Default Re: rsync and data transfer integrity

    Quote Originally Posted by rih5342 View Post
    Each disk is 1.7 Tb, with 1.2 Tb of data, the copy disk has 50 Mb less data.
    Your source medium contains rsync log which is enough to account for this difference.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •