rsync and data transfer integrity

Leap 15 and KDE

For this linux machine I use 3 disks, 1 for the OS, 1 for my-data and 1 for a copy-of-my-data.

As super user, I run an rsync script to copy my data from my-data-disk to the copy-of-my-data-disk.

The script is here, comments and suggestions are welcome.

  

 rsync --archive --checksum  --checksum-choice='md5,md5'  --verbose --delete   \
      --progress  --stats  --whole-file   --include='*.*'                     \
      --log-file='/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/Eight_Disk_Copy_Log.txt'    \
      /run/media/david/776ef86b--2c8023c2581c/david/___David    \
      /run/media/david/98b9e487--edf39e211b93/david


After the script runs, I do a right click on the top directory of each disk and select properties.

The number of directories and number of files is exact, but the amount of disk space is different.

Each disk is 1.7 Tb, with 1.2 Tb of data, the copy disk has 50 Mb less data.

Next I run an ls + diff script which is here, again comments and suggestions are welcome.

  


DIR1=/run/media/david/776ef86b--2c8023c2581c/david
DIR2=/run/media/david/98b9e487--edf39e211b93/david
DIR3=/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/

FILE1=/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/dir1_ls.txt
FILE2=/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/dir2_ls.txt
FILE3=/run/media/david/776ef86b--2c8023c2581c/david/___David/___Eight_Now/shell_scripts/file_compare/dir_ls_diff.txt

cd $DIR1
ls  --almost-all  -Xl  --recursive  ___David  &>$FILE1
cd $DIR2
ls  --almost-all  -Xl  --recursive  ___David  &>$FILE2
cd $DIR3
diff  $FILE1  $FILE2  &>$FILE3 


The ls text files are ~70Mb, and the diff text file is ~ 60Kb.

Looking at all this, I suspect some of the files are of different size.

In the rsync script, I request an MD5sum check of the files.

Q1) Should I request a different checksum?

Q2) Am I doing this right, is my strategy OK?

As a side note, the rsync script always fails if the copy disk is empty. It just stops along the way. Rerunning might get a little further.

Instead, I first copy and paste using the mouse and dolphin, then I run the rsync script, which runs smoothly. After the first copy, every rsync script runs fines.

Q3) How can I be 100% confident of the data transfer integrity.

Thank you.

Your source medium contains rsync log which is enough to account for this difference.