Problems with LuckyBackup and pCloud using appimage among others!

English Applications
#1

Running KDE desktop and opensuse repo version of LuckyBackup I have been trying to create a new backup of selected files and directories from my ~/home directory. I have been using the Advanced option when creating my task properties and have tried two different approaches but am reporting this which is my first and best version in which I have been using the Include tab which greys out the exclude tab.

Here is my command for the Include option:-

rsync -h --progress --stats -r -tgo -p --update --include=/Documents/** --include=/scanned_docs/ --include=*/ --exclude=* --prune-empty-dirs --protect-args /home/alastair/ admin@192.168.169.131:admin@192.168.169.131:/share/ibmserver2/

This should not have included my ~/pCloudDrive subdirectory but even so many files from pCloudDrive are being shown but failing and also my NFS shared file is showing even though I thought NFS shares are supposed to be excluded.

I already have many thousands of files in my Mozilla profiles which I do need to back up and are in hidden directories which are to be backed up. I do not however wish the NFS and pCloud to be included as these would vastly increase storage and also result in duplication which I do not want.

How can I sort this out please?

#2

So you try to use rsync inside of LuckyBackup? Why and how? And how is appimage related to this problem as it is missing in your description? This is kind of a confusing description…

#3

I do not understand how pCliud or appimage are relevant here to be put into the topic subject. And you managed to omit the most important information - that you used rsync. As for your actual problem

The first rule matches every directory (recursively) so it will also match your ~/pCloudDrive subdirectory. The first matching rule wins so the exclude rule will have no effect for directories.

#4

Which means

/home/budgie2/home/

Rather confusing.

#5

Hi and sorry if my explanation was poor. I have been using Luckybackup as an Rsync gui because I use it regularly. I am now trying to do something new and hoped it would be easier for me. Also I like the “Dry run” facility because it easier to check to prevent, I hope, making major mistakes.

I mentioned app image and pCloud because although I have not included these directories in my intended backup, when the Dry run is running and there is a stream of date appearing in the Output Data window, the data includes lines such as :-

rsync: [sender] readdir("/home/alastair/pCloudDrive/Crypto Folder"): Permission denied (13)

Similarly there are very many lines referring to the directory:-

/home.alastair/Temp_NAS_share

This is an NFS share and I had mistakenly understood these shares would not be included by default.

As has been pointed out in subsequent posts I have some errors and that is why I am posting here.
I would be happy to get some help but had hoped I could get LuckyBackup working correctly because it has several convenient and time saving features.

#6

Hi and thanks,
Yes I can see the problem but I am not sure why. I suspect my problem was in trying to include not only the directories specified but also the hidden files in /home/alastair/ when from within LuckyBackup. How may I include only these hidden files and the wanted directories please?

#7

I should have written ~/ or home. I was trying to differentiate between the /home directory and my user’s home directory. My mistake.

#8

This is an NFS mount (not an NFS share). And the meaning of mounting a file system somewhere is to give it a place within the one and only directory tree (starting at /). The fact that a directory or file is on a file system of some type, being it Btrfs, ext4, NFS, zfs, …) is of no importance. Forget it as long as you are just doing on the system what need to be done, specially as end-user. When you understand this, you would not even have thought about the possibility that a file on an NFS file system would be treated different from a file on another file system type.

#9

OK, I now understand and yes it is an NFS mount and I was sloppy with my terminology.
Going back to my problem with LuckyBackup, I have been using the Advanced option and had been using the Task Properties>Advanced>Include tab and had been using the Only Include Mode button. I then defined what I wish to include in a list and at present this has :-

/home/alastair/.*
/home.alastair/Documents/
/home/alastair/scanned_docs/

Even so the script has what to me appears to be an extra term. Here is the script now:-

rsync -h --progress --stats -r -tgo -p --update --include=/home/alastair/.* --include=/home/alastair/Documents/ --include=home/alastair/scanned_docs/ --include=*/ --exclude=* --prune-empty-dirs --protect-args /home/alastair admin@192.168.169.131:admin@192.168.169.131:/share/ibmserver2/

I cannot see where the --include=*/ comes from or why but I think this is not wanted.

If I change Task Properties>Advanced>Include tab to use the Normal Include Mode, I get a script which seems better:-

rsync -h --progress --stats -r -tgo -p --update --include=/home/alastair/.* --include=/home/alastair/Documents/ --include=home/alastair/scanned_docs/ --exclude=**/*tmp*/ --exclude=**/*cache*/ --exclude=**/*Cache*/ --exclude=**~ --exclude=/mnt/*/** --exclude=/media/*/** --exclude=**/lost+found*/ --exclude=/var/run/** --exclude=/run/** --exclude=/proc/** --exclude=/dev/** --exclude=/sys/** --protect-args /home/alastair admin@192.168.169.131:admin@192.168.169.131:/share/ibmserver2/

This makes sense to me although it is a long way to go and I am still trying to understand it. It is still not working and still giving me for example:-

alastair/pCloudDrive/pCloud Backup/IBMW530/Documents/x3400_7976_kmg/ibm_utl_bomc_2.31_sles11_x86-64.bin

I shall therefore forget LuckyBackup for now and go straight to Rsync on CLI but my problem will be the implications for my NAS if I mess up.

#10

Please be patient as I am still trying to understand why I am having a problem with LuckyBackup (LB). I have read and have a hard copy of the Rsync man page document, 80 odd pages of it and I am starting on again using my home HP Z640 workstation which does not have so much critical data in my home directory.

Using LB I have created a task as follows:-

In the Excluded tab I have ticked all 8 of the Templates (Temporary folders, Backup files, Caches folders, Trash, System mount folders, System folders, lost+found, .gvfs). I do not have any user defined exclusions set.

In the Include tab I have two directories as Normal Include: /alastair/Documents/ and /alastair/Downloads/
I created these using the pattern selector button for Specific directory and local browser button to select these two directories and added them to my Include list.

In the Remote tab I have ticked Use remote host and set the user destination User and @Host. I have selected for 2 snapshots and allowed the creation of extra directories. I shall need help with Remote module and ssh but have not used these settings at present.

In the Command options tab I have ticked Preserved; ownership times, permissions, symlinks, device special files.
I have omitted hard links and numeric group and user IDs.

I have ticked the Delete files on destination, Recurse into directories and Skip newer destination files.

I have omitted Ignore files the CVS way, Destination is FAT/NTFS and Attempt super-user activities.

The cli command which this setup gives me is the following command:-

rsync -h --progress --stats -r -tgo -p -l -D --update --delete-after --include=/alastair/Documents/ --include=/alastair/Downloads/ --delete-excluded --exclude=**/*tmp*/ --exclude=**/*cache*/ --exclude=**/*Cache*/ --exclude=**~ --exclude=/mnt/*/** --exclude=/media/*/** --exclude=**/lost+found*/ --exclude=/var/run/** --exclude=/run/** --exclude=/proc/** --exclude=/dev/** --exclude=/sys/** --exclude=**/*Trash*/ --exclude=**/*trash*/ --exclude=**/.gvfs/ --protect-args /home/alastair admin@192.168.169.131:admin@192.168.169.131:/share/HP-Z640-1/

I have been working through these options, referring to the man page and am still trying to work out if this is what I might have set up if I had worked directly in terminal.

I have also added -n for dry run.

What I have not yet worked out the logic of this command but the result is that the command is including all the directories and hidden files in my home directory. I am showing some excerpts of the output below.

This is the initial stream of files showing the start and the first few of my ~.files as wanted:-

alastair@HP-Z640-1:~> rsync -h --progress --stats -r -tgo -n -p -l -D --update --delete-after --include=/alastair/Documents/ --include=/alastair/Downloads/ --delete-excluded --exclude=**/*tmp*/ --exclude=**/*cache*/ --exclude=**/*Cache*/ --exclude=**~ --exclude=/mnt/*/** --exclude=/media/*/** --exclude=**/lost+found*/ --exclude=/var/run/** --exclude=/run/** --exclude=/proc/** --exclude=/dev/** --exclude=/sys/** --exclude=**/*Trash*/ --exclude=**/*trash*/ --exclude=**/.gvfs/ --protect-args /home/alastair admin@192.168.169.131:admin@192.168.169.131:/share/HP-Z640-1/
admin@192.168.169.131's password: 
building file list ... 
 96700 files...
rsync: [sender] readdir("/home/alastair/pCloudDrive/Crypto Folder"): Permission denied (13)
221402 files to consider
alastair/
alastair/.bash_history
alastair/.bashrc
alastair/.emacs
alastair/.gtkrc-2.0
alastair/.i18n
alastair/.inputrc
alastair/.lesshst
alastair/.profile
alastair/.wget-hsts
alastair/.xdm-global-lock
alastair/.xim.template
alastair/.xinitrc.template
alastair/.config/
alastair/.config/QtProject.conf
alastair/.config/Trolltech.conf
alastair/.config/akonadi-firstrunrc
alastair/.config/akonadi-migrationrc

After the hidden files the stream starts at the top of my home directory and includes Directories I was trying to exclude:-

alastair/.thunderbird/tjyvwksv.default-release/storage/permanent/indexeddb+++fx-devtools/.metadata-v2
alastair/.thunderbird/tjyvwksv.default-release/storage/permanent/indexeddb+++fx-devtools/idb/
alastair/.thunderbird/tjyvwksv.default-release/storage/permanent/indexeddb+++fx-devtools/idb/478967115deegvatroootlss--cans.sqlite
alastair/.thunderbird/tjyvwksv.default-release/storage/permanent/indexeddb+++fx-devtools/idb/478967115deegvatroootlss--cans.files/
alastair/.thunderbird/tjyvwksv.default-release/storage/temporary/
alastair/AppImage_Apps/
alastair/AppImage_Apps/AppImageUpdate-x86_64.AppImage
alastair/AppImage_Apps/appimaged-655-x86_64.AppImage
alastair/AppImage_Apps/appimagelauncher-lite-2.2.0-travis995-0f91801-x86_64(1).AppImage
alastair/AppImage_Apps/Kdenlive/
alastair/AppImage_Apps/Kdenlive/kdenlive-22.12.3-x86_64.AppImage

It is clear there is a conflict which may be the result of the order in which the list is created or sorted but I cannot yet see it. Any help would be much appreciated.
Alastair.

#11

Of course it does. The command recursively copies /home/alastair and there are no rules to exclude anything except some global patterns like **/*cache*/ or **/.gvfs/. The --include arguments are redundant because these directories are already included by default anyway.

You provided long description of what you clicked but you never explained what you intended to achieve with these clicks. What result did you expect?

#12

Many thanks. I am trying to backup only my Documents and Download directories and and the hidden files in my home, ie /home/alastair/ directory.

I have been persevering with LB but clearly I am not using it correctly.

I can see the problem on the command line. I understand that if I am recursing into /home/alastair I will get everything and what I want is to recurse only into the selected directories but include the hidden files. I just cannot yet see how to do this.

I shall keep working on it this evening.

#13

For a start, “hidden files” is something that does not exist in Linux/UNIX. If you mean file names starting with dot, the next obvious question - in which (sub-)directory these files should be. Are they really files or they are also directories.

Computers do not read your mind. You need to tell them exactly what you want. If you cannot describe what you want using normal human language, you will not be able to give computer formal rules to do it.

rsync -r --include /Documents --include /Documents/**  --include /Downloads --include /Downloads/** --include '/.*' --include '/.*/**' --exclude '*' /home/alastair/ ...

You will likely want to exclude such “hidden files” as ~/.cache or ~/.gvfs.

#14

Understood. I am referring to the dot files in my home directory and do wish to exclude ~/.cache etc.

To explain why, I use these files occasionally if I have to rebuild a system or do a new install to speed up configuration of apps.

The main need for the backup is for the important subdirectories in my home directory and I am still learning how best to use this given the various storage requirements which are spread over different devices and clouds and the quantity of data involved.

Having made a complete mess earlier I had started anew on a simple system and thank you for your straight forward and simple cli. I shall try to work out why the LB tool is not giving me what I want.

#15

Quoting your previous post.

If you tell your backup software to backup $HOME and do not exclude anything inside of $HOME then why you surprised you got everything inside of $HOME?

You can either list each directory you want to backup or you need to explicitly exclude everything you do not want. You probably misunderstand what --include option does. It is only useful to provide exceptions to more general --exclude rules. It does not mean "only include these files/directories in backup`.

#16

At last I am beginning to understand. I had been using the --include option incorrectly so thanks for the pointer on this.

I am now using only the exclude option and have not yet used any include options.

I have also learned that it is essential to use the browse>select button consistently so that the excluded directories have the required path. Using the tool to create exclusions it is easy to create an incorrect exclusion.

I do not yet understand how the snapshots option works and how they appear in the destination. I have set the number at three and need to how these are rolled around if I make backups on a schedule. Are the snapshots in three discrete subdirectories? I ask because this will have serious implications on how I set the --delete option and using a recovery if required.

Thank you again for all your help.