Help for a rsync error

English Programming/Scripting
1

When I run the executable script

Script “backup.file.sh”

#!/bin/sh # starts

A shell script to backup an updated “a.file” from the

/home/user/project directory on the hard disk to the

/project directory on a flash drive as an exact

“a.file” copy.

The updated “a.file” will then be backed up to

another /project directory on a second flash drive,

to a /project directory on an external hard disk, and last

to a /project directory on a Windows Vista NTFS hard disk.

rsync [options] from local SOURCE …to… local DESTINATION

[options are] progress, archive, verbose, preserve Hard links

rsync -P -a -v -H /home/stephen/project/a.file /dev/sdc1 /media/usb 1/project # the first backup operation for the “a.file”

reminder chmod 755 backup.file.sh [convert to executable]

reminder [run] ./backup.file.sh

I receive the following error message:

sending incremental file list
rsync: link_stat “/media/usb” failed: No such file or directory (2)
rsync: mkdir “/home/stephen/1/project” failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(576) [receiver=3.0.4]
rsync: connection unexpectedly closed (9 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(632) [sender=3.0.4]

I ran the df command and the filesystem “/dev/sdc1” is mounted on “/media/usb 1”. When I ran the mount command it writes onscreen

/dev/sdc1 on /media/usb 1 type vfat (rw,nosuid,nodev,shortname=lower,flush,utf8,uid=1000)

The flash drive “usb 1” also has a /project directory. However, an error occurs that I have tried hard to solve.

2

First. The shebang line

#!/bin/sh

should be the very first line in a script or it will not be seen. But this will not be your problem.

Second. When you realy have white space in the directory name where your device is mounted ( usb 1 ) you should understand that white space is also a seperator in shell commands, so the command

rsync -P -a -v -H /home/stephen/project/a.file /dev/sdc1 /media/usb 1/project # the first backup operation for the "a.file"

has two fiields down there ‘/media/usb’ and ‘1/project’. This is probably not what you want. So escape that white space by e.g.

rsync -P -a -v -H /home/stephen/project/a.file /dev/sdc1 /media/usb\ 1/project # the first backup operation for the "a.file"

or

rsync -P -a -v -H /home/stephen/project/a.file /dev/sdc1 "/media/usb 1/project" # the first backup operation for the "a.file"
3

Thank you very much for your kind assistance.

For The Code changed to and ran as

#!/bin/sh

The shell script “backup.file.sh” is for backing up

an updated “a.file” from the

“/home/user/project” directory on a Linux hard disk

to an existing “/project” directory on a

First flash drive; to an existing

“/project” directory on a SECOND flash drive;

to an existing

“/project” directory on a Maxtor 3200 external HD;

and last to an also existing

“/project” directory on a Windows Vista NTFS HD

rsync [options] from local SOURCE …to… local DESTINATION

the options are progress, archive, verbose, preserve Hard links ]

a reminder chmod 755 backup.file.sh [convert to executable]

a reminder [run] ./backup.file.sh

rsync -P -a -v -H /home/stephen/project/a.file /dev/sdc1/media/usb 1/project

< the prior blank space inserted was removed >

this error message arises

sending incremental file list
rsync: change_dir “/dev/sdc1/media” failed: Not a directory (20)
rsync: change_dir#3 “/home/stephen/usb” failed: No such file or directory (2)
rsync error: errors selecting input/output files, dirs (code 3) at main.c(614) [receiver=3.0.4]
rsync: connection unexpectedly closed (9 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(632) [sender=3.0.4]

For the alternate Code ran as

#!/bin/sh

< the other comments have been omitted here for the sake of space >

rsync -P -a -v -H /home/stephen/project/a.file /dev/sdc1 "/media/usb 1/project"

< /media/usb 1/project appears all between the parentheses after a blank space >

this error message arises

sending incremental file list
rsync: mkdir “/media/usb 1/project” failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(576) [receiver=3.0.4]
rsync: connection unexpectedly closed (9 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(632) [sender=3.0.4]

Do you have any idea what might be happening?

4

At least the error message is now about a directory you name, not about something else.

I have no time atm to digg into rsync man page, but the error says that it can not make the directory “/media/usb 1/project”. This could be because “/media/usb 1” is not writable by the user doing this. Check with

ls -l "/media/usb 1"

and may be also check

ls -l /media

What would make things using the CLI much easier is refraining from having white space in directory/file names. Can you make it just usb1 instead of “usb 1”?

And I also would prefer it when you put computer text (like your script and the error messages) within CODE tags. Makes reading computer text much easier. (Select the text and use the # button above).

5

For the lines changed to and ran as

#!/sbin/sh
#
# < the prior comments in the shell script are omitted here for the sake of space >
#
rsync -P -a -v -H /home/stephen/project/a.file /dev/sdc1/media/USB_1/project
#
# < the storage device was renamed as USB_1 and >
# < the prior blank space inserted was removed >

this error message arises now

sending incremental file list

rsync: ERROR: cannot stat destination "/dev/sdc1/media/USB_1/project": Not a directory (20)
rsync error: errors selecting input/output files, dirs (code 3) at main.c(554) [receiver=3.0.4]
rsync: connection unexpectedly closed (9 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(632) [sender=3.0.4]

For the lines changed to and ran as

#!/sbin/sh
#
# < the prior comments in the shell script are omitted here for the sake of space >
#
rsync -P -a -v -H /home/stephen/project/a.file /dev/sdc1 "/media/USB_1/project" 
#
# < the storage device was renamed as USB_1 and >
# < /media/USB_1/project appears all between the parentheses after a blank space >

this error message arises now

sending incremental file list
rsync: chgrp "/media/USB_1/project/a.file" failed: Operation not permitted (1)
skipping non-regular file "sdc1"

sent 79 bytes  received 15 bytes  188.00 bytes/sec
total size is 135018  speedup is 1436.36
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1040) [sender=3.0.4]

The output for ls -l /media is

total 80
drwxrwxrwx  1 root    root 12288 2009-04-25 13:39 disk
drwxrwxrwx  1 root    root  4096 2009-04-25 07:25 Maxtor_3200
drwxr-xr-x  6 stephen root 32768 1969-12-31 19:00 USB_1
drwxr-xr-x 12 stephen root 32768 1969-12-31 19:00 USB_2

The output for ls -l “/media/disk” is

total 7119313
drwxrwxrwx 1 root root       4096 2009-02-13 08:55 project

< **NOT part of the actual Code:**
  the other directories and files on the Windows Vista NTFS HD are omitted here for the sake of space >

The output for ls -l “/media/Maxtor_3200” is

total 12
drwxrwxrwx 1 root root 4096 2009-03-06 06:16 project

< **NOT part of the actual Code:**
  the storage device was renamed as Maxtor_3200 removing the prior blank space inserted and
  the other directories and files on the external HD are omitted here for the sake of space >

The output for ls -l "/media/USB_1"
is

total 2016
drwxr-xr-x 2 stephen   root  32768 2009-04-25 20:44 project

< **NOT part of the actual Code:**
  the storage device was renamed as USB_1 removing the prior blank space inserted and
  the other directories and files on the First flash drive are omiited here for the sake of space >

The output for ls -l "/media/USB_2"
is

total 19808
drwxr-xr-x 4 stephen root    32768 2009-04-02 05:42 project

< **NOT part of the actual Code:**
  the storage device was renamed as USB_2 removing the prior blank space inserted and
  the other directories and files on the Second flash drive are omiited here for the sake of space >

I do not know if the rw (read|write) permissions are affecting the “rsync” commands. Thanks once again for your help and advice.

6

You gave a lot of versions. I will stick to the first one:

rsync -P -a -v -H /home/stephen/project/a.file /dev/sdc1/media/USB_1/project

In your first post there were two fields “/dev/sdc1” and “/media/USB 1/project”. I now see that you merged them into one field /dev/sdc1/media/USB_1/project.

As I said earlier I did not try to see what your rsync command should do (by reading its man page). I simply said to myself that you forgot about the white space. But now I see you are meaning all the time to do something to /dev/sdc1/media/USB_1/project. That can not be true!

/dev/sdc1 is a device special file and not a directory at all! So this /dev/sdc1/media/USB_1/project can not exist nor being created!

You are talking about a flash device. Is it possible that this device belongs to the device special file /dev/sdc1 and that it is MOUNTED at /media/USB_1? You can check this with the command

mount

When this is the case you can create/read/write directories/files on the flash device by using /media/USB_1/ and NOT what you are doing.

7

And to add a comment to your

I do not know if the rw (read|write) permissions are affecting the “rsync” commands.

Permissions are a property of the file and are used by the kernel to do what has to be done. This has nothing to do with the command that you may happen to use. When a door is closed nobody can get in.

8

When I wrote the rsync commands I assumed incorrectly that it was necessary to indicate the PATH through the “/dev/sdx # ” fields, and this was causing the error messages posted. All of the storage devices were already mounted: this confusion arose because in the several researches that I made I could not find a proper illustration or example for the general expression here](rsync Man Page - Linux - SS64.com) as:

# Local file to Local file
  rsync [options] … Source [Source] … [Destination]

The several ls -l “/media/device” searches were to indicate that on all of the storage devices there was an existing “/project” directory that I was attempting to write to. After reflecting on your keen insight, for which I am very grateful, I rewrote the shell script in this manner:

#!/bin/sh
#
# [the prior comments are omitted for the sake of space]
#
rsync -P -a -v -H /home/stephen/project/a.file /media/USB_1/project
rsync -P -a -v -H /home/stephen/project/a.file /media/USB_2/project
rsync -P -a -v -H /home/stephen/project/a.file /media/Maxtor_3200/project
#

and the desired backup operations are working perfectly well. I dropped the rsync command for the backup to the “/project” directory on the Windows Vista HD since there are blank spaces on a User’s name that makes that backup operation to fail entirely.

Since one can access a NTFS in Linux/Unix I will have to do that backup operation manually by saving the updated “a.file” to the “/media/disk/project” directory. In spite of this minor drawback the automated backups performed by the rsync command do help one to save a lot of time! It would be interesting to set up a cron job. Could you refer me a help web page on that matter?

Thanks once again.

9

I am glad I could be of any help to you. May I advise you a few things?

  1. Try to understand what device special files (short: device files) like /dev/sdc1 are. Try to understand what mounting means.

  2. Never write directly to device files. As a normal user you will normaly be blocked from doing so, but as root you will destroy the filesystem on it and loose all data.

Also do not only read all sorts of howto’s or other useful or fatal stories on the web, but at least check them by reading the man pages of the tools involved.

BTW you not only read man pages by doing in a console/terminal:

man rsync

but you can also use Konqueror typing man:rsync in the address field.

And about cron this is the same: first read the man crontab (both sections 1 and 5), then try a few harmless things to get the feeling. When questions stil arise come to the forums here. People on the forum are willing to help you, but not willing to tell you what you can find in the man page

Basicaly you do

crontab -e

to edit your crontab (it will probably use vi). Enter a new line by going to the end and typing o (the small letter o). The line will be something as:

10 17 * * *   /the/absolute/path/to/your/script >/tmp/scriptlog 2>&1

to start the script every day at 10 past 5 in the afternoon. This will also redirect all output to stdout and stderr to the file /tmp/scriptlog (overwriting it every day). Change to your needs. To end the vi-editing, press Esc (to return from input mode to command mode) and type :wq
Wait until 17:10 and see!

10

I edited a crontab for the backup script, and it is indeed amazing to see some of the things that one can do in linux!

It is also very encouraging to know that the web forums are an invaluable resource not only to find help when needed, but also to understand better the unix man pages that are at times very hard to grasp on a first reading.

At the same I found an effective way to overcome the Windows drawback due to the blank spaces in the “/media/disk/User/project” field by moving the “/project” directory and placing it in another place --a “/media/disk/Downloads” field that I have on the Windows Vista NTFS install.

rsync -P -a -v -H /home/stephen/project/a.file /media/disk/Downloads/project

I trust that this feedback may be useful to help other persons that might face a similar situation as mine.

11

Yes, this is so, and it’s not your fault. Nor is it the fault of the man page authors. Man pages were/are not intended to be tutorials, that is the rôle of, um, tutorials. Rather, they are designed to be concise references and quick reminders of what the program does, what the input and output are and what options are available. A man page is what you reach for when you think: what was that option to do blah-de-blah and what parameter do I give it? Some man pages are more chatty than others but most of them are concise references. So feel free to ask away on forums if the man page is not sufficient for your question.

12

Hello sfzab,

Nice that you got working what you wanted.

Indeed as ken_yap says: sometimes a man page can tell you that the -z option does something, but in the whole page you can not find what the program is doing!

And about the white space.
It is nice you found a way to avoid them. That makes a lot of things easier. There are ways to work with them (when in a GUI program you have a “Save as…” dialogue you can specify filenames with white space in them after all), even in scripts, but that requires an in depth knowledge of quoting.

Enjoy Linux and do not hesitate to come back for questions.

And as a last remark: because you did not quite understand a few things about device files and mountpoints, maybe this primer is interesting: SDB:Basics of partitions, filesystems, mount points - openSUSE . In fact your thread was one of the reasons I wrote this one (published yesterday). I welcome any comments you, as a Linux beginner, may have reading this.

13

#!/bin/sh
###############################################################################
#syncfiles.sh
#Replicate file trees to server using rsync
#usage:

sh syncfiles.sh

#or call from cron (make sure ssh key is loaded beforehand)
#Requirements:

Local user name must match remote (user) paramita

###############################################################################
Host=$(usr)
User=$(ip)
Keychain $/home/icts/Desktop/.ssh/id_rsa
.$HOME/.keychain/$(paramita)-sh
Excludes=
cd home
#rsync -e ssh -av --delete --delete-excluded
rsync -e ssh -a --delete --delete-excluded
–exclude “tmp”
–exclude “[cC]ache”
$Excludes $ $@:/home/paramita/

Here this is the script…whenever i run that script at client machine…it gives error message “cannot cd to home” and then it asks for password…whenever i given that password that it gave error message " permission denied"…what should i do…please advise as earliest…

14

On 2012-01-30 08:36, paramitadasnic wrote:
>
> #!/bin/sh

…

> Here this is the script…whenever i run that script at client
> machine…it gives error message “cannot cd to home” and then it asks
> for password…whenever i given that password that it gave error message
> " permission denied"…what should i do…please advise as earliest…

Hello, newcommer.

I see this is your first post. Are you sure your problem is related to this
years-old thread? IMO, you should post a new post with all relevant info
included and a proper title.

Then, I can not examine your script because lines are wrapped. Please
include code inside text tags - advanced editor, # button.

–
Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

15

Hi,

I wonder whether you shouldn’t cd to /home instead of home?

HTH

Lenwolf

16

On 02/06/2012 07:46 AM, lenwolf wrote:
> I wonder whether you shouldn’t cd to /home instead of home?

nice hint.

–
DD
Read what Distro Watch writes: http://tinyurl.com/SUSEonDW

17

Please start a new thread instead of reusing this old one.