My first script is almost there. Your comments

English Programming/Scripting
#1

Hi
in the past days I have written my first script that will run through crontab.
Please have a look, there are comments to be easier understood from everyone.

After the code part some more comments from my side.


#!/bin/bash

# The scripts should always start with the external devices unmounte.
# 1. First checks that the device is not mounted (that should be always)
# 2. Mounts the device
# 3. Copy the files
# 4. Unmounts the device
# 5. Send emails at the moment a failure appears or at the end when everything was succesful


SCRIPTNAME="BACKUPOFDATA.SH"
MOUNTPOINT=/mnt/BckpMeasurements
MAILADDRESS="hide@host.de"



doEmail() { case "$1" in
 1) REASON="Mounting the Hard Disk Failed." SUBJECT="SUCCESS: rsync (backup).";;
 2) REASON="Device alredy Mounted. Previous umounts failed?"  SUBJECT="Backup Script Failed";;
 3) REASON="Device Failed to be unmounted!" SUBJECT="Backup script failed"  ;;
 4) REASON="Rsync failed for some reasons" SUBJECT="Backup script failed" ;;
 5) REASON="Backup Finished At: Date and Time Sucessfully";; #How to concatenate Date and Time information in that string?
 *) echo "Please don't do that, Dave..." >/dev/stderr;;
esac; echo "${REASON}" | mail -r $MAILADDRESS -s "${SUBJECT}" $MAILADDRESS; }


#  mount | grep $MOUNTPOINT: returns 0 when found (mounted). returns 1 when not found (unmounted)



# 1.
#Check that the device is not mounted already
(mount | grep $MOUNTPOINT) &&  (doEmail 2; exit 1;) # That means that mount is there already and it should not. Report the error and exit

# 2. Proceed mounting for the first time.
mount $MOUNTPOINT 2> /dev/null 
(mount | grep $MOUNTPOINT) ||  (doEmail 1; exit 1;) # Mount is not there so something wrong happened. Report the error and exit

# Do the Copy of The Files

# 3.
# rsync command goes here
# if rsync failes send an email to me too

(rsync -rav -e ssh user@host:/storage/test/ $MOUNTPOINT) || (do Email 4; exit 1;)


# How to check that rsync returned succesfully
# End of Copy of the files


# 4.
# Unmount the Device
umount $MOUNTPOINT
(mount | grep $MOUNTPOINT) && (doEmail 3; exit 1;) # That means that the mount is there already and it should not. Report the error and exit 



# 5.
doEmail 5; exit 0; # Exit 0 when normally returns?????

-As you see in the code there are some questions there already that I need help to finish those
-I was executing in the past a simple script containing only one (just one) rsync command with no checks (literally one line file). This was called then through crontab (crontab entry below)

0 5,14,22 * * * /root/backup/rsyncScript.sh  >  /root/CronErrors/rsync.out    2>  /root/CronErrors/rsync.err

As you can see I was collecting the rsync errors inside the crontab entry. Should I move those in the large script snippet I gave above?

I would like to thank you in advance for your support
Alex

#2

On 03/13/2013 09:36 AM, alaios wrote:
> Please have a look, there are comments

my comment would be that it would be best if posted to the
Programming/Scripting Forum - Questions about programming, bash
scripts, perl, php, cron jobs, ruby, python, etc. which is found here
http://tinyurl.com/5v695w4

that way, it is in the forum where script gurus look for
such…(they may or may not prowl the Applications Forum)

so, suggest you click the triangular icon below and ask a mod to move
it for you…

–
dd

#3

Will be moved to Programming/sripting and is CLOSED for the moment.

#4

Moved from Applications and open again.

#5

I read it from top to bottom, thus my comments may be a bit erratic and a mix of important and minor things.

You have some comments about the flow of your script, but a general introduction about where it is made for is completely missing. It seems to be about some sort of backup, but that is only a guess.

In general. Though your way of formatting your statements is rather consistent, I suggest you rethink it. Indentation is normaly done using the Tab key (which is set at every 8th character in vi by default, I do not know which editor you use and if it has the same tabbing, but I guess that that is rather general).

SCRIPTNAME="BACKUPOFDATA.SH"

Is that realy the case? You use a rather different name in your crontab entry. When this is about the name of the script as it is called by, you can use

SCRIPTNAME="$(basename ${0})"

(mount | grep $MOUNTPOINT) &&  (doEmail 2; exit 1;)

. When you need grep only for it’s retirn code on (not) found, use the -q option. It will suppress the ouput (which will now show somewhere you do not expect it :wink: ) and it will stop on first “found” and thus have a better performance.
. There is no need to use ( and ) around these commands, { and } is sufficient. The first will create a seperate process and that is not needed. Do not forget it is best to have a blank spaces around the { and }. This will change it to

{ mount | grep $MOUNTPOINT } && { doEmail 2; exit 1; }

. But you do not need the first { and } because the return code will be that of the last in the pipe: grep.

Same about the { and ) in the rsync command. There is but one command, thus it is completley superfluous to combine this one command.

Indeed, the

exit 0

at the end is superfluous.

To me the redirections of stdout and stderr in the crontab entry is the best way to do this.
When you run the script as a test, thus not from cron, you will see the output directly.

HTH.

#6 

SCRIPTNAME="BACKUPOFDATA.SH"
MOUNTPOINT=/mnt/BckpMeasurements
MAILADDRESS="hide@host.de"

This is not consistent. Better use " as a good practise allways.

#7

I have reformatted part of your script to show you a complete different way of doing this. You decide.

doEmail() {
case "$1" in
        1)      REASON="Mounting the Hard Disk Failed."
                SUBJECT="SUCCESS: rsync (backup)."
                ;;
        2)      REASON="Device alredy Mounted. Previous umounts failed?"
                SUBJECT="Backup Script Failed"
                ;;
        3)      REASON="Device Failed to be unmounted!"
                SUBJECT="Backup script failed"
                ;;
        4)      REASON="Rsync failed for some reasons"
                SUBJECT="Backup script failed"
                ;;
        5)      REASON="Backup Finished At: $(date) Sucessfully"
                ;;
        *)      echo "Please don't do that, Dave..." >/dev/stderr
                ;;
esac
echo "${REASON}" | mail -r $MAILADDRESS -s "${SUBJECT}" $MAILADDRESS
}

And look at the way I put the date in message #5. (and use the man page of date to see how you can format different).

#8

I would like to thank everyone participating in this .
I have tried to incorporated all your comments and I will soon trying to debug it (by that I mean immitating different conditions and see how the script reacts)
The one comment I have not understood is the quote above. What the {} do?
Please have one more look and then I am ready to go

Regards
Alex

#9

From the bash man page:

A compound command is one of the following:
(list)
list is executed in a subshell environment (see COMMAND EXECUTION ENVIRONMENT below). Variable assignments and builtin commands that affect the shell’s environment do not remain in effect after the command completes. The return status is the exit status of list.
{ list; }
list is simply executed in the current shell environment. list must be terminated with a newline or semicolon. This is known as a group command. The return status is the exit status of list. Note that unlike the metacharacters ( and ), { and } are reserved words and must occur where a reserved word is permitted to be recognized. Since they do not cause a word break, they must be separated from list by whitespace or another shell metacharacter.

The second is simply to group statements (which is what you want), the first is also a grouping, but it is executed in a subshell and that is not something needed here and thus will eat resources for no gain.

#10 
#!/bin/bash

# Description. This script is for taking backups from remote server on a local hard disk on my locahost# It is executed from crontab. The hard disk used is only for this backup and thus the script
# should check that something "strange" is not happening like the hard disk was already mounted.

# The scripts should always start with the external devices unmounted.
# 1. First checks that the device is not mounted (that should be always the case)
# 2. Mounts the device
# 3. Copy the files
# 4. Unmounts the device
# 5. Send emails at the moment a failure appears or at the end when everything was succesful


SCRIPTNAME=$(basename ${0})
MOUNTPOINT="/mnt/Bckp"
MAILADDRESS="Hide@email.com"



doEmail() { case "$1" in
        1)      REASON="Mounting the Hard Disk Failed. $(echo $SCRIPTNAME)"
                SUBJECT="Backup Script Failed. "
                ;;
        2)      REASON="Device alredy Mounted. Previous umounts failed? $(echo $SCRIPTNAME)"
                SUBJECT="Backup Script Failed. "
                ;;
        3)      REASON="Device Failed to be unmounted! $(echo $SCRIPTNAME)"
                SUBJECT="Backup Script failed. "
                ;;
        4)      REASON="Rsync failed. $(echo $SCRIPTNAME)"
                SUBJECT="Backup Script failed"
                ;;
        5)      REASON="Backup Finished At $(date). $(echo $SCRIPTNAME)"
                SUBJECT="Backup Finished "
                ;;
 *) echo "Please don't do that, Dave..." >/dev/stderr;;
esac; echo "${REASON}" | mail -r $MAILADDRESS -s "${SUBJECT}" $MAILADDRESS; }


#  mount | grep $MOUNTPOINT: returns 0 when found (mounted). returns 1 when not found (unmounted)



# 1.
#Check that the device is not mounted already
 mount | grep -q $MOUNTPOINT  &&  { doEmail 2; exit 1; } # That means that mount is there already and it should not. Report the error and exit




# 2. Proceed mounting for the first time.
mount $MOUNTPOINT 2> /dev/null
 mount | grep -q $MOUNTPOINT  ||  { doEmail 1; exit 1;  } # Mount is not there so something wrong happened. Report the error and exit





# Do the Copy of The Files


# 3.
# rsync command goes here
# if rsync failes send an email to me too
 rsync -rav -e ssh hide@hide:/storage/Bckp $MOUNTPOINT    || { doEmail 4; exit 1; }
# End of Copy of the files




# 4.
# Unmount the Device
umount $MOUNTPOINT
 mount | grep -q $MOUNTPOINT  && { doEmail 3; exit 1; } # That means that the mount is there already and it should not. Report the error and exit 

# 5.
doEmail 5; exit 0; # Exit 0 when normally returns

Hi this is the more recent version of my code . It has already run succesfully over the weekend with the following

# Every Sunday at 2:30 in the morning
# 30 2 * * 0 
30 2 * * 0 /root/backup/BackupScrit.sh  > /root/CronErrors/BackupScrit.out    2>  /root/CronErrors/rBackupScrit.err

and to be sure this is also the fstab entry

LABEL=BckpMeasurements /mnt/Bckp ext4       noauto,acl,nofail     0 0
  1. Are there major mistakes still in my code and in the /etc/fstab or crontab -e entries?
  2. I have noticed that when rsync runs and kill it through bash. I still receive the message that the rsync failed. I was impressed that bash can handle the kill signal so nicely. I though that the kill signal just destroys everything at once.
  3. How the following should change if I want to have at step 3 more rsync commands one after another (so instead of the one line there, have two-three more extras below). Should that change the crontab entry redirect to the two files?

I would like to thank you in advance for your help

Regards
Alex

#11

  1. Did not check (maybe later).

  2. Do not understand as you do not realy tell what you did. You kill a process. It depends on what you kill. When your script is running and rsync is on the move, there are at least two processes involved (bash and rsync) and you do not tell which one you killed (and with what). Also you do not realy tell what you expected to happen, nor what happened instead.

  3. The redirect of stdout and stderr in he crontab is of course for the command in the crontab. Thus that is for everything that comes out of the script. Regardless of what is in the script. When you want to catch eventual output of different statements in the script to different files, you have to do that of course on the statements in the script itself.

But I still advise you to have redirects on the command in the crontab entry to catch all other output not catched by eventual redirects inside the script. When you drop the redirects in the crontab entry and there is something written to stdout/stderr, cron will put that in a mail to the crontab owner. But I am not sure you have catered for directing those mails to e.g. your KDE mailbox.

#12
  1. Thanks
  2. The script was called from crontab and I wrote in console killall rsync. I have received the email that the rsync was failed, and that means that it was correctly handled from the script
  3. I did not understand your part though

But I still advise you to have redirects on the command in the crontab entry to catch all other output not catched by eventual redirects inside the script. When you drop the redirects in the crontab entry and there is something written to stdout/stderr, cron will put that in a mail to the crontab owner. But I am not sure you have catered for directing those mails to e.g. your KDE mailbox.

#13

Of course. When rsync stops, the script carries on. And when you kill something, the return code is not 0.
Why does this amaze you? The fact that you wrote a correct script? :wink:

Maybe because I do not understand you about the changing into 3 rsync in the script.

When you have e.g. in a script:


ls -l aap
rsync .......... >rsync1out 2>rsync2er
rsync .......... >rsync2out 2>rsync3er
rsync .......... >rsync3out 2>rsync3er

Then all output of of the three rsync commando will go to their different log files.
But the output (or errors) of all other statements in the script (like the ls command here) will still go to "the terminal’

Thus when you call this script from cron, the stdout and stderr of all three rsync commandos will end up in files, but all other output will end up in a mail to the crontab owner.

Thus when you do a redirect in the crontab, that residue will also be redirected elsewhere. Mostly it will be nothing, but you never know if in two years something in the script will throw up an error.

#14

Exactly.

I will read the rest later my 4 hour sleep does not help too much now!
Alex

#15

I am back again and alive.
I wanted to add the following in the script I gave before that would allow me to download my email to my email to my hard disk

cd /home/backup-email/
python /home/backup-email/imapbackup.py -u hide -s HidingIP -e --ssl -p mypassword ||  { doEmail 6; exit 1;  } # How to redirect correctly here?
cd /root/backup

The question is how I should redirect correctly the output > and 2> from this .py script (I found it here IMAP Backup Tool - tamale.net )
should that be before the || or after the ||

Regards
Alex

#16

Before the || you will redirect output from the python command,

#17

thanks, I changed it as that.
IS not a bit stupid though these cd command I have before and after the backup command? I would prefer to give to the python script the full path of where the output would be, but the given script does not seem to support output directory as an input argument

IMAP Backup Tool - tamale.net

Regards
Alex

#18

I do not understand exactly what you mean with this, but when it is only about having a different working directory during the python command, you can try

PWD=/home/backup-email python ......

and the of course remove the two cd statements. Not 100% sure though (depends a bit on python).

In any case, you can replace the second cd command (where you go back to where you were) with

cd -
#19

On 2013-03-28 12:06, hcvv wrote:
> In any case, you can replace the second cd command (where you go back
> to where you were) with

Have a look at “pushd” and “popd”. Didn’t know that “cd -”, though.

–
Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)