Results 1 to 8 of 8

Thread: [CRON] Mount failing in backup script when run from cron

  1. #1

    Default [CRON] Mount failing in backup script when run from cron

    Hi all,

    So I've assembled a script to snapshot a Logical Volume and rsync this to a remote host running rsyncd. The script works fine when run directly as root on the command line. However when scheduled from cron it fails to execute the mount on line 53. Any tips on how I can troubleshoot why the mount is failing?

    Code:
    #!/bin/sh +x
    
    WORKVG="VG_VMS01_SYS"
    SOURCELV="LV_ROOT"
    SNAPLV="LVROOT-SNAP1"
    SNAPSIZE="10G"
    MOUNTOPTS="" # << if FS == XFS -onouuid,ro
    REMOTEHOST="NAS"
    REMOTEDIR="BACKUP_VMS01/rootfs"
    LOGFILE="/var/log/$WORKVG-$SOURCELV-backup.log"
    
    # Make sure only root can run our script
            if [ "$(id -u)" != "0" ]; then
                    echo "This script must be run as root" 1>&2
                    exit 1
            fi
    
    # write first parameter $1 to logfile
    function write_to_log 
    {
      # get current date and time
      bkp_date=$(date +%Y-%m-%d@%H:%M:%S)
      echo -e "$bkp_date : $1" >> $LOGFILE
    }
    
    # Append log entry
    write_to_log "====== Starting Backup ======"
    
    # Check if SNAPLV exists
            if [[ $(lvs | grep $SNAPLV) ]]; then
                    echo "$SNAPLV already exists! Aborting" 1>&2
                    exit 1
            else
                    lvcreate -L $SNAPSIZE -s /dev/$WORKVG/$SOURCELV -n $SNAPLV >> $LOGFILE
            fi
    
    # Make sure $DIRECTORY exists, if not then create directory!
            if [ -d /mnt/$SNAPLV ]
            then
                    echo "Directory $SNAPLV exists." >> $LOGFILE
            else
                    mkdir /mnt/$SNAPLV
            fi
    
    # check whether mount point is in use
        if [ ! -z "$(mount -l | grep $SNAPLV)" ]; then 
              write_to_log "[X] Error: Mount point already in use"
              write_to_log "====== Backup failed ======"
              exit 1
        fi
    
    # Mount SNAPVOL
    /usr/bin/mount --verbose --read-only $MOUNTOPTS /dev/$WORKVG/$SNAPLV /mnt/$SNAPLV >> $LOGFILE 
        if [ $? -ne 0 ]; then
            rmdir /mnt/$SNAPLV
            lvremove -f /dev/$WORKVG/$SNAPLV >> $LOGFILE
              write_to_log "[X] Error: Could not mount /dev/$WORKVG/$SNAPLV to /mnt/$SNAPLV"
              write_to_log "====== Backup failed ======"
              exit 1
        fi
    write_to_log "Starting rsync of $WORKVG/$SNAPVOL to $REMOTEHOST::$REMOTEDIR"
    rsync --archive --delete-before /mnt/$SNAPLV/ $REMOTEHOST::$REMOTEDIR 
    write_to_log "Unmounting /mnt/$SNAPLV"
    umount /mnt/$SNAPLV
    write_to_log "Removing temp dir /mnt/$SNAPLV"
    rmdir /mnt/$SNAPLV
    lvremove -f /dev/$WORKVG/$SNAPLV >> $LOGFILE
    write_to_log "Backup of $WORKVG/$SNAPLV to $REMOTEHOST::$REMOTEDIR finished succesfully"
              write_to_log "====== Backup finished ======"

  2. #2
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    24,853

    Default Re: [CRON] Mount failing in backup script when run from cron

    What do you mean with "it fails to execute"?
    When I take this as you say it, it would mean imo that bash skips the line or says something like "can not execute".

    When you mean different, e.g. that it executes, but that it gives different result from what you think it should do, please at least show the output (you have --verbose). And showing what the real statement is, after the shell has done all it's replacments,, etc. might help also. e.g. use
    Code:
    set -x
    and
    Code:
    set +x
    at appropriate places.
    Henk van Velden

  3. #3

    Cool Re: [CRON] Mount failing in backup script when run from cron

    Below you can see the output of the log file. The first run (00:05:55) is directly from the shell run as user root. In this output I can clearly see that mount has executed the command as it states that the volume is mounted and the script then continue's running the rsync.

    Second run (00:08:01) seems to fail on the line where it should mount the snapshot. I also noticed however that it did not log the creation of the snapshot so maybe this is where the issue lies?

    Code:
    2016-01-02@00:05:55 : ====== Starting Backup ======
      Logical volume "LVROOT-SNAP1" created.
    mount: /dev/mapper/VG_VMS01_SYS-LVROOT--SNAP1 mounted on /mnt/LVROOT-SNAP1.
    2016-01-02@00:05:57 : Starting rsync of VG_VMS01_SYS/ to NAS::BACKUP_VMS01/rootfs
    2016-01-02@00:06:43 : Unmounting /mnt/LVROOT-SNAP1
    2016-01-02@00:06:44 : Removing temp dir /mnt/LVROOT-SNAP1
      Logical volume "LVROOT-SNAP1" successfully removed
    2016-01-02@00:06:44 : Backup of VG_VMS01_SYS/LVROOT-SNAP1 to NAS::BACKUP_VMS01/rootfs finished succesfully
    2016-01-02@00:06:44 : ====== Backup finished ======
    2016-01-02@00:08:01 : ====== Starting Backup ======
    2016-01-02@00:08:01 : [X] Error: Could not mount /dev/VG_VMS01_SYS/LVROOT-SNAP1 to /mnt/LVROOT-SNAP1
    2016-01-02@00:08:01 : ====== Backup failed ======
    After using absolute paths for all commands in the script it now works as a charm from cron!

    Code:
    #!/bin/sh +x
    
    WORKVG="VG_VMS01_SYS"
    SOURCELV="LV_ROOT"
    SNAPLV="LVROOT-SNAP1"
    SNAPSIZE="10G"
    MOUNTOPTS="" # << if FS == XFS -onouuid,ro
    REMOTEHOST="NAS"
    REMOTEDIR="BACKUP_VMS01/rootfs"
    LOGFILE="/var/log/$WORKVG-$SOURCELV-backup.log"
    
    # Make sure only root can run our script
            if [ "$(id -u)" != "0" ]; then
                    echo "This script must be run as root" 1>&2
                    exit 1
            fi
    
    # write first parameter $1 to logfile
    function write_to_log 
    {
      # get current date and time
      bkp_date=$(date +%Y-%m-%d@%H:%M:%S)
      echo -e "$bkp_date : $1" >> $LOGFILE
    }
    
    # Append log entry
    write_to_log "====== Starting Backup ======"
    
    # Check if SNAPLV exists
            if [[ $(lvs | grep $SNAPLV) ]]; then
                    echo "$SNAPLV already exists! Aborting" 1>&2
                    exit 1
            else
                    /sbin/lvcreate -L $SNAPSIZE -s /dev/$WORKVG/$SOURCELV -n $SNAPLV >> $LOGFILE
            fi
    
    # Make sure $DIRECTORY exists, if not then create directory!
            if [ -d /mnt/$SNAPLV ]
            then
                    echo "Directory $SNAPLV exists." >> $LOGFILE
            else
                    /usr/bin/mkdir /mnt/$SNAPLV
            fi
    
    # check whether mount point is in use
        if [ ! -z "$(mount -l | grep $SNAPLV)" ]; then 
              write_to_log "[X] Error: Mount point already in use"
              write_to_log "====== Backup failed ======"
              exit 1
        fi
    
    # Mount SNAPVOL
    /usr/bin/mount --verbose --read-only $MOUNTOPTS /dev/$WORKVG/$SNAPLV /mnt/$SNAPLV >> $LOGFILE 
        if [ $? -ne 0 ]; then
            rmdir /mnt/$SNAPLV
            /sbin/lvremove -f /dev/$WORKVG/$SNAPLV >> $LOGFILE
              write_to_log "[X] Error: Could not mount /dev/$WORKVG/$SNAPLV to /mnt/$SNAPLV"
              write_to_log "====== Backup failed ======"
              exit 1
        fi
    write_to_log "Starting rsync of $WORKVG/$SNAPVOL to $REMOTEHOST::$REMOTEDIR"
    /usr/bin/rsync --archive --delete-before /mnt/$SNAPLV/ $REMOTEHOST::$REMOTEDIR 
    write_to_log "Unmounting /mnt/$SNAPLV"
    /usr/bin/umount /mnt/$SNAPLV
    write_to_log "Removing temp dir /mnt/$SNAPLV"
    /usr/bin/rmdir /mnt/$SNAPLV
    /sbin/lvremove -f /dev/$WORKVG/$SNAPLV >> $LOGFILE
    write_to_log "Backup of $WORKVG/$SNAPLV to $REMOTEHOST::$REMOTEDIR finished succesfully"
              write_to_log "====== Backup finished ======"
    Code:
    2016-01-02@13:11:01 : ====== Starting Backup ======
      Logical volume "LVROOT-SNAP1" created.
    mount: /dev/mapper/VG_VMS01_SYS-LVROOT--SNAP1 mounted on /mnt/LVROOT-SNAP1.
    2016-01-02@13:11:03 : Starting rsync of VG_VMS01_SYS/ to NAS::BACKUP_VMS01/rootfs
    2016-01-02@13:11:32 : Unmounting /mnt/LVROOT-SNAP1
    2016-01-02@13:11:33 : Removing temp dir /mnt/LVROOT-SNAP1
      Logical volume "LVROOT-SNAP1" successfully removed
    2016-01-02@13:11:33 : Backup of VG_VMS01_SYS/LVROOT-SNAP1 to NAS::BACKUP_VMS01/rootfs finished succesfully
    2016-01-02@13:11:33 : ====== Backup finished ======
    Thank you for taking the time to reply. It's much appreciated and helped me realize where my problem was! Bedankt daarvoor ;-)

    Follow up question. Would it be wise to source profile so there would not be a need to provide absolute paths in the script or is that not advised?

  4. #4
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    24,853

    Default Re: [CRON] Mount failing in backup script when run from cron

    Better use the absolute pathes, the more because you now have them complete.

    Then, but just see this a s my hobyhorse and you might wish not to change the functioning script, a few remarks.
    =====================
    Why using /bin/sh instead of /bin/bash. Most people on Linux write shell scripts in bash. You are of course correct if you program in the POSIX shell. Differences are not great, but there are differences.
    ========================
    Inconsistency in using if [ .... ] and if [[ .... ]] contstructs. Try to standardize upon one of them (I prefer the if [[ ... ]] one).
    =====================
    Instead of
    Code:
    if [[ $(lvs | grep $SNAPLV) ]]; then
                    echo "$SNAPLV already exists! Aborting" 1>&2
                    exit 1
            else
    I would use
    Code:
    lvs | grep -q $SNAPLV) && echo "$SNAPLV already exists! Aborting" 1>&2 && exit 1
    But this one is only to show you how you can use return codes to advantage. On the other hand, I would always suppress grep output in a case like this with -q. After all you do not want to see it, or program on that line, you only want to know if it is there or not.
    ================
    Code:
    # Make sure $DIRECTORY exists, if not then create directory!
            if [ -d /mnt/$SNAPLV ]
            then
                    echo "Directory $SNAPLV exists." >> $LOGFILE
            else
                    /usr/bin/mkdir /mnt/$SNAPLV
            fi
    can be done much shorter:
    Code:
    mkdir -p /mnt/$SNAPLV
    Or, when you want to be sure it is empty:
    Code:
    rm -rf /mnt/$SNAPLV
    mkdir -p /mnt/$SNAPLV
    ==================
    Code:
    if [ ! -z "$(mount -l | grep $SNAPLV)" ]
    I leave it up to you to convert this to the same style as the other one (with grep -q). Suggestion:
    Code:
    if (mount -l | grep -q $SNAPLV)
    ===================
    You may want to change
    Code:
    if [ $? -ne 0 ]
    using your new knowledge.
    ==================

    In any case, happy programming in the new year
    Henk van Velden

  5. #5

    Default Re: [CRON] Mount failing in backup script when run from cron

    Quote Originally Posted by hcvv View Post
    Why using /bin/sh instead of /bin/bash. Most people on Linux write shell scripts in bash. You are of course correct if you program in the POSIX shell. Differences are not great, but there are differences.
    This was something that I modified last night as it was suggested in #openSuSE. Has been set back to /bin/bash as intended in the first place.

    Quote Originally Posted by hcvv View Post
    Inconsistency in using if [ .... ] and if [[ .... ]] contstructs. Try to standardize upon one of them (I prefer the if [[ ... ]] one).
    This happens when you gobble up code from all over the place and try to pour it into a custom script :-) Will clean it up as soon as I'm statisfied with the function. Prio's atm are:

    1. Building in extra checks and having a way to abort the running script while breaking down what it set-up. Like unmounting the dir, removing it and removing the snapshot
    2. Have the script look into a directory under /etc that has different .conf files with the variables of each LV i want to backup.
    3. Adding a command line switch to use dd/partimage instead of mounting the LV and using rsync.


    Quote Originally Posted by hcvv View Post
    Instead of
    Code:
    if [[ $(lvs | grep $SNAPLV) ]]; then
                    echo "$SNAPLV already exists! Aborting" 1>&2
                    exit 1
            else
    I would use
    Code:
    lvs | grep -q $SNAPLV) && echo "$SNAPLV already exists! Aborting" 1>&2 && exit 1
    But this one is only to show you how you can use return codes to advantage. On the other hand, I would always suppress grep output in a case like this with -q. After all you do not want to see it, or program on that line, you only want to know if it is there or not.
    Sound advice on adding -q switch. I've added it to the script!

    I don't see much difference between the 2 codeblocks though. Afaick they will both do the exact same thing only your suggestion is shorter, therefore it is more lazy and lazy == good? On the other hand I like how mine reads between the other codeblocks. Is there any downside on using my current syntax?

    Quote Originally Posted by hcvv View Post
    Code:
    # Make sure $DIRECTORY exists, if not then create directory!
            if [ -d /mnt/$SNAPLV ]
            then
                    echo "Directory $SNAPLV exists." >> $LOGFILE
            else
                    /usr/bin/mkdir /mnt/$SNAPLV
            fi
    can be done much shorter:
    Code:
    mkdir -p /mnt/$SNAPLV
    Or, when you want to be sure it is empty:
    Code:
    rm -rf /mnt/$SNAPLV
    mkdir -p /mnt/$SNAPLV
    I specifically left this in because I planned to stop the script here when the dir already existed. Might go for your option though in the future.

    Quote Originally Posted by hcvv View Post
    Code:
    if [ ! -z "$(mount -l | grep $SNAPLV)" ]
    I leave it up to you to convert this to the same style as the other one (with grep -q). Suggestion:
    Code:
    if (mount -l | grep -q $SNAPLV)
    Code:
    # check whether mount point is in use
            if (mount -l | grep -q $SNAPLV)
            then
                    write_to_log "[X] Error: Mount point already in use"
                    write_to_log "====== Backup failed ======"
                    exit 1
            fi
    Changed!

    Quote Originally Posted by hcvv View Post
    You may want to change
    Code:
    if [ $? -ne 0 ]
    using your new knowledge.
    You're right. The $? seems like a bad way to set this up. I've added my command to the if statement so this should be much cleaner.
    Code:
            if (/usr/bin/mount --read-only $MOUNTOPTS /dev/$WORKVG/$SNAPLV /mnt/$SNAPLV >> $LOGFILE)
            then
                    echo "Mounted /dev/$WORKVG/$SNAPLV on /mnt/$SNAPLV" >> $LOGFILE 
            else
                    rmdir /mnt/$SNAPLV
                    /sbin/lvremove -f /dev/$WORKVG/$SNAPLV >> $LOGFILE
                    write_to_log "[X] Error: Could not mount /dev/$WORKVG/$SNAPLV to /mnt/$SNAPLV"
                    write_to_log "====== Backup failed ======"
                    exit 1
            fi
    Quote Originally Posted by hcvv View Post
    In any case, happy programming in the new year
    Thanks for the good advice!

  6. #6
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    24,853

    Default Re: [CRON] Mount failing in backup script when run from cron

    As my suggestions were only ment to show you alternatives and to let you study certain ways of doing things (leaving it up to you to adopt them or not), I will not comment further.

    Except on one:
    ... is shorter, therefore it is more lazy and lazy == good?
    I assume being lazy is good for a programmer. That is the explanation for the conception of many programs/scripts. They were made because programmers had to do the same (more or less) thing more then once. Thus that resulted in many nice things were the maker good watch lazily with a coffee instead of typing.

    Short is not always better. IMHO a consistent and clear programming style comes first.
    OTOH you will understand that using
    Code:
    program  && other-program-runs-when-first-is-ok
    uses less CPU cycles then
    Code:
    program
    if [[ ${?} -eq 0 ]] ; then other-program-runs-when-first-is-ok ; fi
    where, depemding on exect usage, $? is converted from integer to character.
    Now this is often neglectable, but when it is with a loop that runs many times, it could become noticable.
    Henk van Velden

  7. #7

    Lightbulb Re: [CRON] Mount failing in backup script when run from cron

    Update: I've finetuned the script to use input from the shell as the variables and cleaned up the code (to my ability). Posted here for reference.

    Comments / feedback appreciated.

    Code:
    #!/bin/bash
    
    WORKVG="$1"
    SOURCELV="$2"
    SNAPLV="$2-SNAP1"
    BACKUPDIR="$3"
    MOUNTOPTS="$4" # << if FS == XFS -onouuid,ro
    SNAPSIZE="$5"
    
    
    display_usage() { 
            echo "You need to input WORKVG SOURCELV BACKUPDIR MOUNTOPTS SNAPSIZE" 
            echo "if FS == XFS then MOUNTOPTS should read -onouuid,ro"
            echo -e "\nUsage:\n$0 VG_HOSTNAME LV_LIBVIRT /var/backup/HOSTNAME/ -onouuid,ro 16G \n" 
            } 
    
    
    # if less than two arguments supplied, display usage 
            if [  $# -le 1 ] 
            then 
                    display_usage
                    exit 1
            fi 
     
    # check whether user had supplied -h or --help . If yes display usage 
            if [[ ( $# == "--help") ||  $# == "-h" ]] 
            then 
                    display_usage
                    exit 0
            fi 
     
    # display usage if the script is not run as root user 
            if [[ $USER != "root" ]]; then 
                    echo "This script must be run as root!" 
                    exit 1
            fi
    
    
    # Make sure only root can run our script
            if [ "$(id -u)" != "0" ]; then
                    echo "This script must be run as root" 1>&2
                    exit 1
            fi
                    echo "====== Starting Backup ======"
    
    
    # Check if SNAPLV exists
            if [[ $(/sbin/lvs | grep -q $SNAPLV) ]]; then
                    echo "====== Backup failed: $SNAPLV already exists! Aborting ======"
                    exit 1
            else
                    /sbin/lvcreate -L $SNAPSIZE -s /dev/$WORKVG/$SOURCELV -n $SNAPLV
            fi
    
    
    # Make sure $DIRECTORY exists, if not then create directory!
            if [ -d /mnt/$SNAPLV ]
            then
                    echo "Directory $SNAPLV exists."
            else
                    /usr/bin/mkdir /mnt/$SNAPLV
            fi
    
    
    # check whether mount point is in use
            if (mount -l | grep -q $SNAPLV)
            then
                    echo "[X] Error: Mount point already in use"
                    echo "Removing $SNAPLV"
                            /sbin/lvremove -f /dev/$WORKVG/$SNAPVL
                    echo "====== Backup failed: Error: Mount point already in use !!!PLEASE CLEAN UP MANUALLY!!! ======"
                    exit 1
            fi
    
    
    # Mount SNAPVOL
            if (/usr/bin/mount --read-only $MOUNTOPTS /dev/$WORKVG/$SNAPLV /mnt/$SNAPLV)
            then
                    echo "Mounted /dev/$WORKVG/$SNAPLV on /mnt/$SNAPLV"
            else
                    rmdir /mnt/$SNAPLV
                    /sbin/lvremove -f /dev/$WORKVG/$SNAPLV
                    echo "====== Backup failed: [X] Error: Could not mount /dev/$WORKVG/$SNAPLV to /mnt/$SNAPLV ======"
                    exit 1
            fi
    
    
    # Do the Backup
                    echo "Backing up Disk Images $WORKVG/$SNAPLV/* to $BACKUPDIR"
                            /usr/bin/lzop --force /mnt/$SNAPLV/*  --path=$BACKUPDIR
                    echo "Dumping config files from /etc/libvirt/qemu"
                            /bin/cp /etc/libvirt/qemu/*.xml $BACKUPDIR
    # CLeaning up                
                    echo "Cleaning up"
                                            /usr/bin/umount /mnt/$SNAPLV
                                            /usr/bin/rmdir /mnt/$SNAPLV
                                            /sbin/lvremove -f /dev/$WORKVG/$SNAPLV
                    echo "Backup of $WORKVG/$SNAPLV to $BACKUPDIR finished succesfully!"
                    echo "====== Backup finished ======"
    exit

  8. #8

    Default Re: [CRON] Mount failing in backup script when run from cron

    Hi,

    Crons default PATH is very limited, You could try to put the code below as a cron entry.

    Code:
    * * * * * echo "$PATH" > /tmp/cronpath.txt 2>&1
    That should log every minute (i hope ) Then you can check what was written in the file
    Code:
    /tmp/cronpath.txt
    Now run the same code in your interactive shell less the leading *
    Code:
    echo "$PATH"
    You should see the difference. That being said you can modify the path of the cron entry. see
    Code:
    man 5 crontab
    Or if you have the package
    Code:
    mdadm
    You can check its cron entry in the directory
    Code:
    /etc/cron.d
    the PATH is set/define so absolute path in your cron script can be avoided. There are cron entries which you can read/check as well in the cron directories like
    Code:
    /etc/cron.daily/ 
    /etc/cron.hourly/ 
    /etc/cron.monthly/ 
    /etc/cron.weekly/
    Good luck...
    "Unfortunately time is always against us" -- [Morpheus]

    .:https://github.com/Jetchisel:.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •