Backup

  • First off, to all you people who’ve contributed to OpenSuse (or other distributions). I walked away from Vista x64 this weekend and I think Linux is fab. I’m kicking myself that I hadn’t done this sooner. Everything installed/configured minimum of fuss and anything missing is easy to find in either repositories or forums. I’ve installed on a Lenovo T61 and everything that needs to work is working. I’ve not had a crash of any sort in 72 hours … (Vista x64 had me close to smashing the laptop).

  • The one thing I do not have is an apparently unanimously agreed way of taking a reliable ONLINE system image to a file on disk.

  • Problems/issues:

  • My system is installed on 250 GB disk and I have 2 x 200 GB USB disks available for backups (rotating). There is 80GB free on the system. There is one root partition with everything in it (simple!) taking the full 250 GB disk. / is Ext3.
  • I have TrueImage and the TI+DD rescue media.
  • DiskDirector 10.0 from Acronis Rescue CD cannot see my SATA drive. It is running in ACHI mode. I can obviously change it but don’t want to have to flip it for backups.
  • TrueImage 11.0 from Acronis Rescue CD cannot clone Ext3 to a smaller sized disk [if someone can counter that I’d like to hear it]. I’m guessing it is using a sector-by-sector copy so needs a disk at least as big as the one I’m backing up.
  • TrueImage 11.0 from Boot CD was going to take about 15 hours to create a system image file to a USB NTFS formatted drive (as “tib” file); this is not a runner for weekly use.
  • In any event, I would rather avoid having to boot from a CD in order to perform a backup.
  • DD cannot clone a partition to a smaller disk; there is a risk of data loss.
  • I’m guessing it would not be particularly safe to use DD on a running system; it won’t affect the running system but would lead to inconsistent results in the backup image.
  • The System Backup in openSuse is not a complete system image. In addition it seems to need a huge amount of space to make the actual backup so is not very practical.
  • “tar” doesn’t backup certain types of special files (sockets, etc)? I’m not sure how safe it is for a complete system backup.
  • Ideally:
  • I would like to be able to perform an ++online++ backup of the complete system (AIX can do it, so can Windows, does Linux have a way to “shadow copy” files or similar?).
  • It would be nice if the image was in “a file” and nicer if that could be on NTFS so it could be read easily on other systems. I have figured out how to mount NTFS external drives r/w.
  • Would also be cool if you could extract individual files from the backup image, for example, a 40 GB VMWare image …
  • If tools are required I’d like them to be completely open source/free (if possible).
  • All system settings, versions, permissions should be preserved by the backup.
  • Thoughts:
  • PartImage. Can it clone Ext3 to a smaller disk? Still leaves the problem of being able to access the backup from a non-Linux system. Also still requires that you boot the system from CD to create the image?

  • DD. I suspect I could boot from openSuse LiveCD and do:

    dd if=/dev/sda | gzip - > /media/mydisk/sda.gz

    This doesn’t satisfy all of the criteria but if the command is right looks like one hell of an easy way to take a complete system image to a file. Is this a valid thing to do?

  • Is there a recognised way to do a complete system backup image (to file on disk) on a running system in such a way that the image is accessible and can have individual files restored and also that the creation of the image doesn’t take as much space on the running system as the running system itself is occupying …

-Tia

Your backup strategy depends on how much you are going to tweak the system files or whether you use mysql.

If you do no system tweaking and don’t use mysql, all you need to do is backup /home. If that is on a separate partition, then if the system goes down, you simply reinstall the system. Because of the way the installer lays down the files in a fresh install, this will normally be faster than restoring a backup of the system.

If you do a small amount of system tweaking, copy the files you have tweaked to a backup disk.

If you are using mysql, use mysqldump to copy everything to a backup disk (I actually copy the backup to /home and then backup /home).

If you really intend to change a lot of system files, you may find using System backup is easier in the end.

Others may have other suggestions.

I think I covered off on my requirements; I want a full system image. I also covered off on a separate problem with System Backup; it takes a lot of space (over and above the backup file) in order to make the backup [ie: system almost ran out of space when backing up]. Not withstanding that, it doesn’t give me the ability to do a bare metal restore. The DD option would probably come close; I’m about to try TI Home 2009 to see if it’s any improvement over TI Home 11 regarding a) Recognition of ACHI and b) USB performance c) Clone Ext3 to smaller disk.

(Would seem initially that TI 2009 offers no improvement.)

My question would be:why do you want a full system image?

This is something that people have got used to in Windows because of the chaotic organisation of files on a Windows drive.

One reason similar software isn’t widely available for Linux is that it is unnecessary in Linux.

Historically Linux users who required a mirror of their system have taken the RAID route rather than the system image route or had a second drive and used rsync to keep it uptodate.

In effect, you have told us what your solution is: make a full system image - but we don’t know what the problem you are trying to solve is - so it is difficult to know what would be the best solution for your problem in a Linux environment. It’s unlikely to be a Windows style solution.

The problem is being able to do a bare metal restore of the complete os with zero or next to zero configuration required as part of the restore process.

Full system image/clone works well; if the disk (most likely component to fail) dies, you simply put in the disk with the cloned image (not even a restore process).

I tried a restore with OpenSuse restore recently and what I got back didn’t appear to be right. Might have something to do with how -I- did it but the point is that I shouldn’t need to know anything special to get my system back.

Actually just tested and 2009 works with openSUSE 11x

/Geoff

I maintain a mirror on another disk using rsync in a script. I schedule it to run twice a day (rsync is bloody fast); frequency just depends on how current a snapshot one desires. If I wanted to, I could easily compress the mirror - I might consider 7z for its tight compression and OS portability. I haven’t got around to trying to access individual files in a 7z - but I think that can be done; if not, there are other formats available (and it’s possible one of the others might actually yield a smaller file).

By the way, I have my system set up to be able to boot from production into that mirror, or, switch the bios and boot directly from the mirror itself. Since it would require disk space for the backup anyway, and the space saved via compression is while appreciable not expensive given how cheap disk is now, this approach gives me on-the-fly recovery without restoring. Restoring to a new production disk is just a matter of reversing the process. And rsync is very flexible yet comprehensive, so I don’t capture, e.g., /sys nor /proc, which you will with dd.

But as indicated above, I could take the 2-step approach and make it an accessible single file, too.

I also use the same script to create a backup over my LAN on to a file server - this is my disaster recovery backup (as in, power supply blows and fries machine - I don’t have a strategy for if the house burns down, though). It also runs extremely fast, so like with this workstation the mirroring is virtually transparent.

Of course, my approach would not work with NTFS, as I want the backup to be bootable. It would work on NTFS if all I wanted is an image file for restoration.

Bear in mind that bare metal re-imaging sometimes requires certain preparatory steps in the data being imaged, so that the new drive will boot and mount correctly. For example, you want to use persistent yet portable partition ID’s (I suggest volume-label) rather than drive-name (e.g., sda) in fstab and menu.lst. Also advisable - particularly if imaging at the partition level - to not install grub to the MBR (which will have an embedded pointer to a particular partition); better to install generic boot code to the MBR and have root (or, separately, /boot) on a primary partition marked active.

If you search in the repository on “backup” there are some interesting tools. There is dar, which creates a portable cataloged compressed file. Rdiff-backup uses rsync but also supports incrementals. Bakula is more of an enterprise-class client/server backup management tool. And I know a guy who uses dd to backup to NTFS, where he has cygwin running for his *nix tools.

Hope some of this is useful.

@mingus725 Would you mind posting a quick howto. Just bought a new backup disk and been thinking of dumping Acronis
Thought I’d give rsync a try just need the basics

/Geoff

My script is very long and specific to these machines, network, and my preferences (bash logging, time-stamps, mount checks, etc.); posting that wouldn’t be very useful. I can provide a general idea, however:

Presume another disk in this machine, with an adequately sized partition to hold the entire installation. That partition is mounted in the script at /mnt/mirror. The rsync syntax might be:

/usr/bin/rsync -avuxz --stats --delete --exclude-from=/home/user/excludes // /mnt/mirror/

-a essentially copies all files exactly w/permissions, etc.
-v verbose
-u skips files if newer on the target (unlikely)
-x stay on filesystem, will suppress dynamic directories like /proc
-z use compression in transfer for speed

–stats to report files and summary to $stdout

–delete to delete any files on target not on source

–exclude-from= a file listing any files/directories you want to suppress in the transfer. This is important, e.g., if you mount data partitions and don’t want that included in the backup. It is also important to exclude the mount point of the target partition; otherwise, you would get a transfer on top of itself. I also exclude fstab, menu.lst, and device.map.

// - this is the source directory, it is “/” for root and a trailing “/” to say everything in that directory with that structure (otherwise you would get a / under /).

/mnt/mirror/ - this is the target. Since “mirror” is the root, and with the root source specified as above, then you get your entire root subdirectory with that structure under /mnt/mirror/.

I then copy over a modified version of fstab because when the mirror is booted root is different, and menu.lst because the boot device sequence is also different. I also copy over a custom wallpaper with “Failover” on it in bold letters so that when I boot the mirror I’m sure where I am!

Doing this to another machine can be set up in more than one way. You can use rsync asynchronously, but I prefer running the daemon on the target machine. When that is done, you set up an alias which corresponds to a mount point. Here’s an example of the transfer syntax, very similar to the above:

/usr/bin/rsync -avxz --stats --delete --exclude-from=/home/user/excludes-lan-xfer // 192.168.0.102::bkup-lan-server

Essentially the same except for the LAN address and target machine rsync alias. Rsync handles all the networking for you (i.e., you don’t need to use NFS or CIFS, although that can be done, too).

I use this same technique for backing up a web server, an ftp server, audio and video and photo libraries, a software library server, etc. There are other clients on this network backed up to the same file server, similar scripts.

Hope that is useful.

Problem with TI is it’s h/w support, not the particular version of OpenSuse. The boot media appears to have trouble with some current hard disk drives (doesn’t see the drive).

Interesting comment. I have occasionally found this to be true with different imaging tools, going back to Ghost years back. I know others swear by certain bit imaging tools, but I’ve grown to be gun shy about it and therefore gravitated to alternate methods.