What's the best tool for backing up / restoring partitions to / from image files?

As I said on other threads, I’m always paranoid about something happening and breaking my system (specifically its ability to boot). Although I use Yast - System Backup to create backups of installed packages and their data, I won’t feel safe until I can also create images of my openSUSE partition which I can keep on other drives. In Windows 7 I used Macrium Reflect which did its thing well, but now I need an alternative for SUSE.

I already found one called dd, and am currently cloning my partition with the command “dd if=/dev/sdb3 of=/windows/D/System/sdb3.img” which appears to be working well. But this has a big issue: It creates a file the same size as the partition, instead of saving just the data. My openSUSE partition is 177 GB large, but currently filled up to 50 GB. Therefore about 125 GB of disk space (as well as the backup time) are being wasted to save nothing, which I don’t want to go with unless no other option exists. Someone on IRC explained that dd doesn’t know about file systems and can only store partitions as a whole. If so, is there any tool that does and can backup / restore partitions to / from images?

Something else I’d like clarified about partition cloning; Many suggest doing it with a live DVD, but I really prefer to leave the backup tool running while I’m away and sleeping for instance, without booting into something else each time. The problem I heard about is that a partition might be saved as corrupted if it’s modified during the backup. Since I’m backing up from the openSUSE installation located on the partition I’m backing up, I likely can’t set it as read-only (that would be like setting C:/ read-only while Windows is running). Do partition backups get corrupted if data on them is modified while they’re being saved? If so, is it possible to set all hard drives to read-only while openSUSE is running and backup to an external drive during that time? I know that means no application can save or modify anything (and some programs might even crash), but if I do this while I’m sleeping it doesn’t bother me.

I’m beginning to doubt your setup. It looks like you have a single partition openSUSE setup of 177 GB. The installer will always suggest a separate /home partition. The big advantage is that you keep ./home at a next install.
The one thing you IMHO should do, is backup your entire /home, reinstall 12.2 and create three partitions
2 GB of swap
25 GB for /
the rest for /home.
Then, after install you restore the content of the backupped /home to the newly created partition mounted on /home

That would allow you to install, reinstall, update, upgrade without touching your /home.

I like dar. I use it on my 11.3 system. I have not checked if it is available for 12.2. It is a file aware backup and runs under the system that is being backed up. It can restore to a different partition format and you can restore individual files. What it does not do is restore the partition structure and MBR. If as part of your backup you record the partition structure and know how to recreate the grub MBR all should be well. You also need a bootable system on USB or CD to recreate and format partitions and this must include a working version of dar.

I typically will use firefox while running a dar backup and never had any problems. However if you do too much you will likely have inconsistent ( but not corrupt ) files rather than a corrupt file system

I have not checked the availability of dar on 12.2 or its ability to work with UEFI based systems

@ Knurpht: Yeah, I never used a separate home partition. Back when I did the partitioning for my drives I didn’t even know why Linux would suggest such a thing. I like this setup though… even if backing up the system can be a bit more problematic. My home folder is closely related to installed applications, so it feels ok to have both on the same partition. If I ever change my partitioning drastically though, I might reconsider this.

@ vindevienne: I don’t need the MBR backed up. Especially since my system is dual-boot with Windows 7, and restoring the bootloader like that could likely be problematic. If I’d ever have to restore a partition backup, what I’d do is wiping / formatting the openSUSE installation’s partition, then restoring the backup image to it. GRUB isn’t aware of this so normally it should boot the same way after that. If not, is it possible to use the installation DVD, re-detect operating systems and generate a new GRUB setup?

As for dar, I see it in the repos for 12.2. Might try it out if it works for what I want. Keep in mind I do want to make a full partition backup (not just copy my files to an archive) which can be used to restore the entire OS to an empty partition. With the only exception of not wasting 125 GB to also store the unused area of that partition :stuck_out_tongue: Someone also mentioned partimage, but I don’t see it in the repos.

On 2012-09-25 22:46, MirceaKitsune wrote:
>
> As I said on other threads, I’m always paranoid about something
> happening and breaking my system (specifically its ability to boot).
> Although I use Yast - System Backup to create backups of installed
> packages and their data, I won’t feel safe until I can also create
> images of my openSUSE partition which I can keep on other drives. In
> Windows 7 I used Macrium Reflect which did its thing well, but now I
> need an alternative for SUSE.

Backup software for Linux is not that “cute” as Windows software. And sometimes very below par

  • I compare with PCTools Backup from the 80’s 90’s, for MSDos. I sorely miss the ability to
    compress with verification and with error recovery data.

> I already found one called dd, and am currently cloning my partition
> with the command “dd if=/dev/sdb3 of=/windows/D/System/sdb3.img” which
> appears to be working well. But this has a big issue: It creates a file
> the same size as the partition, instead of saving just the data. My
> openSUSE partition is 177 GB large, but currently filled up to 50 GB.
> Therefore about 125 GB of disk space (as well as the backup time) are
> being wasted to save nothing, which I don’t want to go with unless no
> other option exists. Someone on IRC explained that dd doesn’t know about
> file systems and can only store partitions as a whole. If so, is there
> any tool that does and can backup / restore partitions to / from images?

Well, dd does a blind and exact image. Advantages: it restores all, even the non used sectors
where grub resides, so it restores booting. For example, if you have grub on a small /boot
partition, say 200 MB, saving it with dd is an advantage. It is the most reliable tool.

For big partitions, non bootable, clonezilla does a good job. It does not save unused sectors,
and it also does compressing. Another similar tool is SystemRescueCd (the one I use).

> Something else I’d like clarified about partition cloning; Many suggest
> doing it with a live DVD, but I really prefer to leave the backup tool
> running while I’m away and sleeping for instance, without booting into
> something else each time. The problem I heard about is that a partition
> might be saved as corrupted if it’s modified during the backup. Since
> I’m backing up from the openSUSE installation located on the partition
> I’m backing up, I likely can’t set it as read-only (that would be like
> setting C:/ read-only while Windows is running).

You simply can not do an image backup of a mounted partition. Forget it. At best, when you
recover it and attempt booting, the system will have to fsck it and find lots of broken sectors.

Halt the computer, boot the dvd or usb stick, and do the image backup.

> Do partition backups
> get corrupted if data on them is modified while they’re being saved?

Always. Absolutely.

> If
> so, is it possible to set all hard drives to read-only while openSUSE is
> running and backup to an external drive during that time?

No.

> I know that
> means no application can save or modify anything (and some programs
> might even crash), but if I do this while I’m sleeping it doesn’t bother
> me.

It will. The machine would not work the next day when you get up, forcing a reboot.

You can do an rsync backup, and this will also have problems.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

Ok. Sucks that you have to use a live DVD each time… I was hoping I could leave the backup tool running while I’m sleeping and find it done in the morning (dd didn’t warn me about any errors). Might look into Clonezilla in this case.

Anyway, I’m currently looking at how dar works, as it apparently saves the entire drive to a special type of archive. Is it possible to save the entire partition to it, and if needed restore it to a fresh / formatted ext3 partition and be able to boot Linux from there like nothing happened (after re-configuring GRUB if necessary)?

Also, can the installation DVD of openSUSE be used to scan the drive for operating systems and re-generate a GRUB setup, without modifying anything else? Would be helpful in the event of such a restoration.

On 2012-09-26 01:56, MirceaKitsune wrote:
>
> Ok. Sucks that you have to use a live DVD each time… I was hoping I
> could leave the backup tool running while I’m sleeping and find it done
> in the morning (dd didn’t warn me about any errors). Might look into
> Clonezilla in this case.

There are two problems while backing up an image of a mounted partition. One, is that files can
be caught at the middle of writing operations: some you will catch empty, others half written,
others inconsistent… and maybe more cases I forget about.

Another is that the metadata of the disk is partly in memory, partly modified. For example, a
file might be using records 1, 2, 5, & 7, but this might not still be written on the disk. When
you try to mount a restored image in this state the system detects that it was not closed
properly and will do an fsck on it - which might succeed or not, there is always some danger of
lost files.

It is possible to do it, there are machines where that is the best that can be done, because
the machine is not stoppable. I know of a machine where what they did was disconnect one half
of a raid disk, backup that half while the first half kept the system working, and then
reassemble the raid.

> Anyway, I’m currently looking at how dar works, as it apparently saves
> the entire drive to a special type of archive. Is it possible to save
> the entire partition to it, and if needed restore it to a fresh /
> formatted ext3 partition and be able to boot Linux from there like
> nothing happened (after re-configuring GRUB if necessary)?

Dar will not restore the boot sectors, I think. It backups files.

>
> Also, can the installation DVD of openSUSE be used to scan the drive
> for operating systems and re-generate a GRUB setup, without modifying
> anything else? Would be helpful in the event of such a restoration.

I’m not sure if I understand.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

Makes sense. Hope some way to properly backup this way will eventually exist. Might go for Clonezilla in the meantime.

Yes, not boot sectors. But for example: Can I use dar to backup the entire openSUSE partition? Then if something bad happens restore it to a freshly formatted ext3 drive, then use the installation DVD to re-detect operating systems on drives and generate a new boot configuration.

What I said above. I was wondering if I can insert the openSUSE DVD, boot it, then use a tool located there to detect operating systems and generate a new GRUB. When you install SUSE for the first time, a bootloader setup is generated. But I’d need to regenerate only that bootloader without changing anything else.

On 2012-09-26 10:56, MirceaKitsune wrote:
>
> robin_listas;2490577 Wrote:

>
> Makes sense. Hope some way to properly backup this way will eventually
> exist. Might go for Clonezilla in the meantime.

Only partial solutions are possible.

For example, a running database may have a feature to make a dump of the database to an
external backup file in a consistent state. For example, all incoming modifications are written
to another file while the central database is stabilized, then copied, and finally the
modifications that were written externally are committed to the central database. I have worked
with such a system.

But it is a partial solution.

You have to stop all running applications, office applications, graphical editors, email
daemons, log writing… absolutely every thing! Then copy that partition somewhere else, and
fast, for backup. This is not doable with a general purpose operating system.

> robin_listas;2490577 Wrote:
>>
>> Dar will not restore the boot sectors, I think. It backups files.
>>
>
> Yes, not boot sectors. But for example: Can I use dar to backup the
> entire openSUSE partition? Then if something bad happens restore it to a
> freshly formatted ext3 drive, then use the installation DVD to re-detect
> operating systems on drives and generate a new boot configuration.

Perhaps…


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

I’ve used Clonezilla with good results.
It can also be found in the Parted Magic cd together with other back up tools.

Got VirtualBox setup again so I can see how this works. Will probably try it later today to make sure I get the hang of using it, then I’ll try to backup my system as well. Normally, I want to backup my Linux partition to a NTFS partition (which in openSUSE is mounted as /windows/[letter]/) in a subfolder of that partition, both partitions being part of the same hard drive. So I hope this will work safely.

On 09/27/2012 12:06 PM, MirceaKitsune wrote:
> Normally, I want to backup my Linux partition to a NTFS partition

i’m not sure that is a great idea because NTFS has no capability to
handle standard Linux file permissions…so, if you have an emergency
need for the backed up file you might be stuck without the correct file
permissions.


dd http://goo.gl/PUjnL

That’s ok, I can mount all NTFS partitions with read access easily. I’m still divided between Windows and openSUSE, and will be using my external drives and memory sticks on Windows computers too, so most of my partitions will need to stay NTFS. Sucks that Microsoft never bothered to make ext4 readable in Win, while we bothered for their NTFS :slight_smile:

Am 27.09.2012 13:06, schrieb MirceaKitsune:
> Sucks that Microsoft never
> bothered to make ext4 readable in Win, while we bothered for their NTFS
> :slight_smile:
>
There are some third party tools for windows
http://sourceforge.net/projects/ext2fsd/
this can also handle ext4. I never used it and if you want to try it I
recommend you use something for testing where you do not care if the
file system gets corrupt (testing on an USB stick for example which
contains only data for which you have a backup somewhere).


PC: oS 12.2 x86_64 | i7-2600@3.40GHz | 16GB | KDE 4.8.4 | GeForce GT 420
ThinkPad E320: oS 12.2 x86_64 | i3@2.30GHz | 8GB | KDE 4.9.1 | HD 3000
eCAFE 800: oS 12.2 i586 | AMD Geode LX 800@500MHz | 512MB | KDE 3.5.10

On 09/27/2012 01:06 PM, MirceaKitsune wrote:
> That’s ok, I can mount all NTFS partitions with read access easily.

no, you don’t understand…

if you copy /etc/fstab on your openSUSE to an NTFS partition, and then
the original (on your Linux) gets mangled and you need to go grab your
safe copy of the original…you would go to your “read access easily”
NTFS, and copy it back to overwrite the corrupted fstab…but, that
won’t work because the file permission of the fstab on the NTFS
partition are all messed up, what will you do then?

so, you may wonder how will the file permission get messed up just by
copying a file to NTFS?

the answer: as said, “NTFS has no capability to handle standard Linux
file permissions” so when the file is on NTFS it has NO file permissions
or ownerships (meaning it lost its original permission) and therefore
when copied back i will either have no permissions or those of the user
who copied it back…


dd
*

Ah, I understand. But I don’t think it’s the case here, since this would apply if I just copy the file back and forth directly. In the case of a partition backup like I’m doing (Clonezilla), the partition is stored in an image file (img or something), and all contents are in there. The partition you store the image file on does not modify what’s inside the file. And once I’d be restoring it directly to an ext4 partition, permissions would likely come back too as they were. But yes, this would happen if the files were copied back and forth directly, and thanks for pointing that out so I know.*

If he is doing the backup with a tool like “dar” or “clonezilla”, so that all that shows up on NTFS is an archive file, then I don’t see that as a problem. If he wants to do a recursive “cp” to build a similar directory tree on NTFS, then it is a problem.

I backup “/home” using “dar” with the archive on an external NTFS formatted drive, and that works fine for me.

Yeah, that’s what I’ll be doing. Already tried this out on VirtualBox and it was a success. I installed openSUSE 12.2 to the ext4 partition, made a separate ntfs partition, used Clonezilla to backup the ext4 one to an image on the NTFS (a folder with multiple files), formatted the ext4 with Live KDE openSUSE, made sure nothing boots and it’s all gone, used Clonezilla to restore the image from the NTFS partition, system booted as if nothing happened :slight_smile:

On 09/27/2012 03:16 PM, MirceaKitsune wrote:
> I’m doing (Clonezilla), the partition is stored in an image file

ah, i use rsync and don’t bother to wrap stuff up in a tar ball/zip or
whatever…


dd

Ok. Sucks that you have to use a live DVD each time… I was hoping I could leave the backup tool running while I’m sleeping and find it done in the morning (dd didn’t warn me about any errors). Might look into Clonezilla in this case.

and

Makes sense. Hope some way to properly backup this way will eventually exist.

Well actually, it doe exist…that’ s what LVM and Btrfs snapshots are for…