Backup & Recovery using tar, dd, knoppix

[kbdeamon]

Hi all,

I’m new to Linux so I’m not sure if this is the right place to post.

I installed suse 11.4 and tried to test a bare-metal recovery by using KNOPPIX

1. fdisk -l > fdisk.bk

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006a70e

Device Boot Start End Blocks Id System
/dev/sda1 * 2048 104855551 52426752 83 Linux
/dev/sda2 104855552 171960319 33552384 82 Linux swap / Solaris
/dev/sda3 171960320 1953503231 890771456 83 Linux

2. cat fstab > fstab.bk

/dev/disk/by-id/ata-ST31000528AS_6VPBRQM5-part1 / ext4 acl,user_xattr 1 1
/dev/disk/by-id/ata-ST31000528AS_6VPBRQM5-part3 /home ext4 acl,user_xattr 1 2
/dev/disk/by-id/ata-ST31000528AS_6VPBRQM5-part2 swap swap defaults 0 0
proc /proc proc defaults 0 0
sysfs /sys sysfs noauto 0 0
debugfs /sys/kernel/debug debugfs noauto 0 0
usbfs /proc/bus/usb usbfs noauto 0 0
devpts /dev/pts devpts mode=0620,gid=5 0 0

3. backup mbr to flash: dd if=/dev/sda of=/media/sdf1/backup/mbr.img bs=512 count=1

4. backup /: tar cvfpj /media/sdf1/backup/system.tar.bz --exclude /mnt --exclude /proc --exclude . (this was down while i was in the / directory)

5. backup /home: tar cvfpj /media/sdf1/backup/home.tar.bz . (done while i was in /home)

6. tried testing by wiping out MBR and partitions:
   dd if=/dev/zero of=/dev/sda bs=512 count=1
   dd if=/dev/zero of=/dev/sda1 bs=1024k count=50

7. the system did not load after reboot which was to be expected.

8. using KNOPPIX again i restored the MBR from the flash drive

9. using mke2fs -t ext4 /dev/sda1 -L / (to restore the root filesystem)

10. used mke2fs -t ext4 /dev/sda3 -L /home (to restore the home filesystem)

11. After that / and /home mounted and I untared my backups into the respective directories.

12. When I try to reboot all I see is:

“Error No operating system” and am told to “Reboot and Select proper Boot device or Insert Boot Media in selected Boot device and press a key”

How can I make this work? Where did I go wrong? How can I properly perform a bare metal recovery?

[hcvv]

Hlloo, welcomee here.

It i s bit difficult to read and interprete your post because your story and the computer output that goes with it are not realy discernable from each other.
Please use CODE tags around the copied/pasted computer text: http://forums.opensuse.org/english/information-new-users/advanced-how-faq-read-only/451526-posting-code-tags-guide.html

I did thus not completely follow what you did, but if by any change you are trying to boot from another disk where you hardcopied (by e.g. using dd) fom te original working system, then you have a problem because the disk id is used in openSUSE for identifying partitions.

You will find that in* /etc/fstab*, /boot/gub/menu.lst and last but not leat in initrd. See the /dev/disk/by-id/ usage there.

[robin_listas]

On 2012-03-05 11:06, kbdeamon wrote:
>
> Hi all,
>
> I’m new to Linux so I’m not sure if this is the right place to post.

It doesn’t look you are so new, looking at the commands you use :-.)

> I installed suse 11.4 and tried to test a bare-metal recovery by using
> KNOPPIX

It is best to post computer text inside code tags, so that the formatting
is not destroyed. It is difficult to understand code when line wraps.

> 6. tried testing so by wiping out MBR and partitions:
> dd if=/dev/zero of=/dev/sda bs=512 count=1
> dd if=/dev/zero of=/dev/sda1 bs=1024k count=50
>
> 7. the system did not load after reboot which was to be expected.

Ok…

> 8. using KNOPPIX again i restore the MBR from flash drive and
>
> 9. used mke2fs -t ext4 /dev/sda1 -L / (to restore the root
> filesystem)
> 10. used mke2fs -t ext4 /dev/sda3 -L /home (to restore the home
> filesystem)

Where was grub installed to, MBR or a partition? Because if it went to a
partition then you also have to restore the boot record of that partition
using dd.

Also, grub does not only consist of one sector on the mbr or boot sector:
there are more records stored in the reserved unused space at the start of
the partition, and those are not restored by tar.

A better tool for recovery is something like partimage or clonezilla, at
least to cover grub completely. Home can be done as you did.

Which is one advantage of using a small, separate, boot partition: it can
be imaged fast and easy. Imaging the entire root is slower.

–
Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

[kbdeamon]

Hi hcvv and robin_listas,

thank you so much for your prompt responses. I am making this easier to read by including code tags. Some of my replies are also below. I hope this is the right way to do it.

robin_listas, I’m fairly new to linux. I have been fixing and breaking my test system for about 3-4 weeks to date. Although I am fairly computer savvy and know my way around windows, my linux knowledge is limited to unix theory. Thus, the commands are not to difficult to find. Using the commands to break the system is even less of a challenge

robin_listas: Where was grub installed to, MBR or a partition? Because if it went to a
partition then you also have to restore the boot record of that partition using dd.

GRUB was installed to the MBR.

Also, grub does not only consist of one sector on the mbr or boot sector: there are more records stored in the reserved unused space at the start of the partition, and those are not restored by tar.

I don’t understand.

robin_listas: Which is one advantage of using a small, separate, boot partition: it can be imaged fast and easy. Imaging the entire root is slower.

I did not have a separate /boot partition which means that /boot was part of / and should have been saved with tar, no?

hcvv: any change you are trying to boot from another disk where you hardcopied (by e.g. using dd) fom te original working system, then you have a problem because the disk id is used in openSUSE for identifying partitions

I have noticed the by-id issue and thought that I would correct it after I was able to get a working backup and recovery solution. Right now I am not attempting to boot from another disk. I am copying everything (i.e., MBR, /home, / etc) to the previously backed up disk and trying to boot from it. But it should be clearer with the code text added.

Here goes.

I installed suse 11.4 and tried to test a bare-metal recovery by using KNOPPIX. To prepare the recovery disk I backed up /etc/fstab, fdisk-l, the MBR, / and /home. I did not do anything to the Swap partition. The code follows.

1. backup of fdisk

fdisk -l > fdisk.bk

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006a70e

Device Boot Start End Blocks Id System
/dev/sda1 * 2048 104855551 52426752 83 Linux
/dev/sda2 104855552 171960319 33552384 82 Linux swap / Solaris
/dev/sda3 171960320 1953503231 890771456 83 Linux

2. backup of fstab

cat fstab > fstab.bk

/dev/disk/by-id/ata-ST31000528AS_6VPBRQM5-part1 / ext4 acl,user_xattr 1 1
/dev/disk/by-id/ata-ST31000528AS_6VPBRQM5-part3 /home ext4 acl,user_xattr 1 2
/dev/disk/by-id/ata-ST31000528AS_6VPBRQM5-part2 swap swap defaults 0 0
proc /proc proc defaults 0 0
sysfs /sys sysfs noauto 0 0
debugfs /sys/kernel/debug debugfs noauto 0 0
usbfs /proc/bus/usb usbfs noauto 0 0
devpts /dev/pts devpts mode=0620,gid=5 0 0

3. backup of MBR

 mbr to flash: dd if=/dev/sda of=/media/sdf1/backup/mbr.img bs=512 count=1 

4. backup / (this was down while i was in the / directory):

 tar cvfpj /media/sdf1/backup/system.tar.bz --exclude /mnt --exclude /proc --exclude .  

5. backup /home (done while I was in /home):

 tar cvfpj /media/sdf1/backup/home.tar.bz . 

6. tried testing by wiping out MBR and partitions:

 dd if=/dev/zero of=/dev/sda bs=512 count=1 

 dd if=/dev/zero of=/dev/sda1 bs=1024k count=50 

7. the system did not load after reboot which was to be expected.

8. using KNOPPIX again i restored the MBR from the flash drive

9. Restored /

 mke2fs -t ext4 /dev/sda1 -L / 

10. Restored /home

 used mke2fs -t ext4 /dev/sda3 -L /home 

11. After that / and /home mounted and I untared my backups into the respective directories.

12. When I try to reboot all I see is:

“Error No operating system” and am told to “Reboot and Select proper Boot device or Insert Boot Media in selected Boot device and press a key”

Thank you for all your help.

[hcvv]

tlooks better then before ;). But it is not perfect. Please not take this as a questionto post everything again, but II have the following remarks on e.e. the first (fstab) paart.
When you do

fdisk -l > fdisk.bk

I do not expect to see any output because output is redirected to a file. Also what you show there is not a copy/paste from the screen because fdisk does create neat columns. As an example how to do this preferable:

boven:~ # fdisk -l

Disk /dev/sda: 320.1 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders, total 625142448 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x1549f232

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1              63     4209029     2104483+  82  Linux swap / Solaris
/dev/sda2         4209030    46154744    20972857+  83  Linux
/dev/sda3        46154745   240974999    97410127+  83  Linux
/dev/sda4   *   240975000   625137344   192081172+   f  W95 Ext'd (LBA)
/dev/sda5       240975063   282920714    20972826   83  Linux
/dev/sda6       282920778   492633224   104856223+  83  Linux
boven:~ # 

We all now see things I do not have to explain in a story, like:
. I was superuser;
. the working directory I was in;
. the command I used;
. all the output that came from it (up to the new prompt).

Back to the real problem.

What I see as strange (but I may be wrong) is:

tar cvfpj /media/sdf1/backup/system.tar.bz --exclude /mnt --exclude /proc --exclude .

I see that /media is not excluded, thus the tar file will tar itself, more or less. Also you exclude . It is late here, but I fail to undertstand what is tar-ed now. OTOH, you use the v command, thus you must have seen that it is more or less doing what you expected.

Could you post from your rescue system (Knoppix if I am right) your fdisk -l as it is now to check if it is the same as before? well, if it is the same, we trust you when you say so ;).

Hm I am waitting for inspiration, maybe to night. One has the best ideas about problem when sleeping IMHO.

[robin_listas]

On 2012-03-05 21:46, kbdeamon wrote:

> Also, grub does not only consist of one sector on the mbr or boot
> sector: there are more records stored in the reserved unused space at
> the start of the partition, and those are not restored by tar.
>
> I don’t understand.

Grub stores parts of itself (stages 1.5 and 2, IIRC) after sector 0 of the
partition, before the partition data starts. This code can not be saved by
tar because it is not files. You need to image it, or reinstall it manually
using a live system

A dd of the first 512 bytes is not sufficient to backup up grub. But I
don’t know how many sectors exactly grub occupies, I haven’t seen a recipe
for this.

> I did not have a separate /boot partition which means that /boot was
> part of / and should have been saved with tar, no?

The files, yes. But grub is not stored completely as files. The files in
/boot/grub is what you see, but “grub-install” puts the copy used during
boot just behind sector zero.

Notice also that if you do a recovery as files of an installations, the
disks ID’s are not restored (the uuid, labels…) because they are created
again when you format a filesystem. Thus after restoration and before
booting you have to edit fstab, grub/menu.lst, grub/device.map, and
possibly regenerate /boot/initrd*. And if you are restoring files you also
need to reinstall grub manually.

–
Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

[kbdeamon]

Also what you show there is not a copy/paste from the screen because fdisk does create neat columns.

You are right it is not a copy paste from the test system. I’m Sorry. I am posting this from my Win system as the system is currently not bootable.

What I see as strange (but I may be wrong) is:

tar cvfpj /media/sdf1/backup/system.tar.bz --exclude /mnt --exclude /proc --exclude .

I see that /media is not excluded, thus the tar file will tar itself, more or less. Also you exclude . It is late here, but I fail to undertstand what is tar-ed now. OTOH, you use the v command, thus you must have seen that it is more or less doing what you expected.

I did not think that I had to exclude /media because it is on the KNOPPIX cd. When the command ran it did not look like it was trying to tar itself. The complete backup is about 1.2 GB which seems reasonable for a what I presume was a 4 GB installation (sorry I did not note the install size).

Also, I just checked. It does not seem like my flash drive mounts into the / filesystem. I booted from Knoppix and mounted the drive. Then mounted / and went into /media. The path to the /media dir is /media/sda1/media. So I first go into /media on Knoppix and then to /media on /. I ran ls on /media/sda1/media and although it showed the label of the flash drive the dir was empty. So, it does not seem that I had to include /media.

Thoughts?

Your point is good, however! What I do not understand is why am I able to mount the root filesystem if the backup did not go as planned? I also do not understand why /root is so small now. Here is what I get when ls on the Knoppix

ls -l -h

total 8.0K
drwxr-xr-x 22 root    root    4.0K Mar  5 09:09 sda1
drwxr-xr-x  2 knoppix knoppix    0 Mar  5 21:16 sda3
drwxr-xr-x  2 knoppix knoppix    0 Mar  5 21:16 sdb
drwxr-xr-x  2 knoppix knoppix    0 Mar  5 21:16 sdc
drwxr-xr-x  2 knoppix knoppix    0 Mar  5 21:16 sdd
drwxr-xr-x  2 knoppix knoppix    0 Mar  5 21:16 sde
drwxrwxrwx  3 knoppix knoppix 4.0K Jan  1  1970 sdf1
drwxr-xr-x  2 knoppix knoppix    0 Mar  5 21:16 sr0

Could you post from your rescue system (Knoppix if I am right) your fdisk -l as it is now to check if it is the same as before? well, if it is the same, we trust you when you say so ;).

Before Knoppix

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006a70e

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   104855551    52426752   83  Linux
/dev/sda2       104855552   171960319    33552384   82  Linux swap / Solaris
/dev/sda3       171960320  1953503231   890771456   83  Linux

Knoppix

The command

fdisk -l -u > /media/sdf1/backup/fdisk.Knoppix 

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006a70e

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   104855551    52426752   83  Linux
/dev/sda2       104855552   171960319    33552384   82  Linux swap / Solaris
/dev/sda3       171960320  1953503231   890771456   83  Linux

Disk /dev/sdf: 8011 MB, 8011120640 bytes
180 heads, 28 sectors/track, 3104 cylinders, total 15646720 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0004e101

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1            2048    15646719     7822336    c  W95 FAT32 (LBA)

The fdisk appear the same to me. I don’t get it:(

[kbdeamon]

Thank you for the explanation:) But now I have more questions.

> I did not have a separate /boot partition which means that /boot was
> part of / and should have been saved with tar, no?

The files, yes. But grub is not stored completely as files. The files in
/boot/grub is what you see, but “grub-install” puts the copy used during
boot just behind sector zero.

Notice also that if you do a recovery as files of an installations, the
disks ID’s are not restored (the uuid, labels…) because they are created
again when you format a filesystem. Thus after restoration and before
booting you have to edit fstab, grub/menu.lst, grub/device.map, and
possibly regenerate /boot/initrd*. And if you are restoring files you also
need to reinstall grub manually.

I see. But if the disk is the same disk then the id would also be the same because it is in the hardware, no? I just compared the before and after fstab and they look the same.

So, reinstalling grub could be the answer here. How would I do this? I cannot login when I use the suse cd. It tells me that my login is incorrect.

Any thoughts?

–
Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)[/QUOTE]

[please_try_again]

@kbdeamon

GRUB was installed to the MBR.

Also, grub does not only consist of one sector on the mbr or boot sector: there are more records stored in the reserved unused space at the start of the partition, and those are not restored by tar.

I don’t understand.

When installed in MBR, Grub writes between 17 and about 30 sectors on the first track - the size varies depending on Grub version. This is called stage 1.5 and allows Grub - among other things - to access the file system and find stage2 … and finally present you with the boot menu. If Grub is not installed in MBR, it doesn’t use stage 1.5 (the reason is that there is no room for it) and looks for stage2 at the offset written in the boot loader (stage1) at installation time. This works as long as stage2 doesn’t move (like if you resize/move partitions or delete/restore file systems). Stage 1.5 is located in an aread outside of all partitions and is not part of any file system. Thus it can not be saved or restored with tools such as tar.

[please_try_again]

@kbdeamon

I installed suse 11.4 and tried to test a bare-metal recovery by using KNOPPIX. To prepare the recovery disk I backed up /etc/fstab, fdisk-l, the MBR, / and /home. I did not do anything to the Swap partition.

What about mounting by UUID (not by disk ID), getting another HD of the same size, duplicating the whole disk with dd … and watching “The Artist” while it is busy copying sector by sector? Everything would work on both hard disks, Grub, booting, etc (you have to use UUID or labels in /etc/fstab and /boot/grub/menu.lst though). Wouldn’t it be easier?

[kbdeamon]

@please_try_again

When installed in MBR, Grub writes between 17 and about 30 sectors on the first track - the size varies depending on Grub version. This is called stage 1.5 and allows Grub - among other things - to access the file system and find stage2 … and finally present you with the boot menu. If Grub is not installed in MBR, it doesn’t use stage 1.5 (the reason is that there is no room for it) and looks for stage2 at the offset written in the boot loader (stage1) at installation time. This works as long as stage2 doesn’t move (like if you resize/move partitions or delete/restore file systems). Stage 1.5 is located in an aread outside of all partitions and is not part of any file system. Thus it can not be saved or restored with tools such as tar.

So what I have now is not recoverable? Is there a way to rewrite grub because that seems to be the problem?

[kbdeamon]

@please_try_again

When installed in MBR, Grub writes between 17 and about 30 sectors on the first track - the size varies depending on Grub version. This is called stage 1.5 and allows Grub - among other things - to access the file system and find stage2 … and finally present you with the boot menu. If Grub is not installed in MBR, it doesn’t use stage 1.5 (the reason is that there is no room for it) and looks for stage2 at the offset written in the boot loader (stage1) at installation time. This works as long as stage2 doesn’t move (like if you resize/move partitions or delete/restore file systems). Stage 1.5 is located in an aread outside of all partitions and is not part of any file system. Thus it can not be saved or restored with tools such as tar.

So what I have now is not recoverable? Is there a way to rewrite grub?

I don’t totally follow this, so pardon the following question. If I use /boot for grub would it be enough to just dd if=/dev/sda1 of=mbr.img bs=512 count=1?

[kbdeamon]

@please_try_again

What about mounting by UUID (not by disk ID), getting another HD of the same size, duplicating the whole disk with dd … and watching “The Artist” while it is busy copying sector by sector? Everything would work on both hard disks, Grub, booting, etc (you have to use UUID or labels in /etc/fstab and /boot/grub/menu.lst though). Wouldn’t it be easier?

I think that the dd method would be easier. I can’t get another HD right now though :(.

From testing how dd worked it was taking up a bunch of space even if compressed. The problem was not backing up / but backing up /home which is approximately 850 GB. If I understand correctly dd copies empty space as well and when it is piped through an achiever such as tar, the output is not void of the empty space.

So the space issue was one of the reasons why I went with tar. With tar, I am able to backup the system to a flash drive. Is there a way to get rid of the empty space?

I think that I should start over, has anyone used tar to backup their SuSe?

Thank you for all your help.

[please_try_again]

@kbdeamon

So what I have now is not recoverable? Is there a way to rewrite grub because that seems to be the problem?

Anytime from a live CD with the grub shell (the command “grub”) . The method is explained in many posts and howtos. Just search the forum.

[please_try_again]

Of course, you can use tar to archive everything on the file system, just not the part of Grub on the first track. Saving a Grub bootloader in a VBR won’t help either, since it would try to load stage2 from a position that will have changed after restoring stage2 from the tar archive. And reinstalling Grub just takes a couple seconds.

[kbdeamon]

@please_try_again

Anytime from a live CD with the grub shell (the command “grub”) . The method is explained in many posts and howtos. Just search the forum.

OK:)

I rewrote grub and the system booted into gnome:)

Question: I rewrote grub quickly by following a how to. At the end (i.e., after the reboot command was entered) the system went through some processing displaying how it was shutting down but at the end did not reboot all the way and I had to do it manually. How can I tell, what if anything, went wrong?

[please_try_again]

@kbdeamon

Question: I rewrote grub quickly by following a how to. At the end (i.e., after the reboot command was entered) the system went through some processing displaying how it was shutting down but at the end did not reboot all the way and I had to do it manually. How can I tell, what if anything, went wrong?

It’s difficult to say and probably not related with the fact that you just reinstalled Grub. It happens sometimes that the system doesn’t shut down correctly with systemd. It can have many different causes.

[kbdeamon]

@please_try_again

It’s difficult to say and probably not related with the fact that you just reinstalled Grub. It happens sometimes that the system doesn’t shut down correctly with systemd. It can have many different causes.

Is there anywhere I can look to determine what the long term impact could be because now the system seems to reboot properly?

[gogalthorp]

Could be that the image you copied was in the wrong state. After rebooting things are fixed.

[kbdeamon]

@gogalthorp

Could be that the image you copied was in the wrong state. After rebooting things are fixed.

Can you please elaborate? I thought that GRUB found where it was supposed to be installed and copied itself to the appropriate location. I don’t see how this is an image problem.