Restoring the entire ETC directory in similar hardware

We are looking for a solution that allows us to restore a servers “identity” in the event of hardware failure…ie Name, IP address, dhcpd, named configs etc.

We have ISO images of the appliance images that contains all the software and stuff. There is no real data on these servers.

If we backup the entire ETC directory and perhaps the /var/lib can we simply restore that and get everything back?

What concerns me is how some versions of Suse use UUID for the network config and the mounts points.

Has anyone tried this?

Backing up /etc/ and restoring it back down after a hardware replacement will work fine.

Just be sure to avoid using “disk by-id” entries in things like the fstab.

For example don’t have something like this in your fstab


/dev/disk/by-id/scsi-36001e4f0215a65000f4ea6a604c134b6-part3 /                    ext3       acl,user_xattr        1 1

Use this


/dev/sda3 /                    ext3       acl,user_xattr        1 1

What are you planning on backing up in /var/lib?

Good luck,
Hiatt

On 2011-05-04 18:36, mww wrote:

> If we backup the entire ETC directory and perhaps the /var/lib can we
> simply restore that and get everything back?

And perhaps the grub configuration in /boot.

However, what you need is a script that backups everything that changed
from the initial configuration, except the data directories that do not
interest you.

> What concerns me is how some versions of Suse use UUID for the network
> config and the mounts points.

You can use other things, like LABEL, instead of UUID. Do not use device
names like /dev/sda.


Cheers / Saludos,

Carlos E. R.
(from 11.2 x86_64 “Emerald” at Telcontar)

After you restore /etc you should also clear out the udev rule for your network devices if you replace a NIC or a motherboard with an onboard NIC


cat /dev/null > /etc/udev/rules.d/70-persistent-net.rules

jthiatt08 wrote:
> Just be sure to avoid using “disk by-id” entries in things like the
> fstab.
>
> For example don’t have something like this in your fstab
>
> Code:
> --------------------
>
> /dev/disk/by-id/scsi-36001e4f0215a65000f4ea6a604c134b6-part3 /
ext3 acl,user_xattr 1 1
>
> --------------------
>
>
> Use this
>
> Code:
> --------------------
>
> /dev/sda3 / ext3 acl,user_xattr 1 1
>
> --------------------

Carlos E. R. wrote:
> On 2011-05-04 18:36, mww wrote:
>
>> If we backup the entire ETC directory and perhaps the /var/lib can we
>> simply restore that and get everything back?
>
> And perhaps the grub configuration in /boot.
>
> However, what you need is a script that backups everything that changed
> from the initial configuration, except the data directories that do not
> interest you.
>
>> What concerns me is how some versions of Suse use UUID for the network
>> config and the mounts points.
>
> You can use other things, like LABEL, instead of UUID. Do not use device
> names like /dev/sda.

I think it’s time for a review of disk/partition identification to make
it clearer.

jthiatt08 is correct in saying not to use disk-by-id, which
unfortunately is what opensuse uses by default. That’s a value tied to
the particular disk that’s in use.

But jthiatt08 is wrong, as carlos says, to suggest using /dev/sda. Those
names should no longer be used because of the way the linux kernel scans
disks nowadays. They can theoretically change between boots and
certainly on apparently identical systems that have a minor difference.

LABEL is good. You can assign arbitrary human-readable names.

UUID is also good. The system assigns a meaningful but unreadable
identifier, but this can be copied from disk to disk and system to
system. It’s what Ubuntu uses by default. The only problem is that it
can get pretty confusing.

As for the nitty gritty details, there’s no substitute for experiment.
Set up your proposed method and test it a few times

But jthiatt08 is wrong, as carlos says, to suggest using /dev/sda. Those
names should no longer be used because of the way the linux kernel scans
disks nowadays. They can theoretically change between boots and
certainly on apparently identical systems that have a minor difference.

That’s interesting.
I have been managing 1200+ Linux servers for the past 4 years and I’ve never seen this behavior.
All the servers are have identical hardware and even have an external USB drive attached.
They always boot to the device that I specified in the fstab and the menu.lst.
What devices, other than /dev/sdX, will the kernel point me to?
It sounds like I will need to make changes to all the systems to prevent a boot failure.

@mww - The more I think about it, I think you should scrap the idea of backing up directories.
Identify which files need to change on the system and then write a script to change those files to your liking.

Good luck,
Hiatt[QUOTE][/QUOTE]

Thanks for all the tips.

I’ll look at using label or uuid.

On 05/05/2011 05:06 PM, jthiatt08 wrote:
>
> That’s interesting.
> I have been managing 1200+ Linux servers for the past 4 years and I’ve
> never seen this behavior.
>

i’ve not seen it either (i’m on 11.3) so perhaps it is in 11.4 (and
kernel), and we get to sweat it out for a year or two before it makes it
into a SLES SP…maybe…

things do change…


CAVEAT: http://is.gd/bpoMD
[openSUSE11.3 + KDE4.5.5 + Firefox3.6.17 + Thunderbird3.1.10 via NNTP]
HACK Everything → http://www.youtube.com/watch?v=j5b4CCe9pS8&NR=1

You are right DenverD, things do change.
I’m glad I learned about this change now, it would have really bit me in the future.
I will be using UUID from now on.

Thanks,
Hiatt

The problem is in devices with plug and play type situation with pluggable drives. If you have a stable drive system /dev/sda notation is fine . USB and mixing IDE and SATA drive is also another good way get unstable BIOS labeling. In a production environment this should never be a problem if things are done right. /dev/sda notation is useful in cloning situation where you have firm control over disk discovery order.

If we bother to label the partitions LABEL_ID is the best way to keep things straight.

On 2011-05-05 17:06, jthiatt08 wrote:
> I have been managing 1200+ Linux servers for the past 4 years and I’ve
> never seen this behavior.

I have seen it. Mine does change.
It depends on the hardware.


Cheers / Saludos,

Carlos E. R.
(from 11.2 x86_64 “Emerald” at Telcontar)

jthiatt08 wrote:
> That’s interesting.
> I have been managing 1200+ Linux servers for the past 4 years and I’ve
> never seen this behavior.
> All the servers are have identical hardware and even have an external
> USB drive attached.
> They always boot to the device that I specified in the fstab and the
> menu.lst.

What happened with previous kernels is no guide to what can now happen. :slight_smile:

The kernel can scan controller ports in parallel and it assigns drive
letters as it recognizes the drive, so it can also depend on how fast
individual drives spin up. I have one system where opensuse assigns the
usb disk after the sata disks whilst ubuntu does the opposite (different
kernel versions). Ditto for a machine with mainboard ports and plugin
adapters. Device letters change if the USB drive is not plugged in.
Device letters change if one hot swap disk is not present when the
machine is booted. etc etc.

I’ve never yet seen a change between boots of the same OS on the same
hardware, but AFAIK it’s theoretically possible.

Since you can avoid all these possible problems by using any of labels,
ids, or uuids, it seems a no-brainer to me.

I freely confess it did take me a while to stop being annoyed at the
kernel devs for having ‘broken’ my cosy reliable hd/sd world though. :slight_smile:

> What devices, other than /dev/sdX, will the kernel point me to?

Sorry, I don’t understand that question.

jthiatt08 is correct in saying not to use disk-by-id, which
unfortunately is what opensuse uses by default. That’s a value tied to
the particular disk that’s in use.
This has puzzled me for some time, any idea why that unfortunate default ?

I totally agree. Problems like the one described in this thread Grub dual boot error would have been fixed in 2 minutes if the user had mounted by UUIDs rather than disk-by-id.

If 1200 servers have the same hardware, they pretty much count for 1. That’s a very well known behaviour due to the asynchronous way the Linux kernel (since 2.6) loads modules, causing udev to generate devices in a random order. It actually doesn’t just do that with hard disk devices. If you boot 10 times from a live system or install CD a machine with several netcards, you will notice that eth0, eth1, eth2, etc do not always refer to the same device. The same goes for hard disks, while mixing SATA and IDE or booting while external USB drives are plugged in - and this is particularly nasty if you update a kernel when the USB drive has become the first one since the scripts which rewrite the Grub menu under openSUSE make use of the device.map (!!!) It just hit me (again) the other day.