How upgrade mono-disk system into a RAID5 without losing data

English Forums Feedback
#1

From one disk Installation to software RAID5 with Suse 12.2

After the second crash of my disk, recovered by backup copies, decide to have a more secure system with less job for recover data.

Decide to start to configure a software RAID5 with 3 disks.
Starting from an existent and working installation.

My configuration was:

/dev/sda – Installed non-RAID system disk
/dev/sda1 – swap partition
/dev/sda2 – root partition

|
|Adding 2 disks become:||
|
|
|
|/dev/sdb| – First empty disk for the RAID mirror|
| /dev/sdc| – Second empty disk for the RAID mirror|
| /dev/md0| – Mirrored swap partition|
| /dev/md1| – Mirrored root partition|

First of all: Backup the system, I use PartImage (Partimage) inside SystemResqueCD (SystemRescueCd) on USB stick.

Following http://www.novell.com/communities/node/4520/migrating-sles-software-raid1#degradedarray i modify:

(Modified document available: https://www.dropbox.com/s/c5wszpxo8c8udys/Novell%20Migration.rtf)

Create the degraded Array using:
mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 missing
mdadm --create /dev/md1 --level=5 --raid-devices=3 /dev/sdb2 /dev/sdc2 missing

Remember to use the already existent disk format of the existent partition (/dev/sda1), ext3, reiserfs,…for the Array partition.

mkreiserfs /dev/md1

mkfs.ext3 /dev/md1

[FONT=Helvetica]Follow: **Creating a SLES9 initrd **[/FONT]
adding raid5 module
INITRD_MODULES="raid1 raid5 mptscsih reiserfs"

Relating the Grub installation I had many errors and the system wasn’t able to reboot itself.

Reading I discover that the Array partition of RAID5 cannot be read on startup and Grub cannot find the /boot directory, also I read that Grub is not able to manage mdX partitions of RAID5.

I try many suggestions on internet forums without success, take a decision to upgrade my system to openSuse 12.3 with Grub2 stable (not in beta like in my openSuse 12.2).

First of all i have to modify the partitions table for have a boot partition outside the RAID5 Array.

My system with single disk had a 16 GB of swap, now with RAID5 swap partition you can configure 3 partitions of 8GB each that will be 16 GB in RAID5 configuration.
Delete the swap partition with fdisk.
Create, with fdisk, two new partitions (sdx1 and sdx3) of 8 GB each on each disk (you can decide different size, the boot partition need no more than 200 MB of space).

Create a new array RAID5 for swap:
create /dev/md0 --level=5 --raid-devices=3 /dev/sdb3 /dev/sdc3 /dev/sda3

Format the Array:
mkswap /dev/md0

Make active for swap:
swapon /dev/md0

Format the boot partition:
mkfs.ext3 /dev/sda1
mkfs.ext3 /dev/sdb1
mkfs.ext3 /dev/sdc1

Upgrade the system to openSuse 12.3:

Boot from LiveCD openSuse 12.3
When the update ask for configure the Grub2 select inside the boot loader location:
Custom Boot Partition > /dev/sda1
the installation goes in error with:

<< Because the partitioning the boot loader cannot be installed properly >>

doesn’t matter about it.

Reboot the system with the System Rescue CD (or with openSuse live CD)

mount the array with the new (openSuse 12.3) /boot directory:
mkdir /mnt/array
mount /dev/md1 /mnt/array

mount the boot partition:
mkdir /mnt/sda1
mount /dev/sda1 /mnt/sda1

Copy all the data inside:
cp -ax /mnt/array/boot /mnt/sda1

check if /mnt/sda1 contain the boot directory

reboot the system

Now Grub2 is doing its job and boot the system itself.

How now preserve the security of the boot partition without copy the /boot dir to each partition each change ?

Create a RAID1 array (that is in mirroring and the partitions are directly readable) with /dev/sda1, /dev/sdb1 and /dev/sdc1.

copy on /dev/sdb1 and /dev/sdc1 the /dev/sda1 contents:

mkdir /mnt/sda1
mount /dev/sda1 /mnt/sda1

mkdir /mnt/sdb1
mount /dev/sdb1 /mnt/sdb1

mkdir /mnt/sdc1
mount /dev/sdc1 /mnt/sdc1

cp -ax /mnt/sda1 /mnt/sdb1
cp -ax /mnt/sda1 /mnt/sdc1

Create a copy of the MBR of sda:
dd if=/dev/sda of=/tmp/sda_mbr.bin bs=512 count=1
copy the .bin file in a safe place (NFS Backup, USB,…)

Copy MBR on the other disks for make them bootable:
dd if=/tmp/sda_mbr of=/dev/sdb bs=512 count=1
dd if=/tmp/sda_mbr of=/dev/sdc bs=512 count=1

This command work only for same sized disks (as we need to have)

Alternative can be:
dd if=/dev/sda of=/dev/sdb_mbr.bin bs=512 count=1
dd if=/dev/sda of=/dev/sdc_mbr.bin bs=512 count=1

But i prefer to have a copy of the MBR saved in a safe place.

Create the RAID1 array without format it:
create /dev/md2 --level=1 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sda1

Reboot the system

Now you have one data RAID5 partition and one RAID1 partition for boot the system.

If one disk fail, you can change in the BIOS the alternative disk for boot and take your time for substitute the failed one.

Now I need to be advised when one disk fail and looking around i modify this script (http://www.novell.com/support/kb/doc.php?id=7001034)

#!/bin/bash

mdadm RAID health check

Events are being passed to xmessage via $1 (events) and $2 (device)

Setting variables to readable values

event=$1
device=$2

Check event and then popup a window with appropriate message based on event

if $event == ‘Fail’ ];then
xmessage “A failure has been detected on device” $device
echo “A failure has been detected on device” $device | mailx -s “Report Array” -r sender_mail@yahoo.com destination@gmail.com
else
if $event == ‘FailSpare’ ]; then
xmessage “A failure has been detected on spare device” $device
echo “A failure has been detected on spare device” $device | mailx -s “Report Array” -r sender_mail@yahoo.com destination@gmail.com
else
if $event == ‘DegradedArray’ ]; then
xmessage “A Degraded Array has been detected on device” $device
echo “A Degraded Array has been detected on device” $device | mailx -s “Report Array” -r sender_mail@yahoo.com destination@gmail.com
else
if $event == ‘TestMessage’ ]; then
xmessage “A Test Message has been generated on device” $device
echo “A Test Message has been generated on device” $device | mailx -s “Report Array” -r sender_mail@yahoo.com destination@gmail.com
fi
fi
fi
fi
#~~~~~~~~~~~~~~~~~~~~~~~~~ End of Script ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This script as to be saved as /etc/raid-events.

Add in /etc/mdadm.conf the red row:

ARRAY /dev/md/2 metadata=1.0 UUID=ec581aef:88267373:d93089ad:89ff5267 name=linux.workgroup:2
ARRAY /dev/md/1 metadata=1.2 UUID=7d0a0a37:2f6158e9:cdbbb404:a0166df8 name=linux:1
ARRAY /dev/md/0 metadata=1.0 UUID=bcbff4ac:701c47ad:aa66ad83:36465b3d name=linux.workgroup:0
PROGRAM /etc/raid-events

The script will open a popup window on your screen and send a mail when disk failure happen.

Test it:
mdadm --monitor --scan --test

If you need to load the monitor on boot:
edit /etc/init.d/boot.local adding the row:
mdadm --monitor --scan --daemonize

For permit mailx to send mail through an external SMP edit /etc/mail.rc and add the red row:
set asksub append dot save crt=20
set smtp=smtp.domain.com (with the SMTP of your domain)
ignore Received Message-Id Resent-Message-Id Status Mail-From Return-Path Via

Now my system is up and running and SAFE.

#2

Sorry, I tried to read your post and to get to the question/problem you have, but I failed. This may partly be due to me being a bit impatient, but you did not make the reading easier by not using CODE tags around your computer texts. You get them by using the # button in the toolbar of the post editor.

Your main question seems to be about how to reorganise your disks without loss of data. Which in general is rather easy: make a good (extra) backup, reorganise your disks and restore from the backup. But yoy sem to have a special psroblem question deep in there. Can you pleae point to the real problem/question.

#3

sorry, is only a document how do the upgrade, no question in it.

#4

On 2013-04-02 19:56, simgambu wrote:
>
> sorry, is only a document how do the upgrade, no question in it.

There is a forum dedicated to posting documents.
But you have to do it readable, code tags are essential.

However, I noticed this sentence:


Follow: *Creating a SLES9 initrd *

Then you must remember that this forum is dedicated to openSUSE.
SLES/SLED has its own forum:

SLES/SLED forums

Same login as here.

–
Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

#5

ok, i’ll move the document.

SLES9 is present only cause i take ideas from a Novell Suse SLES doc.

simone

#6

Oh boy. Please, next time browse through all of our (sub)forums and read what they sau about what should go in them.

No you can not move this. I will do.

This is CLOSED for the moment.

#7

Moved from Install/Boot/Login and open again.

#8

Something to consider for anyone who wants to deploy on RAID…

As inexpensive as hardware controllers can be, there are many performance and management benefits to implementing Hardware RAID instead of Software RAID.

Despite the many “How To” describing setting up software RAID, it should be considered when the hardware option is impossible.

If deployed on hardware RAID, typically the OS is presented a single disk volume which can be installed and managed as though there is a single disk (unless you want to modify the RAID configuration). All RAID complexities become irrelevant in the OS when configured this way.

And, as always whenever moving from one type of disk subsystem to another, you will almost always want to start with a clean RAID array first, then restore the data to the new array from backup.

IMO,
TSU

#9

On 2013-04-03 19:36, tsu2 wrote:
> If deployed on hardware RAID, typically the OS is presented a single
> disk volume which can be installed and managed as though there is a
> single disk (unless you want to modify the RAID configuration). All RAID
> complexities become irrelevant in the OS when configured this way.

Real hardware raid is, of course, faster. The problem is if the
controller hardware fails, it has to be replaced with the same card if
you want to recover the hard disks. Often the format is proprietary, so
you can not replace with a different card.

As always, raid of any type doesn’t protect from all disasters. In
particular, keep a good backup. If you don’t have the hardware for both
raid and backups, renounce to use raid and use the backup instead. IMNSHO.

–
Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

#10

Not just faster.
I’m sure there are plenty of articles on the Internet that compare software and hardware RAID.

Besides performance (onboard chip handling RAID overhead instead of in regular CPU)

Probably the biggest reason why no truly competent SysAdmin would consider software RAID (and therefor YMMV is you’re not designing to Enterprise standards),

  • Onboard battery in better RAID controllers. Why? - Because in the event of a sudden power loss, if your data is really valuable you have to value ATOMIC transactions, in lay language ensuring that data in flight gets committed or rolled back. You can’t have data in mid-transaction due to a power loss, it would corrupt your database then requiring recovery efforts. Before you say “But I’m not running a major data store!” consider all the Desktop applications you’re running that’s using MySQL (eg Amarok) or PostgreSQL. They’re all exposed to corruption and are a big ball of frustration if you’ve never recovered a database before. Yeah, learn when you have no choice when it could have been cheaply avoided.

Also consider
With many hardware RAID, they come with management software which setup, convert, and recover with a click.
Many if not most support “Hot Plug” replacements, which means you don’t need to shutdown to remove/replace/add a disk. Just insert your new drive, and run the management software. AFAIK this cannot be supported by software only solutions.
Most I’ve used automatically integrate, and monitor using SMART and more. I know of software SMART tools, but so far haven’t used any that continuously monitor your RAID, and notify when detecting impending failure. Even if possible, do you want it using CPU cycles?
As I’ve described, the OS only sees the logical volume, so at the OS level everything is simple. No complications with LVMs, Disk IDs, more.

I’ve seen RAID cards start at about $60 US. Might be cheaper on the gray market.
Hard to argue against the benefits of hardware RAID.

TSU

#11

On 2013-04-07 02:06, tsu2 wrote:
>
> robin_listas;2544238 Wrote:
>
> Not just faster.
> I’m sure there are plenty of articles on the Internet that compare
> software and hardware RAID.
>
> Besides performance (onboard chip handling RAID overhead instead of in
> regular CPU)
>
> Probably the biggest reason why no truly competent SysAdmin would
> consider software RAID (and therefor YMMV is you’re not designing to
> Enterprise standards),

But the OP did not specify that he was talking about an enterprise
situation - and anyway, I’m aware of all those advantages you talk about :slight_smile:

…

> I’ve seen RAID cards start at about $60 US. Might be cheaper on the
> gray market.

Not FAKE raid?

> Hard to argue against the benefits of hardware RAID.

I know that hardware raid, if real, is better.

I like software raid because it is more flexible in choices. If the raid
card goes down, I have seen people lose the entire array because the
replacement can not use the existing disks. It needs a recreate from
scratch.

Of course, in a good enterprise situation you would have replacement
hardware for all the pieces.

With software raid you can do things so weird as using disks of
different manufacturers and sizes, if you must.

On the other hand, I have seen people (even enterprise) setting up raid
“for security” or because they talk them into it, and forego having backups.

–
Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

#12

I agree with all of you.
I create a Software RAID5 with open SUSE only for speed up the recovery time.

I am not an enterprise, we are using openSUSE, we are in three workers putting docs and nightly backups on “the server”

Our last crash forced us to stay off the server for 5 hours (during the restore), now with the soft RAID we have only to power off the server, change the disk, switch on the server, copy the partitioning with dd on the new one, make it active and start to work (also during the sync of the disks).

Happen one time and we spent 20 minutes for do all.

So for us, soft RAID, is enough and sufficiently sure.

Thanks to all.

cheers,
simone

#13

On 2013-04-08 16:16, simgambu wrote:
> So for us, soft RAID, is enough and sufficiently sure.

Ok… but do not forget backups :slight_smile:

–
Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

#14

[QUOTE=robin_listas;2545745]On 2013-04-08 16:16, simgambu wrote:
> So for us, soft RAID, is enough and sufficiently sure.

Ok… but do not forget backups :slight_smile:

Yes, sure. Tnx