Rebuilding Array (Intel Matrix)

[NhatPhongTran]

Hi!

I got opensuse 11 running on a dedicated HDD and got another 4 HDDs created as RAID 10 via the Intel Matrix RAID Controller of the ICH10 southbridge. I simulated a device failure and disconnected one of the drives and during POST the controller simply says that I need to rebuilt the array from the OS. So how am I supposed to do that since there were no usable raid utitlities for openSuSe delivered out of the box.

Thanks very much!


Phong

NhatPhongTran wrote:

> I got opensuse 11 running on a dedicated HDD and got another 4 HDDs
> created as RAID 10 via the Intel Matrix RAID Controller of the ICH10
> southbridge. I simulated a device failure and disconnected one of the
> drives and during POST the controller simply says that I need to rebuilt
> the array from the OS. So how am I supposed to do that since there were
> no usable raid utitlities for openSuSe delivered out of the box.

Use the same tools you used to create the raid level.
Probably "dmraid" -device mapper- tools (man dmraid) or "md" (man md).

Usually, motherboard raid bios has also the option to rebuild arrays. Be
*very* carefull with this, sir. First read the docs.

Greetings,

--
Camaleón

[NhatPhongTran]

Hi!

Thank you for the quick reply. The BIOS doesn't have an option for rebuilding arrays. It says whenever the status goes to 'degraded' that I need to rebuild from OS.
I built the array using the RAID configuration setup during POST, so I did not use dmraid or md or anything like that. So what tools do I need then to rebuild my array?

NhatPhongTran wrote:

> Thank you for the quick reply. The BIOS doesn't have an option for
> rebuilding arrays. It says whenever the status goes to 'degraded' that I
> need to rebuild from OS.
> I built the array using the RAID configuration setup during POST, so I
> did not use dmraid or md or anything like that. So what tools do I need
> then to rebuild my array?

Mmm, IIRC raid setup has to be done in both places: bios and OS.

If you did not setup your hard disks to use raid when installed and
partitioned with openSUSE, then the system is not aware of your raid level.

openSUSE provides two tools to manage and create software raid ("dmraid"
or "md"). First thing you have to know is how did you configure your raid
within your system and then use the same tools.

Anyway, by looking into Intel matrix papers, it seems the reconstruction has
to begin as soon as you connect the drive and restart the system.

RAID 10 Volume Recovery
http://support.intel.com/support/chi.../CS-021017.htm

Sir, raid management is a very delicate task. You can lose you data. Be sure
to completely understand the steps involved in a rebuild operation.

Greetings,

--
Camaleón

[madhusker]

It looks as if dmraid rebuilds are NOT supported for Intel Matrix in version 1.0.0.rc15. Here is output of trying to rebuild a Raid 1 mirror on an Asus P6T Deluxe v2.

# dmraid -R isw_bddgbciihb_Server

Volume "isw_bddgbciihb_Server" is not in rebuild state (current: 4)
Rebuild: cannot rebuild from current state!

This shows the mirror is ok, but its NOT.

# dmraid -s
*** Group superset isw_bddgbciihb
--> Active Subset
name : isw_bddgbciihb_Server
size : 1953519872
stride : 128
type : mirror
status : ok
subsets: 0
devs : 2
spares : 0
*** Group superset isw_ecfjdjhfcc
--> Active Subset
name : isw_ecfjdjhfcc_Spare
size : 3907039744
stride : 256
type : stripe
status : ok
subsets: 0
devs : 2
spares : 0

Everything written to disk after the failure was not save. It seems it was maybe put into these files??

# ls -l
total 4
drwxr-xr-x 2 root root 4096 2009-07-05 10:07 dmraid.isw
root@:~# ls -l dmraid.isw/
total 48
-rw------- 1 root root 512 2009-07-05 11:00 sda.dat
-rw------- 1 root root 14 2009-07-05 11:00 sda.offset
-rw------- 1 root root 11 2009-07-05 11:00 sda.size
-rw------- 1 root root 512 2009-07-05 11:00 sdb.dat
-rw------- 1 root root 14 2009-07-05 11:00 sdb.offset
-rw------- 1 root root 11 2009-07-05 11:00 sdb.size
-rw------- 1 root root 512 2009-07-05 11:00 sdc.dat
-rw------- 1 root root 14 2009-07-05 11:00 sdc.offset
-rw------- 1 root root 11 2009-07-05 11:00 sdc.size
-rw------- 1 root root 512 2009-07-05 11:00 sdd.dat
-rw------- 1 root root 14 2009-07-05 11:00 sdd.offset
-rw------- 1 root root 11 2009-07-05 11:00 sdd.size

~# dmraid -r

/dev/sdd: isw, "isw_ecfjdjhfcc", GROUP, ok, 1953525166 sectors, data@ 0
/dev/sdc: isw, "isw_ecfjdjhfcc", GROUP, ok, 1953525166 sectors, data@ 0
/dev/sdb: isw, "isw_bddgbciihb", GROUP, ok, 1953525166 sectors, data@ 0
/dev/sda: isw, "isw_bddgbciihb", GROUP, ok, 1953525166 sectors, data@ 0

[NhatPhongTran]

Raids indeed seem to be a very delicate task. I'll take your advice in consideration and abandon the raid container and backup manually instead. What Raid controller can you recommend which can rebuild from ROM without needing an OS running? One with which you got good experience. I'm looking for a Raid controller around $ 300-400.

[madhusker]

I don't have experience with real hardware raid and was somewhat fooled into buying this board thinking it has some type of cheap hardware raid that was a little better than software. I've used software raid for my home servers for years with no trouble at all but for critical and busy servers you have to go true hardware raid (like you intend to do). For this P6T I am going to revert back to mdadm managed software raid for my application since it's not critical here at home, but I do need some redundancy. I have an incremental rsync backup script I will backup my software raid to over the local network or external drive.

NhatPhongTran wrote:

> Raids indeed seem to be a very delicate task. I'll take your advice in
> consideration and abandon the raid container and backup manually
> instead. What Raid controller can you recommend which can rebuild from
> ROM without needing an OS running? One with which you got good
> experience. I'm looking for a Raid controller around $ 300-400.

madhusker's advice is one option.

"md" software raid is a very good option if you want to use a raid level in
linux and you don't even need a real hw raid card to manage it. It is
reported to be very stable (I personally have not tested) and has a smooth
setup under openSUSE. The pros of this solution is easy of manegeability
and non hardware dependant. The cons is just a bit of work to do when the
array goes down :-)

Another option is going with real hardware raid card. I use on my servers
Adaptec 2020sa (zero channel raid card) but not sure if I can recommend
this manufacturer, I'm not so happy with the adapter itself :-/. Areca,
3ware, ICP, ITE and LSI branded card used to be a good options, although
they are a bit expensive.

Remember that hardware raid cards also need to be fully supported by the
manufacturers driver in order you can fully manage inside openSUSE the
rebuild, creation and monitoring of the raid level status. Look for any
card that works 100% with the version of openSUSE you have installed and
that will support future version of openSUSE (just note that openSUSE is
not SLES).

You have some raid cards listed in openSUSE wiki:

http://en.opensuse.org/HCL/RAID_Controllers

Another very good pages, plenty of useful info:

http://linux-ata.org/faq-sata-raid.html
http://linuxmafia.com/faq/Hardware/sata.html

THT... and sorry for the delay :-)

Greetings,

--
Camaleón