I think the OP has decided to do something else, anyway, so it now all becomes a moot point.:\
FB >> I did not decide to do something else per see, and even if I did I would still want to figure out what happened to be prepared and guard against something like this in the future.
as to clarify things for some other posters, just in case it makes spotting the possible cause of the issue.
I designated 2 1T drives (one seagate and one WD) to be a system drives. as in all system related stuff going on them.
both drives are connected to the MB SATA ports
segate to port 0
WD to port 1
all other drives (3 in counting so far) are strictly DATA drives and have not/ should not have anything to do with booting or system. they were added later on after the system setup.
they also are connected to one of the 8 port SASLP cards
ALL drives are using GPT. not MBR.
the current system is a dual opteron CPU setup with 56 GB of RAM
thus during initial setup (that is when I booted the system from DVD and run the new install)
I chose to have BTRFS as default FS and separate /home partition
and used custom partitioning utility to
a. create a “/” partition on seagate with full drive - 100GB
and a 100GB swap partition
b. mark this drive as bootable.
c. create a “/home” partition on WD drive again full drive size less 100GB
and a 100GB swap partition on it.
I did a clean install of the system on this.
the only stupid thing was creating swap on the home drive. I am sure I have some sick reason behind it but can not remember now why.
after install and full update, (in fact I run updates several time after the install of some drivers and after install of XEN)
I have run Xen setup ever since. and had rebooted the system several time since than as well.
all this have worked perfectly until last week when I run updates on the system( zypper up)
the system run for couple of days after the update as I was moving some files from my old file server onto the new share on this file server at the time (big media files, multiple transfers )
once this was done I did the reboot and here we are 
even though I have a functioning setup with this system, it lack of full IOMMU support prevents me from setting some windows VM and doing some other specialty VMs setup I want.
I come upon a deal for an newer 8 core CPU and MB that has full IOMMU support and decided I will rebuild this system based on that hardware instead. yes it has some issues as it will be a downgrade in system capability (I currently have dual CPU with 12 cores and 56GB RAM where new system will have a single CPU and 8 core only and for a wile I will have to do with 16GB of RAM and could only go to 32 later) but full virtualization support is more important for me, and since my plan will not really tax the system that much anyway it is a good plan IMHO. I may be able to recover some cash for the current hardware as well or perhaps re-porpoise the MB/RAM for something else.
so here we are.
thanks to you all.
V “Live well and prosper”
You mentions everything except the most important the type of RAID
As I recall it was BTRFS For BTRFS RAID you must consider it BETA at best. For somethings in BTRFS they work great for others not so much.
If you are looking for stability I’d take another approach. If your want to test some of BTRFS.s more exotic modes you will need to allow for problems which of course should be reported to bugzilla
On 2014-05-27 17:26, vl1969 wrote:
> I designated 2 1T drives (one seagate and one WD) to be a system drives.
> as in all system related stuff going on them.
>
> both drives are connected to the MB SATA ports
> segate to port 0
> WD to port 1
>
> all other drives (3 in counting so far) are strictly DATA drives and
> have not/ should not have anything to do with booting or system. they
> were added later on after the system setup.
> they also are connected to one of the 8 port SASLP cards
And theses data drives are a raid?
btrfs raid is kind of flaky, I heard. But the system disks are not those
3, so booting should not be affected.
> ALL drives are using GPT. not MBR.
Mmmm… GPT disks do have an MBR, called protective MBR, so it could be
used (and maybe cause problems?)
If you have BIOS (improbable) it would use the MBR, but an UEFI would
expect the efi boot partition marked as such instead, in a GPT
partition. But an UEFI system must still be able to boot from a MBR boot
code, if present… So, somehow it has to decide which one to use.
> the only stupid thing was creating swap on the home drive. I am sure I
> have some sick reason behind it but can not remember now why.
I do not see why that would be stupid at all :-?
> V “Live well and prosper”
Heh…
I have the new version of Star Trek on the telly at this minute 
–
Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 “Bottle” at Telcontar)
well here are some clarification, not in the order they been ask though.
a. the System drives are not raided. I have single 1TB drive hosting “/” root partition and what ever makes it boot-able
on a GPT. and I have a single drive 1TB hosting the “/home” partition , I have selected separate home partition option during install.
it did strike me as interesting that on the install I was not asked to create the “/boot” partition as it was needed before, and yet the system had installed and booted just fine. maybe that was my mistake and I should have manually partition the first drive as I did before with “/boot” = 1GB + “/” =9899 + “/swap” = 100GB
b. I say it was stupid to add the swap partition to the second system drive, because as far as I can tell the system only uses one swap partition unless you use raid or LVM to combine the space on several devices into a single volume.
and it used the one on the system drive by default on my setup.
c. the data drives are NOT the issue here. they work and actually work well IMHO.
granted I haven’t encounter issues with them thus can not say anything about the stability/data recover ability or anything else of this matter, but I have the volumes that present them self as I expect them to and data is moving in and out just fine.
all drives are used with BTRFS in Raid 1 config . this gives you the protection against data rot on a single drive,
ofcouse a drive failure will take the data out.
as far as I can tell BTRFS Raid only consider raid 5 and 6 as Beta at the moment.
RAID 0, 1,10 are fully supported(unless I missed something on the WiKi page.)
in fact any BTRFS setup even a single drive setup uses default raid configuration.(Raid 1 for metadata and Raid 0 for data)
which you can override to use Raid 1 for both or Raid 0 for both (the single mode) which would be not advisable as you are not protected at all with Raid0 level, not even sure if you can use raid0 level for metadata.
Yast/partitioner however does not support
multi-device BTRFS raid setup in any form. it will not recognize the drives as having the FS at all and present them empty in the list.
only data drives are using any kind of raid config(in my case a multi-device raid1 ) other than default.
I think I had figured out my issue.
since I did not setup separate “/boot” partition to boot from the last update must have overrode my boot data to default and now nothing is found. maybe my fstab file was changed as well.
I think when I rebuild the system I will create the “/boot” partition manually and use that to boot instead.
this should protect against updates overriding the boot data in the future, right?
It is not so much BTRFS but the myriad of other things that have to work with it. Also in the past what the developers think as working ie no longer beta may or may not work in the real world.
So an update broke things maybe because something that had to reconfigure the boot did not understand RAIDed BTRFS.
As I said if you are looking for stable then don’t use a complex BTRFS configuration. If you want to test have at it. The community always need people willing to try stuff and report results to the developers on bugzilla.
On 2014-05-28 15:36, vl1969 wrote:
>
> well here are some clarification, not in the order they been ask though.
>
>
> a. the System drives are not raided. I have single 1TB drive hosting
> “/” root partition and what ever makes it boot-able
> on a GPT.
A single root partition, no more partitions on that drive? Wait, I think
you posted fdisk output earlier… yep.
> Device Boot Start End Blocks Id System
> /dev/sde1 * 2048 1743808511 871903232 83 Linux
> /dev/sde2 1743808512 1953523711 104857600 82 Linux swap / Solaris
> /dev/sde4 1 1 0+ ee GPT
> Partition 4 does not start on physical sector boundary.
>
>
> Partition table entries are not in disk order
>
>
> Disk /dev/sdf: 1000.2 GB, 1000204886016 bytes, 1953525168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disk label type: dos
> Disk identifier: 0x00000000
>
>
> Device Boot Start End Blocks Id System
> /dev/sdf1 2048 1743808511 871903232 83 Linux
> /dev/sdf2 1743808512 1953523711 104857600 83 Linux
> /dev/sdf4 1 1 0+ ee GPT
Well, sde is home (I see swap in there), and sdf is your boot disk (with
two partitions). Well, you have a big problem here…
Neither are GPT partitions, not pure GPT. They are hybrids, they contain
valid classical partition data, which may match or not the separate GPT
data. I suggest you use “gdisk -l /dev/sdf” to find out.
A pure GPT disk should show like this:
Telcontar:~ # fdisk -l /dev/sda
Disk /dev/sda: 3000.6 GB, 3000592982016 bytes, 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sda1 1 4294967295 2147483647+ ee GPT
Partition 1 does not start on physical sector boundary.
Telcontar:~ #
Telcontar:~ # gdisk -l /dev/sda
GPT fdisk (gdisk) version 0.8.7
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): FFB65404-E3F0-47FB-9F1A-BB36FC130FF0
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2925 sectors (1.4 MiB)
Number Start (sector) End (sector) Size Code Name
1 2048 307202047 146.5 GiB 0700
2 307202048 614402047 146.5 GiB 0700
3 614402048 921602047 146.5 GiB 0700
4 921602048 1228802047 146.5 GiB 0700
5 1228802048 1536002047 146.5 GiB 0700
6 1536002048 1843202047 146.5 GiB 0700
7 1843202048 2150402047 146.5 GiB 0700
8 2150402048 2457602047 146.5 GiB 0700
9 2457602048 2764802047 146.5 GiB 0700
....
The problem is that the MBR table on a GPT disk must not be used as
such. It must contain a single partition, type “GPT”, that when seen by
tools that can not handle GPT means “bail out”.
Notice, in my data, that fdisk sees only one GPT partition,
called “protective partition”, and that it is smaller than
the full disk because the max size is 2 TiB. The real
partitions are only seen with gdisk. The idea is that
traditional tools see the disk as fully used, no new
partitions can be made, and bail out.
In your case, the GPT protective partition in the protective MBR has
been “destroyed”, it is empty.
I don’t know what grub will do in this case…
Also, it is missing the dedicated EFI boot partition (or it is in the
missing GPT table that I can not see).
I would seriously consider a proper reinstall.
Notice that, as your system disks are smaller than 1 T, you can use
traditional MBR partitions alone, no need for GPT - unless you double
boot with Windows 8.
> it did strike me as interesting that on the install I was not asked to
> create the “/boot” partition as it was needed before, and yet the system
> had installed and booted just fine. maybe that was my mistake and I
> should have manually partition the first drive as I did before with
> “/boot” = 1GB + “/” =9899 + “/swap” = 100GB
Grub is able, so they say, to boot from btrfs now, so a separate boot is
not needed. However… I would use it, formatted as ext2. I’m not
confident that grub supports fully btrfs.
> b. I say it was stupid to add the swap partition to the second system
> drive, because as far as I can tell the system only uses one swap
> partition unless you use raid or LVM to combine the space on several
> devices into a single volume.
> and it used the one on the system drive by default on my setup.
No, you can have as many swap partitions as you want, in any disk you
want. You can set different priorities on each, so that they are filled
in sequence, or have the same priority, so that the kernel distributes
blocks equally amongst them.
They can be independent, on LVM, or on RAID. No matter.
(data drives)
> all drives are used with BTRFS in Raid 1 config . this gives you the
> protection against data rot on a single drive,
> ofcouse a drive failure will take the data out.
AFAIK, corruption on a raided filesystem, corrupts all sides of the raid
simultaneously. I don’t really know how btrfs affects this, but I expect
you have the snapshots to go back to a previous version of the data - as
long as the filesystem is not corrupted.
–
Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 “Bottle” at Telcontar)
well robin_listas there is where problem lays.
I did not create this layout.
during install I choose use BTRFS default, use separate /home partition.
than go into custom partitioning
select create new disk layout for each of the system disks
choosed GPT as type of the disk layout.
than add partition with type BTRFS, and mount point “/” size = 900GB (well not really but what ever is equal to 1TB less 100GB)
than add second partition type swap mount point /swap(not sure if it actually asked for mount point, I think not.)
same setup for the second disk only creating “/home” partition and swap partition on it.
after that continue setup and the install process did everything else. on the final screen it showed the layout and selected option to boot from “/” not MBR.
I can only assume that as part of supporting boot from BTRFS the setup creates the /boot sub-volume with in the system and pseudo MBR space on the disk to redirect into it. and something corrupted that link during the update.
I think that even though I really like the BTRFS snapshot option, I will rebuild with EXT4 on a single drive and keep the second drive as hot spare with periodic full disk copy on to it. using dd for full disk cloning. will need to research this for how-to but I have seen it mention as a possibility somewhere.
Hm?
I would rather say, sdf1 is home, and sde1 is the / partition.
sdf2 would be the second swap partition of which he was talking, albeit not marked as swap, that’s why it was not used as swap.
@vl1969:
As Carlos wrote later on it is perfectly possible to have and use two swap partitions. The reason why it didn’t work, is that this one is not marked as swap and not formatted as swap.
And you shouldn’t a separate /boot either.
> it did strike me as interesting that on the install I was not asked to
> create the “/boot” partition as it was needed before, and yet the system
> had installed and booted just fine. maybe that was my mistake and I
> should have manually partition the first drive as I did before with
> “/boot” = 1GB + “/” =9899 + “/swap” = 100GB
Grub is able, so they say, to boot from btrfs now, so a separate boot is
not needed. However… I would use it, formatted as ext2. I’m not
confident that grub supports fully btrfs.
grub2 works just fine with/on btrfs.
It has one limitation though, at least the version in 13.1: it cannot write to btrfs.
So if you use grub-once to set a entry to boot automatically on the next restart (hibernating does that f.e.), grub2 cannot reset this.
Therefore you will never see the boot menu again…
>>>>
AFAIK, corruption on a raided filesystem, corrupts all sides of the raid
simultaneously. I don’t really know how btrfs affects this, but I expect
you have the snapshots to go back to a previous version of the data - as
long as the filesystem is not corrupted.
<<<<<
BTRFS raid is not the same as your regular raid or even soft raid (mdam)
BTRFS does not simply mirror the drives in raid 1 for example. it mirrors data in predefined chunks across all/any 2 devices available. so you can actually have 2,3,4,5 device raid 1 volume using BTRFS where devices can also be of different size.
where in traditional raid systems, hardware or software the devices must be of the same size
the size of the volume in this setup is roughly equals to half the sum of the device capacity +/- additional space for metadata and such.
so if you have 2x3TB drives setup with BTRFS Raid 1 you will end up with 3TB available space but actual usable space maybe smaller as snapshots accumulate.
you can adjust the chunk size used via CLI but only for raid1 and 10 I think.
so.
if you have a BTRFS raid 1 volume on 3 devices d1/d2/d3
and save several files to the volume f1/f2/f3
it can put file1(f1) on d1 and second copy on d2
for file 2(f2) it can put one copy on d1 and second one on d3
I think this is better explained here https://btrfs.wiki.kernel.org/index.php/FAQ raid explanation is close to the bottom of the page
that’s the thing, I did not do anything to it.
and my server does not hibernate.
it have survived several reboots just fine until I run latest update.
not really sure what happened.
Well, that problem wouldn’t have affected you.
It boots just fine in that case, just doesn’t show the boot menu so you can’t boot to anything else any more.
it have survived several reboots just fine until I run latest update.
not really sure what happened.
Hard to say.
But I still think that your BIOS tried to boot from a disk without boot loader code.
Well, might even be caused by a weak CMOS battery (you have many disks and are apparently not booting from the first one), so the update might just have been a coincidence.
OTOH, of course the boot loader got re-installed during the updates. So something could have been messed up there.
You should probably check the settings in YaST->System->Boot Loader (especially “Details”).
But maybe you should choose to install the boot loader to the MBR there, might be safer in your case (that’s no dual boot system anyway)
On 2014-05-28 22:36, wolfi323 wrote:
> OTOH, of course the boot loader got re-installed during the updates. So
> something could have been messed up there.
Well, those system partitions are messed up, neither GPT nor MBR.
Se my previous post about that…
–
Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 “Bottle” at Telcontar)
well, it’s all moot now anyway.
I am swiping the MB and CPU on this machine to Asrok extream3 and FX8320 8 core cpu
and loading the XenServer on it. will run my opensuse file server as a VM, leaving the virtualization to the real world hypervisor
loosing some RAM and cpu cores in the process but gaining full IOMMU support, a fair trade I think.