Installation and Partition Configuration with both Hard Drives and NVME SSD

English Install/Boot/Login
1

I am planning the installation of a Leap 15.3 system which has both hard drives and NVME SSD. The hard drives are configured as a single RAID array with hardware raid card and the NVME SSD is on a PCIe M2 adaptor. This appears as:-

Disk /dev/nvme0n1: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: KINGSTON SNVS2000G                      
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

.

What I seek is to take advantage of the speed of the SSD and my present thinking is to have the “working” partitions of /,/home and swap on the SSD and have the hard drive as multimedia data which for me takes up significant amounts of space but does not have to work too hard. (I am not a gamer!).

The only exception to this might be if I wish to use the machine to store the database for my digiKam which I understand can have performance issues if saved on NAS and is much better on workstation.

I would appreciate advice and suggestions for partition plan.

Finally, on the ssd, this is shown as /dev/nvme0n1. First time I have tried all this so how is this shown if I divide the drive into partitions? How do they appear as in /dev/sdb1, /dev/sdb2, /dev/sdb3 … Would it be /dev/nvme0n11, /dev/nvme0n12 or /dev/nvme0n1, /dev/nvme0n2…

Grateful for all advice before I try the installation.
Budge.

2

First thing to determine is whether your PC’s firmware supports booting from NVME on an M.2 adapter card. If not, then booting files will need to be on an SATA drive.

3

Hi mrmazda, yes a very good point.
I checked out the BIOS configuration screens but could not recognise the newly added PCi card which has the SSD installed.

Using the first BIOS setup screen I have choices:-
System Information,
System Settings,
Date and Time,
Start Options
Boot Manager
System Events Logs
User Security
Save Settings
Restore Settings
Load Default Settings
Exit Setup.

Start Options have three options:-
opensuse-secureboot
USB Storage
Hard Disk 0

Boot Manager has:-
Primary Boot Sequence
Add Boot Option
Delete boot Option
Change Boot Order
Seconary (WOL) Boot Sequence
Add WOL Boot…
Delete…
Change…
Boot From File>This takes me to a somewhere I do not recognise:-
NO VOLUME LABEL
[PciRoot(0x0)/Pci(0x9,0x0/Pci(0x0,0x0)/Ctrl(0x0)/Scsi(0x0,0x0)/HD(1,gpt,95C9969E-2604-4D00-8D33-88C27BB11C37,0x800,0x100000)]

in the systems settings menu I have choices:-
Processors,
Memory,
Devices and I/O Ports,
Power
Operating Modes,
Legacy Support,
Integrated Management Module,
System Security,
Adapters and UEFI Drivers,
Network,

of which the most relevant seem to be:-
Devices an I/O Ports >
Configure SATA as > IDE is only option at present
Configure IDE mode > set as Native mode (alternative is Compatibility mode,)
System CDROM/DVD>set as Enable (alternative is Disable,)
Active Video>set as Onboard Device (alternative is Add-in Device,)
Interrupt Round Robin>set as Disabled (alternative is Enable,)
Scheme:
Enable /Disable Onboard Devices > Ethernet>Enabled
Enable / Disable Adapter Option ROM Support>Ethernet and Slots 1thro 6> All Enabled
Set Option ROM Execution Order>Ethernet>Slot 1, Slot 2 etc to Slot 6
PCIe Gen1/Gen2 Speed Selection>Slots 1 thro 5 all set for Gen2

The other option which I believe is relevant is;-
Adapters and UEFI Drivers.
>Available Adapters and Drivers which are listed:-
Broadcom Gigabit Ethernet Driver
I-PciRoot(0x0/Pci(0x1,0x0)/Pci(0x0,0x0)
I-I-PciRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x0)/MAC(34 . 40B592A054,0x0)

Broadcom Gigabit Ethernet Driver
I-PciRoot(0x0/Pci(0x1,0x0)/Pci(0x0,0x1)
I-I-PciRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x1)/MAC(34 . 40B592A055,0x0

LSI EFI SAS Driver
I-PciRoot(0x0)/Pci(0x9,0x0)/Pci(0x0,0x0)
I-I-PciRoot(0x0)/Pci(0x9,0x0)/Pci(0x0,0x1)/Ctrl(0 . x0)
I-I-PciRoot(0x0)/Pci(0x9,0x0)/Pci(0x0,0x1)/Ctrl(0 . x1)

I have not been able to identify the NVME device in all the above and it is quite possible that I need a particular driver.
I think it work investigating a bit more if you have the time.

If there is no option to boot from SSD available and I cannot get any firmware update would it be possible to set up so my initial boot is to a USB and then have this run the SSD thereafter.

Hope you can advise further please.
Budge

4

I doubt a firmware update would add such a basic function. An NVME should simply show up in boot options along with SATA, USB and SCSI disks. I should have written SATA or USB as location for booting files.

5

OK understood. AFAIK the ability to boot NVME was excluded in the original Red Book for this server. I was told it had now been enabled, ie the exclusion removed but I have no details.

Since there is nothing obvious (to me) to try, I think I should not waste time on this now and concentrate on the other options.

My objective is to get the speed benefits from SSD for my day to day work and multimedia library on the raid array.

Please could you suggest a suitable approach for booting and then getting my root and home partitions working from SSD.

As a footnote to getting my new hardware I had hoped to find an inexpensive PCIe X8 M2 NVME adaptor which would occupy all of my slot, but could only easily source an X4. The X8 were eight times more expensive and were for 2 memory cards. I assume they must offer some on-board raid capability for the two memory cards otherwise I can’t understand why the cost difference. I do not know what benefits can be achieved from the X8 vs X4.

6

Hi
The expensive boards have Bifurcation, did see one that is not RAID, but this HP supports nvme boot…

My intel M/B doesn’t so when running that just setup to boot from SSD, does your motherboard have a spare USB 2.0 or 3.0 header free, if so get a USB header adaptor to plug in for an internal USB key to boot from (then no chance of getting removed accidentally)?

I also configured my nvme for 4K not 512;


 nvme list
Node             SN                   Model                                    Namespace Usage                      Format           FW Rev  
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1     xxxxxxxxxxxxx        WDC WDS250G1B0C-00S6U0                   1         250.06  GB / 250.06  GB      4 KiB +  0 B   201000WD

You might want to explore the nvme command for more info on your device…

7

Hi Malcolm, exactly my thoughts as I too couldn’t work out how to boot from the mobo but there is an USB connection on the mobo. I think I can use this although it was intended for a VMware ESXi virtualisation key. Should be fine.

Thanks for the suggestions on formatting. I have a bit more reading to do and problems with the on-board display which is not doing well under Leap 15.3 but otherwise looking good. I am working to get this machine ready to swap out the main machine. If all goes well I have a model for the upgrade of the second machine. Hope it goes OK as I have already purchased the upgraded memory! Some rather good prices available. Had some bad buys but no problems thanks to the now normal Ebay arrangement for returns.

8

I agree using an internal USB instead of external is a good idea, next best thing to an SATA device. You ought to be able to count on it to maintain its place in device enumeration at each boot nearly as well as an SATA (or NVME) device. Second place to SATA is due to the massive difference in longevity expected.

9

Hi Malcolm,
I looked through the man pages and there are plenty of useful commands available but sadly the command doesn’t work on my memory because as yet I have no driver.
I have a couple of USB slots on the board which I can try but with uefi now required, please could I have some detailed instructions on how to create a boot USB device. Does it have to hold the uefi partition and OS or something else. How do I arrange it so that root and home are stored on the nvme and how do I format it please?

10

Your new boot device needs a VFAT ESP partition (260MB recommended minimum[li], >512MB seriously wastful; unlikely more than 10% of 260MB would actually be used for less than 5 installed operating systems), to be mounted at /boot/efi/; and a native (EXTx or BTRFS formatted) partition to be mounted at /boot/. 500MB is probably more than enough, but 1GB or more wouldn’t hurt, probably allowing better long term reliability of the inferior quality of USB sticks compared to SSD or HDD. You could devote the entire balance of a stick remaining after ESP creation to the /boot/ partition, maximizing reliability from the periodic placement/replacement of the relatively large kernel and initrd files that will live there.
[/li]
You’ll need to use the expert partitioner option during installation whether you create the partitions ahead of installation, or during.

Partitioning is art as much as science. There is no one right or wrong way.

[*] The only reason to use this much space is to avoid a complaint from the openSUSE installer. 100MB is actually more than 99.999% of installations would ever actually use. 260MB is the legal minimum on a 16TB or larger disk.

11

Hi
I always use 260MB, there are some extra tools that your system may support eg HP can use it for the likes of efi shell to set the boot log, BIOS update (laptops) and diagnostic/recovery tools. On this HP there is a separate linux program for flashing the BIOS.

The disk needs to be gpt, the efi partition type ef00.

If you boot from a rescue system, the nvme command should work?

12

I have found an USB socket intended for a tape drive which is in a good physical location so have put in the smallest physical sized USB memory I had available which is rather large in memory terms. Here is what my machine looks like now:-

alastair@localhost:~> sudo fdisk -l
[sudo] password for root: 
Disk /dev/sda: 2.73 TiB, 2996997980160 bytes, 5853511680 sectors
Disk model: ServeRAID M5014 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 391BF99A-839F-4098-957C-ED9C8AF9F265

Device          Start        End    Sectors  Size Type
/dev/sda1        2048    1050623    1048576  512M EFI System
/dev/sda2     1050624 5849315327 5848264704  2.7T Linux filesystem
/dev/sda3  5849315328 5853511646    4196319    2G Linux swap


Disk /dev/sdb: 115.48 GiB, 123992014848 bytes, 242171904 sectors
Disk model: DT microDuo 3C  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xbe063dc5

Device     Boot   Start       End   Sectors  Size Id Type
/dev/sdb1          2048   1026047   1024000  500M  b W95 FAT32
/dev/sdb2       1026048 242171903 241145856  115G 83 Linux


Disk /dev/nvme0n1: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: KINGSTON SNVS2000G                      
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
alastair@localhost:~>

You can see I have created the partitions for UEFI (vfat or fat32) and Boot (btrfs) and the NVME drive is also there.

I think therefore I am ready so how do I get to re build the system please?
I assume I must either re-install or perhaps use a live disk, in which case which live system?
Grateful for advice please.

13

Given the 3TB disk has an ESP partition, I would have given it a /boot/ partition as well, which would dispense with need for the internal USB stick. If the USB is going to be the sole stepping stone to using the NVME, then its VFAT needs its type changed from 0B to EF in order to become an ESP. It would be best to repartition the USB stick to use GPT instead of DOS, if it is to be kept for running your OS from NVME.

14

OK that makes sense to me although as you know the 3TB disk is actually 4x1TB hard drives in a raid array. All the booting is done on it at present.

I am not anxious to use the USB if it can be avoided. I came to this only because I cannot boot from NVME.
What do I do to arrange to leave the minimum required for booting where it is and have everything else that can work on NVME there.

I should then also change the setup so I have a /multimedia partition using the rest of the drive.

15

I don’t know the boot or partition capabilities of your RAID host. It may be that having an ESP on “it” is pointless, and thus a /boot/ partition on “it” might be pointless as well. It could be your internal USB is ultimately required. That you are able to boot from it currently suggests you could carve out another partition for /boot/, and thus not need the internal USB for booting.

If I was using several HDDs in MD RAID I’d partition the disks so that /boot/ and ESP could be on modest size partitions un-RAIDed, thus unqualifiedly usable as /boot/ and ESP, with the rest of the disks RAIDed, thus no USB would be needed. This partitioning of disks and making up RAID from less than entire disks is essentially what I have now in three PCs that do not have UEFI BIOS.

16

Good morning. The raid is managed by an hardware board which is not related to the operating system and should be treated as a single drive until we find out otherwise. I do not want to rebuild it or divide it.

I looked at the Expert Partitioner by booting from the installation network version CD. The Partitioner may be expert but sadly not the user. Try as I might I cannot divorce the sub-volumes between devices. Some of the choices I try and make are not offered, others are in the too difficult (for me) box.

Rather than struggle with this I am reconciled to booting from the USB which is installed and seems to be working fine. I set up the partitions on the USB as suggested by Malcolm earlier, which seems to fit with what has been suggested previously. I can change whatever you suggest but I now need explicit guidance please so I can get this machine set up and ready to put to work.
Regards,
Budge

17

1: As directed by malcolmlewis, repartition the USB GPT with the VFAT partition labeled type ESP (EFI System Partition; EF00), and the other type Linux. Optionally, do this with the expert partitioner after beginning openSUSE installation.
2: Boot the installation media in UEFI mode
3: Select expert partitioner using existing partitions
4: Select the Linux partition on USB to mount on /boot/ and format EXTx or BTRFS. Personally I would select type EXT2 so as to save wear on the USB by not having a journal. (It’s also possible to omit journal on the more advanced EXT4 type.)
5: Select the ESP partition to mount on /boot/efi/ (and format FAT32/ESP if you didn’t already)
6: Create and format partition(s) as desired on the NVME for / at least, if not for /home/ or others
7: Select mount point for your RAID’s data partition (and create a filesystem on it if not already done)
8: Proceed with an otherwise typical UEFI installation

18

Hi and many many thanks, I believe I have it. It does matter what order it is done and that what was giving me trouble.
The partition plan was accepted anyhow and the installation process is proceeding as I type. We are only one notch up from carrier pigeon here so installation takes an age. Will report back tomorrow.
Many thanks again.
If all goes well I might be able to continue with the display problem. You should have the log available.

19

Hi mrmazda,
Just a quick note of thanks for your efforts on my behalf.
I am now able to boot into the new system. I shall revise the boot order on BIOS to make it more “correct” for my system.
A good few supplementary questions please, in no particular order:-

  1. I would appreciate you looking over my configuration and optimizing it with the hardware I now have installed please.

  2. I didn’t use the RAID drive. It is shown as /dev/sda and is formatted as an XFS disk.
    I propose to have it mounted as /multimedia. When I try and mount it using Yast it only offers me a few options which are already included on /dev/nvme0n1 as sub-volumes. I seek a bit of help creating a mount point in / as /multimedia and then mounting the XFS partition there. My intention is to load it with my multimedia database.

  3. KDE Desktop default shows the USB I have set up for booting as a removable device in my task bar. How can I prevent this particular USB showing this way and make it “fixed?”

  4. The USB “stick” I have used for booting is much larger than needed but afaik the size of the btrfs partition is not set. I have written off the cost of this one but for next time, when I am setting up the other machine in the same way, is there a better way or do I just get a smaller USB stick.

No doubt more questions will follow but I shall start new threads as required.

Many thanks again,
Budge.

20

Hi
Create a udev rule to ignore it… I use a file called /etc/udev/rules.d/99-hide-disks.rules containing for example;


KERNEL=="sdb2", ENV{UDISKS_IGNORE}="1"

Modify as required for your setup.