Because I could not find a PCI passthrough guide made for openSuSE, but only for some other distros, and because there are differences which might discourage newer openSuSE users from setting up a gaming VM with PCI passthrough, I decided to make one myself. I hope you’ll find this usefull:
Notes
I’ve successfully set GPU passthrough on my PC - i5-3570 CPU, Asus P8H77-V LE motherboard, one Radeon R7 250 (for the virtual machine) videocard and one nVidia 750 Ti (for the host) - running openSuSE Leap 42.2 kernel 4.4.36-8-default.
Although I’ll try to make a guide that anyone can follow, I’ll assume that the reader has at least some Linux knowledge.
Prerequisites
- you need at least 2 video cards - one will be used by the Linux host and the other one will dedicated to the virtual machine (let’s use VM from now on) that’s running Windows.
- you can’t use a Intel integrated GPU for the VM, you can use only AMD or nVidia. You can use Intel integraded GPU for the host, though.
- you need a CPU and a motherboard that supports VT-D (Intel) or AMD-Vi (AMD) - a list can be found <a href=“https://en.wikipedia.org/wiki/List_of_IOMMU-supporting_hardware” target="_blank">here</a> - and you need to enable it in BIOS .
- the PCI root port should not be part of the same IOMMU group as the GPU you want to use with the VM (that will be explained more later).
- a secondary display or free input port on your primary display to be used by the VM.
- Install software
zypper in libvirt libvirt-client libvirt-daemon virt-manager virt-install virt-viewer qemu qemu-kvm qemu-ovmf-x86_64 qemu-tools
Also, download VirtIO drivers from here.
- Enable IOMMU
IOMMU is a generic name for Intel VT-x/Intel and AMD AMD-V/AMD-Vi. Enable it by setting intel_iommu=on (for Intel CPUs) or amd_iommu=on (for AMD CPUs) bootloader kernel option. To do this, edit /etc/default/grub and add intel_iommu=on or amd_iommu=on to GRUB_CMDLINE_LINUX_DEFAULT options. Save the file and regenerate grub by executing
grub2-mkconfig -o /boot/grub2/grub.cfg
Reboot and execute
dmesg|grep -e DMAR -e IOMMU
to verify that IOMMU is enabled properly.
http://i.imgur.com/EJTyZmv.png
If you have a similar output, IOMMU is enabled.
- Make sure that IOMMU groups are valid
Run this script (I’ve named it lsiommu):
#!/bin/bash
shopt -s nullglob
for d in /sys/kernel/iommu_groups/*/devices/*; do
n=${d#*/iommu_groups/*}; n=${n%%/*}
printf 'IOMMU Group %s ' "$n"
lspci -nns "${d##*/}"
done;
The result should be an output like this:
Here you can find 2 things: what IOMMU group the device you want to use for the virtual machine is (in my case, group 11) and its vendor/model ids (in my case, 1002:683f and 1002:aab0). This guide will work only for devices which are alone (ignoring their associated audio device) in their IOMMU group.
- GPU isolation
We need to isolate the GPU. This is pretty easy if you have 2 GPUs from different manufacturers (like me - just blacklist the module for the GPU you want to use in VM), but a bit more complicated if both GPUs are from the same vendor. And a lot more complicated if both GPUs are the same model. This guide can be followed as long as both your GPUs are NOT the exact same model, even if they are from the same vendor. It will NOT work with 2 identical GPUs.
First, create a file named
/etc/modprobe.d/gpu-passthrough.conf
. Edit it and insert:
options vfio-pci ids=**_your_GPU_ids_**,**your GPU's audio device id**]
in my case, that was:
options vfio-pci ids=**1002:683f**,**1002:aab0**
Please note the ids. Edit again /etc/default/grub and add rd.driver.pre=vfio-pci to GRUB_CMDLINE_LINUX_DEFAULT options. Save it and regenerate grub.
- Rebuild initrd
You need to rebuild the initial ram disk to include all the needed modules. Create a file named /etc/dracut.conf.d/gpu-passthrough.conf and insert this line into it:
add_drivers+="pci_stub vfio vfio_iommu_type1 vfio_pci vfio_virqfd kvm kvm_intel"
Rebuild now the initrd by executing
dracut --force /boot/initrd $(uname -r)
Please pay attention, because if you did something wrong, this might make your Linux unbootable.
- Reboot and check that your GPU was isolated
After you’ve rebuild you initrd, check if your GPU was isolated; run lspci -k and look for **Kernel driver in use: **for the GPU that you want to isolate (use with the VM). It should state vfio-pci. Example:
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde PRO [Radeon HD 7750/8740 / R7 250E]
Subsystem: PC Partner Limited / Sapphire Technology Device a001
Kernel driver in use: vfio-pci
Kernel modules: radeon
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
Subsystem: PC Partner Limited / Sapphire Technology Device aab0
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
- Define a VM
First, open /etc/libvirt/qemu.conf, find the nvram option and edit it like this:
nvram =
"/usr/share/qemu/ovmf-x86_64.bin:/usr/share/qemu/ovmf-x86_64-code.bin"
]
Restart libvirt daemon:
systemctl restart libvirtd
Start Virtual Machine Manager and create a new machine.
Select “Local install media” and “x86_64” for architecture options:
http://i.imgur.com/Sf04FVi.png
Select the Windows installer ISO image:
http://i.imgur.com/e5uHIM2.png
Chose how much memory and how many CPU cores your VM will have:
(because there is a limit for the number of images a post can contain, all the other images can be found here)
Create (or select one if you already have one defined) a disk image for your VM (it can even be a whole disk - just put its path there and make sure it’s not mounted in the host):
Name your VM, make sure you’ve checked “Customize configuration before install” and click on Finish:
You’ll be taken now to VM configuration screen. On Overview page, select UEFI firmware and i440FX chipset (q35 did not work for me):
On Processor page set “host-passthrough” for your processor (if it’s not in the list, just type it):
Click on “Add Hardware” button and add a SCSI controller (make sure to change the type to VirtIO SCSI for improved performance):
You’ll have a IDE disk by default. Change that to SCSI:
Add your isolated GPU - if it has a sound device, like mine, and if you have sound output (via HDMI or Display Port) on your display, add the soundcard (in my case, PCI device 02:00.1) as well:
Add a keyboard and a mouse - would be better if you have an extra pair as the input will be grabbed by the VM once it starts:
Remove all unnecessary devices - Tablet, Display Spice, Console, Channel spice, Video QXL. You’ll need a soundcard if your display does not have sound output or if you’ll connect it via VGA. You can buy a cheap USB sound card - you can find one of those for 10$ - and add it like you’ve added the keyboard and the mouse:
Connect your the video card that you’ve just assigned to the VM to a display and click on “Begin Installation”. If you did everything right, you should see the VM booting on that display.
-
Install Windows. You’ll need the VirtIO drivers downloaded at step 2 for the HDD (can be found under vioscsi directory).
-
After Windows is installed, you can install Synergy which is really amazing - with it you can use the same keyboard and mouse on multiple PCs without a physical KVM (Keyboard-Video-Mouse) switch.
