I have an odd issue with using vfio passthrough of a secondary graphics adapter to Linux guests. It works just fine when I pass it through to my Windows 7 virtual machine.
My hardware is:
Xeon E5-2620 v3
intel c612 motherboard (supermicro)
primary (host) gpu: radeon r7 250
secondary (guest) gpu: radeon r9 270
assorted other pci-e devices (both host and guest)
This is not an exhaustive list, the important thing is the CPU, chipset, and GPU are fully PCIe ACS compliant, so each passthrough device is in its own iommu group with this configuration.
I set up both the windows and Linux guests using virt-manager, more or less akin to the process lessershoe describes in his excellent thread:
The problem is, when I assign the secondary GPU to the linux guest, it is never initialized. The console output alwaysredirects to the virt-manager window. If I remove the display spice and qxl video devices, then I get no output from the secondary GPU at all.
Pls include all basics, all you’ve described is libvirt which can be used to manage nearly a dozen virt technologies.
Include
Virtualization technology
HostOS including version
GuestOS including version
Guest property config you attempted exactly (screenshot if necessary)
The passthrough devices on this system are 01:00.0, a USB3 controller, 03:00.0 + 03:00.1 (the Radeon 270).
When the GPU is assigned, and I remove the qemu graphics and spice thing, the linux guest won’t even boot. If I remove the GPU and put the qemu graphics back, but leave the USB controller, the guest boots fine, and passed through USB controller works just fine.
In it, down in the appendix which lists supported and unsupported commands, I see some vfio commands. You didn’t describe how you’re trying to implement passthrough, I suspect if you’re trying to implement by command line then you’re bypassing the guidance the libvirt vm manager would provide to implement a supported method
I did use virt-manager to create the guest machines, and also to assign the PCI devices. The xml is just the output of virsh dumpxml for those machines.
This is most certainly using vfio
alcyone:~ # lspci -v -s 03:00.003:00.0
VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Curacao PRO [Radeon R9 270] (prog-if 00 [VGA controller])
Subsystem: ASUSTeK Computer Inc. Device 048d
Physical Slot: 15
Flags: bus master, fast devsel, latency 0, IRQ 109
Memory at e0000000 (64-bit, prefetchable) [size=256]
Memory at fbc00000 (64-bit, non-prefetchable) [size=256]
I/O ports at d000 [size=256]
Expansion ROM at fbc40000 [disabled] [size=128]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150] Advanced Error Reporting
Capabilities: [270] #19
Capabilities: [2b0] Address Translation Service (ATS)
Capabilities: [2c0] #13
Capabilities: [2d0] #1b
Kernel driver in use: vfio-pci
Kernel modules: radeon
The problem isn’t the passthrough, that part works. The problem is getting the linux guests to use the passed through GPU as the display device instead of any virtual GPU that qemu presents. If I remove the virtual graphics, then the guests fail to boot.[/size][/size][/size][/size]
The information you posted isn’t sufficient to provide an answer (the xml dumps don’t describe your graphic card method of connection at all, in fact your Windows dump describes using the spice protocol which may be nothing more than how vm manager is connecting to the Guest) but your lspci does describe the card configured using the vfio driver.
So,
Let’s start with understanding what vfio is. The following is the official documentation/description with the following critical ideas
It’s a kernel mode implementation
It’s an IOMMU implemenation, facilitating DMA but with special security
Once you understand what it is and more or less what it exposes, my guess is that may explain why vfio pass-through may be difficult or impossible for KVM to support, connecting to a vfio device is too different than other devices with supported PCI passthrough.
But, there appears to be a solution at the following online document, the highlights include
Although KVM does not support, the “older” QEMU does, so with the commands in this document you can modify your libvirt vm manager connection to the hypervisor to implement full QEMU instead. In a nutshell, today KVM and QEMU are something like 98% merged, but since QEMU is full emulation it has better capabilities supporting diverse hardware than KVM, but with the penalty losing the performance that KVM paravirtualization provides. In this case though, I suspect the DMA characteristics of vfio may compensate somewhat for any QEMU deficiencies.
I’m providing the link starting at “PCI pass through” for full understanding of the entire picture although you should not implement anything in this first section that describes normal PCI pass through. The section that follows describes setting up the IOMMU Groups and then Device Assignment.
Note that the documentation describes starting a Guest from the machine type “qemu-system-ARCH” but you can select any type of machine you want, more likely an x64 system (but since you’re running full qemu you can specify any kind of machine, maybe even some ancient 8088 a TRS-80 or the extinct Alpha 64). Just fooling, of course any really odd machine will require more emulation with a likely greater performance penalty. But, read the QEMU documentation to learn how to list available machine types, CPU, specifying memory and disk and more.
I don’t see instructions for using vm manager to manage a Guest with a vfio device, so I’m not sure how that would be done, maybe you’ll figure out something.
HTH, post back here with your success or additional issues…
TSU
a TRS-80 or the extinct Alpha 64). Just fooling, of course any really odd machine will require more emulation with a likely greater performance penalty. But, read the QEMU documentation to learn how to list available machine types, CPU, specifying memory and disk and more.