Hi!
is there somebody out there who solve the following problem:
SuSE 10.X or 11.X (‘SuSE’ Kernel and ‘SuSE’ Xen)
Hardware AMD or Intel 64bit
Xen 3.2. oder 3.3
passthrough a NIC to DomU
Goal:
running a Netware 3.12 in DomU (works)
with full HW control to a e100 Nic (failed)
(forum-> installation keywords: pciback xen)
i’ll try a lot and search a while but with no success any information appreceated!
Hi,
If adding a pci device currently bound to a driver in the dom0 guest to one of your domU guest is what your are after, you are on the right way: you have to use the module pciback.
The idea is to bind the pci device (in your case a nic) to the pciback module in the dom0 guest and then assign it to the target domU.
There are two main ways to achieve that, depending on how long you want the change to last:
a) If you want to assign the pci device to the target domU on a transient basis, do a manual allocation.
The allocation will only last until you reboot the system or the domU guest.
b) If you want the assigment to remain persistent among system reboots, stick to the automatic approach, that basically consists on editing the modprobe conf file, building a new initrd file system and changing the affected domU guest configuration file so that it includes the PCI device id in the pci parameter.
I’ll only cover here the transient one.
For illustration purposes I’ll use my wireless nic. Adapt the procedure to your system.
a) Manual allocation
Having your system started in Xen mode, from a dom0 bash root session do the following:
-Step 1: Find if your dom0 have support for the pciback module. If you are using open Suse 11, and are using the stock kernel, it should have it:
lustmord:~ # uname -r
2.6.25.18-0.2-xen
lustmord:~ # lsmod | grep pciback
lustmord:~ # modprobe pciback
lustmord:~ # lsmod | grep pciback
pciback 33940 0
xenbus_be 8192 4 pciback,netbk,blkbk,blktap
lustmord:~ #
Check that the directory /sys/bus/pci/drivers/pciback exists and looks like this:
lustmord:/sys/bus/pci/drivers/pciback # ls -l
total 0
--w------- 1 root root 4096 Dec 4 23:19 bind
lrwxrwxrwx 1 root root 0 Dec 4 22:58 module -> ../../../../module/pciback
--w------- 1 root root 4096 Dec 4 23:19 new_id
--w------- 1 root root 4096 Dec 4 23:19 new_slot
-rw------- 1 root root 4096 Dec 4 23:19 permissive
-rw------- 1 root root 4096 Dec 4 23:19 quirks
--w------- 1 root root 4096 Dec 4 23:19 remove_slot
-r-------- 1 root root 4096 Dec 4 23:19 slots
--w------- 1 root root 4096 Dec 4 23:19 uevent
--w------- 1 root root 4096 Dec 4 23:19 unbind
lustmord:/sys/bus/pci/drivers/pciback #
If it does, pciback was loaded correctly.
-Step 2: Find out the PCI device id corresponding to the device you want to allocate to the target domU:
lustmord:~ # lspci | grep [Ww]ireless
03:00.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN Network Connection (rev 61)
lustmord:~ #
The long format for the PCI Id is then : 0000:03:00.0
Step 3: Find out the driver (module) to which this PCI device is currently bound:
lustmord:~ # find /sys/bus/pci/drivers -name "0000:03:00.0"
/sys/bus/pci/drivers/iwl4965/0000:03:00.0
lustmord:~ #
Good, iwl4965 is bound to it. Double check that the directory content looks like this
lustmord:/sys/bus/pci/drivers/iwl4965 # ls -lrt
total 0
--w------- 1 root root 4096 Dec 4 21:44 unbind
--w------- 1 root root 4096 Dec 4 21:44 uevent
--w------- 1 root root 4096 Dec 4 21:44 new_id
lrwxrwxrwx 1 root root 0 Dec 4 21:44 module -> ../../../../module/iwl4965
--w------- 1 root root 4096 Dec 4 21:44 bind
lrwxrwxrwx 1 root root 0 Dec 4 21:44 0000:03:00.0 -> ../../../../devices/pci0000:00/0000:00:1c.1/0000:03:00.0
lustmord:/sys/bus/pci/drivers/iwl4965 #
Step 4: Unbind the PCI device from the iwl4965 driver:
lustmord:~ # echo -n "0000:03:00.0" > /sys/bus/pci/drivers/iwl4965/unbind
lustmord:~ #
We can verify that, in fact, the driver iwl4965 has been unbound from the PCI device located at 0000:03:00.0 by doing this:
lustmord:~ # cd /sys/bus/pci/drivers/iwl4965/
lustmord:/sys/bus/pci/drivers/iwl4965 # ls
bind module new_id uevent unbind
lustmord:/sys/bus/pci/drivers/iwl4965 # ls -lrt
total 0
--w------- 1 root root 4096 Dec 4 21:44 uevent
--w------- 1 root root 4096 Dec 4 21:44 new_id
lrwxrwxrwx 1 root root 0 Dec 4 21:44 module -> ../../../../module/iwl4965
--w------- 1 root root 4096 Dec 4 21:44 bind
--w------- 1 root root 0 Dec 4 23:10 unbind
lustmord:/sys/bus/pci/drivers/iwl4965 #
You will notice that the link to the PCI id vanished. That’s what we want.
Step 5: Bound the PCI id 0000:03:00.0 to the pciback module
lustmord:~ # echo -n "0000:03:00.0" > /sys/bus/pci/drivers/pciback/new_slot
lustmord:~ # echo -n "0000:03:00.0" > /sys/bus/pci/drivers/pciback/bind
Good. Now, verify that the module pciback is seeing the PCI 0000:03:00.0 device:
lustmord:~ # cd /sys/bus/pci/drivers/pciback/
lustmord:/sys/bus/pci/drivers/pciback # ls -l
total 0
lrwxrwxrwx 1 root root 0 Dec 4 23:25 0000:03:00.0 -> ../../../../devices/pci0000:00/0000:00:1c.1/0000:03:00.0
--w------- 1 root root 0 Dec 4 23:23 bind
lrwxrwxrwx 1 root root 0 Dec 4 22:58 module -> ../../../../module/pciback
--w------- 1 root root 4096 Dec 4 23:19 new_id
--w------- 1 root root 0 Dec 4 23:23 new_slot
-rw------- 1 root root 4096 Dec 4 23:19 permissive
-rw------- 1 root root 4096 Dec 4 23:19 quirks
--w------- 1 root root 4096 Dec 4 23:19 remove_slot
-r-------- 1 root root 4096 Dec 4 23:19 slots
--w------- 1 root root 4096 Dec 4 23:19 uevent
--w------- 1 root root 4096 Dec 4 23:19 unbind
lustmord:/sys/bus/pci/drivers/pciback #
Right, just as we wanted.
Step 6: Allocate the PCI 0000:03:00.0 device to the target domU:
Now, we are in conditions to attach the device to the xen guest which is going to use it. Having the target domU stopped, run the following command to start it with the new device:
lustmord:~ # xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 1857 2 r----- 217.7
vm1-sles10 512 1 0.0
vm2-sles10 512 1 0.0
vm3-opensuse11 384 1 0.0
Now, we launch the domU:
lustmord:~ # xm create pci=0000:03:00.0 /etc/xen/vm/vm3-opensuse11
Using config file "/etc/xen/vm/vm3-opensuse11".
Started domain vm3-opensuse11
lustmord:~ #
Wait until the target domU is running, connect to it and verify that the device was attached correctly using lspci. Once you have done that, proceed like always. Configure the module manually or using yast:
lustmord:~ # xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 1592 2 r----- 321.6
vm1-sles10 512 1 0.0
vm2-sles10 512 1 0.0
vm3-opensuse11 1 384 1 -b---- 20.2
lustmord:~ #
Now, connect to the domU:
lustmord:~ # xm console 1
Security Framework initialized
AppArmor: AppArmor initialized <NULL>
AppArmor: Registered secondary security module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 512
CPU: L1 I cache: 32K, L1 D cache: 32K
....
...
...
EXT3 FS on xvda2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
iwl4965: Intel(R) Wireless WiFi Link 4965AGN driver for Linux, 1.2.23ks
iwl4965: Copyright(c) 2003-2007 Intel Corporation
PCI: Enabling device 0000:00:00.0 (0000 -> 0002)
iwl4965: Detected Intel Wireless WiFi Link 4965AGN
iwl4965: Tunable channels: 13 802.11bg, 19 802.11a channels
....
....
....
ip_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (7168 buckets, 28672 max)
Welcome to openSUSE 11.0 (i586) - Kernel 2.6.25.5-1.1-xen (xvc0).
linux-i8s4 login:
Now you log in and run lspci:
linux-i8s4 login: root
password:
Have a lot of fun...
linux-i8s4:~ # lspci
00:00.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN Network Connection (rev 61)
linux-i8s4:~ #
There you have it. That is good. Now you run yast and do all the other things…And that’s all there is to it.
Hope this helps.
Regards.
Hi carboncore:
Does the pass-through IO in Xen require hardware support, like Intel VT-x(or d) or AMD SVM?
Thanks.
G’day, I followed this all the way through to starting the guest domU. My domU is Win2k3server which requires full virtualisation (FV).
I get the following error when I start the domU:
/etc/xen/vm # xm create pci=0000:00:02.0 Win2k3Server
Using config file “./Win2k3Server”
Error: failed to assign device: maybe the platform doesn’t support VT-d, or VT-d isn’t enabled properly?
Does this mean there is a problem on the dom0 side or that the domU guest doesn’t support something?
Can I enable VT-d?
System is AMD Phenom II 940 (3.0GHz Quadcore)
OpenSUSE 11.1 x86_64 with Xen that came with that (3.3?)
domU as mentioned is Windows 2003 server
Cheers
Hi, carboncore
Cheers, could you explain how you made the initrd with pciback, I tried to do the same thing but got an error message and I’d like to have pci cards unbound from boot. could you give me a step through like you did with the manual method?
Thanks in advance!
Sly