opensuse 12.2 How do I set dom0_mem to stop dom0 balooning?

[nanite_uk]

Hi, My Opensuse 12.2 server running the xen hypervisor opennebula 3.9.80 is showing the following error in my xend.log file.

Searching the opensuse documentation Chapter I found the following extract but there is no "Additional Xen Hypervisor Parameters" in the boot loader!!

When I set dom0_mem=1536M in the kernel options and re-boot it does not seem to register.

2.2.1. Setting a Maximum Amount of Memory¶

• Determine the amount of memory to set for Domain0.
  
• At Domain0, type xm info to view the amount of memory that is available on the machine. The memory that is currently allocated by Domain0 can be determined with the command xm list.
  
• Run YaST+Boot Loader.
  
• Select the Xen section.
  
• In Additional Xen Hypervisor Parameters, add dom0_mem=mem_amount where mem_amount is the maximum amount of memory to allocate to Domain0. Add K, M, or G, to specify the size, for example,dom0_mem=768M.
  
• Restart the computer to apply the changes

xend.log

[2013-04-18 00:32:06 2498] DEBUG (XendDomainInfo:2576) XendDomainInfo.constructDomain
[2013-04-18 00:32:06 2498] DEBUG (balloon:206) Balloon: 35675552 KiB free; need 16384; done.
[2013-04-18 00:32:06 2498] DEBUG (XendDomain:482) Adding Domain: 1
[2013-04-18 00:32:06 2498] DEBUG (XendDomainInfo:2917) XendDomainInfo.initDomain: 1 256
[2013-04-18 00:32:56 2498] ERROR (XendBootloader:60) Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible
[2013-04-18 00:32:57 2498] ERROR (XendDomainInfo:489) VM start failed
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 475, in start
XendTask.log_progress(31, 60, self._initDomain)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendTask.py", line 209, in log_progress
retval = func(*args, **kwds)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 2919, in _initDomain
self._configureBootloader()
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 3393, in _configureBootloader
bootloader_args, kernel, ramdisk, args)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendBootloader.py", line 61, in bootloader
raise VmError(msg)
VmError: Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible
[2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:3164) XendDomainInfo.destroy: domid=1
[2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:2484) No device model
[2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:2486) Releasing devices
[2013-04-18 00:32:57 2498] ERROR (XendDomainInfo:108) Domain construction failed
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 106, in create
vm.start()
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 475, in start
XendTask.log_progress(31, 60, self._initDomain)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendTask.py", line 209, in log_progress
retval = func(*args, **kwds)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 2919, in _initDomain
self._configureBootloader()
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 3393, in _configureBootloader
bootloader_args, kernel, ramdisk, args)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendBootloader.py", line 61, in bootloader
raise VmError(msg)
VmError: Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible


Thanks
Dennis

[jdmcdaniel3]

Hi, My Opensuse 12.2 server running the xen hypervisor opennebula 3.9.80 is showing the following error in my xend.log file.

Searching the opensuse documentation Chapter I found the following extract but there is no "Additional Xen Hypervisor Parameters" in the boot loader!!

When I set dom0_mem=1536M in the kernel options and re-boot it does not seem to register.

2.2.1. Setting a Maximum Amount of Memory¶

• Determine the amount of memory to set for Domain0.
• At Domain0, type xm info to view the amount of memory that is available on the machine. The memory that is currently allocated by Domain0 can be determined with the command xm list.
• Run YaST+Boot Loader.
• Select the Xen section.
• In Additional Xen Hypervisor Parameters, add dom0_mem=mem_amount where mem_amount is the maximum amount of memory to allocate to Domain0. Add K, M, or G, to specify the size, for example,dom0_mem=768M.
• Restart the computer to apply the changes

xend.log

[2013-04-18 00:32:06 2498] DEBUG (XendDomainInfo:2576) XendDomainInfo.constructDomain
[2013-04-18 00:32:06 2498] DEBUG (balloon:206) Balloon: 35675552 KiB free; need 16384; done.
[2013-04-18 00:32:06 2498] DEBUG (XendDomain:482) Adding Domain: 1
[2013-04-18 00:32:06 2498] DEBUG (XendDomainInfo:2917) XendDomainInfo.initDomain: 1 256
[2013-04-18 00:32:56 2498] ERROR (XendBootloader:60) Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible
[2013-04-18 00:32:57 2498] ERROR (XendDomainInfo:489) VM start failed
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 475, in start
XendTask.log_progress(31, 60, self._initDomain)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendTask.py", line 209, in log_progress
retval = func(*args, **kwds)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 2919, in _initDomain
self._configureBootloader()
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 3393, in _configureBootloader
bootloader_args, kernel, ramdisk, args)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendBootloader.py", line 61, in bootloader
raise VmError(msg)
VmError: Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible
[2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:3164) XendDomainInfo.destroy: domid=1
[2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:2484) No device model
[2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:2486) Releasing devices
[2013-04-18 00:32:57 2498] ERROR (XendDomainInfo:108) Domain construction failed
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 106, in create
vm.start()
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 475, in start
XendTask.log_progress(31, 60, self._initDomain)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendTask.py", line 209, in log_progress
retval = func(*args, **kwds)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 2919, in _initDomain
self._configureBootloader()
File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 3393, in _configureBootloader
bootloader_args, kernel, ramdisk, args)
File "/usr/lib64/python2.7/site-packages/xen/xend/XendBootloader.py", line 61, in bootloader
raise VmError(msg)
VmError: Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible


Thanks
Dennis

My first question is, do you really need to use xen? Have you considered using using VirtuaBox? Next, are you still using Grub2? If so, you can use this to edit your menus: GNU Grub2 Command Listing Helper with --help & Input - Blogs - openSUSE Forums and find the grub2 menu you use to load xen up. With that, we could suggest the location for the dom0_mem=mem_amount which should limit the memory given to your clients. I don't use xen myself, but I think we can help, but are you sure this is the software tool you want to use?

Thank You,

[nanite_uk]

Hi,

Thank you for your quick response, I will answer your questions in-line.

My first question is, do you really need to use xen?
I want a full hypervisor because I am building an Opennebula cloud management platform.
http://opennebula.org

Have you considered using using VirtuaBox?
Opennebula can manage Virtualbox/KVM/VMWare and a few other hypervisors but I do not want the controller node to be Virtualbox.

Next, are you still using Grub2? if so, you can use this to edit your menus: GNU Grub2 Command Listing Helper with --help & Input - Blogs - openSUSE Forums and find the grub2 menu you use to load xen up. With that, we could suggest the location for the dom0_mem=mem_amount which should limit the memory given to your clients. I don't use xen myself, but I think we can help, but are you sure this is the software tool you want to use?

I have installed vanilla opensuse 12.2, installed the xen virtualization tools and grub2 is used to manage which kernel to use but there is no menu.lst file that I can find to edit to add dom0_mem=mem_amount in the Yast bootloader gui.

Your blog mentions Grub2 configuration, I have looked at the many files, /etc/grub.d/10_linux/20_linux?_xen etc but not sure which one to update.

Please advise

Kind regards
Dennis

[jdmcdaniel3]

Hi,

Thank you for your quick response, I will answer your questions in-line.

My first question is, do you really need to use xen?
I want a full hypervisor because I am building an Opennebula cloud management platform.
http://opennebula.org

Have you considered using using VirtuaBox?
Opennebula can manage Virtualbox/KVM/VMWare and a few other hypervisors but I do not want the controller node to be Virtualbox.

Next, are you still using Grub2? if so, you can use this to edit your menus: GNU Grub2 Command Listing Helper with --help & Input - Blogs - openSUSE Forums and find the grub2 menu you use to load xen up. With that, we could suggest the location for the dom0_mem=mem_amount which should limit the memory given to your clients. I don't use xen myself, but I think we can help, but are you sure this is the software tool you want to use?

I have installed vanilla opensuse 12.2, installed the xen virtualization tools and grub2 is used to manage which kernel to use but there is no menu.lst file that I can find to edit to add dom0_mem=mem_amount in the Yast bootloader gui.

Your blog mentions Grub2 configuration, I have looked at the many files, /etc/grub.d/10_linux/20_linux?_xen etc but not sure which one to update.

Please advise

Kind regards
Dennis

Both openSUSE 12.2 and 12.3 default to using the new Grub 2 boot loader. Using Grub Legacy was an install option in openSUSE 12.2 and is what used the old menu.lst file. In Grub 2, the file "/etc/default/grub" contains all your default settings, the file "/boot/grub2/grub.cfg" is your active boot menu, equivalent to the old menu.lst file. However, while you can edit the file "/boot/grub2/grub.cfg", any new kernel added to your system will cause this fill to be recreated from scratch based on the bash scripts located in "/etc/grub.d/" and your "/etc/default/grub" file. I suspect a setting may be required in "/etc/default/grub" for xen, but I am not sure. The bash script Grub2Cmd will not run if Grub2 has not been install on your PC and worth a try to see if it can help you find the right files.

Thank You,

[tsu2]

When I first saw this and another post about Xen Dom0 memory ballooning, the whole idea seemed preposterous to me since I'd never heard of a similar concern using any other virtualization technology.

But it seems that this has been raised <very> recently, a Google search turns up two hits, a private blog and a xen wiki
Managing Xen Dom0′s CPU and Memory - Linux Tutorials - Fclose
Xen Best Practices - Xen

It looks like the methods described in both links are easily implemented in any distro including openSUSE.

But, IMO if you read both the main concern isn't restricting growth, the main concern is shrinkage <after> ballooning and then whether sufficient resources are already available for normal administration and spinning up new Guests.

So, read <why> and you might find that simply setting some reasonable min limits especially on a "Controller only" node is likely optional and only insurance, and not likely necessary.

IMO,
TSU

[tsu2]

Re your original post.

Looks like you're pin pointing only a notification, it's not an error.

Also, according to the links I posted you shouldn't be setting a max memory limit (that's not actually the issue of concern), if anything (again, it <depends> on your node's load if even necessary) you should set a reserved and/or min setting.

TSU

[nanite_uk]

Re your original post.

Looks like you're pin pointing only a notification, it's not an error.

Also, according to the links I posted you shouldn't be setting a max memory limit (that's not actually the issue of concern), if anything (again, it <depends> on your node's load if even necessary) you should set a reserved and/or min setting.

TSU

I agree with all your comments after more testing it seems I was barking up the wrong tree, I have found a permissions bug in Opennebula 4 RC.

I now have opensuse 12.2 xen installed on two nodes, one is the controller node and the other is a just a node, the controller node has a NFS share that is mounted on node 2 with the correct share options. I can spawn a vm on the controller node without any issues but if I spawn a vm to node 2 or migrate the vm from node 1 to node 2 I get a disk.0 isn't accessible error.

The test I did was to change the permission on disk.0 on node 2 while the vm was being generated and the vm started correctly on node 2.

Thanks for all your help.

Regards
Dennis