Results 1 to 7 of 7

Thread: opensuse 12.2 How do I set dom0_mem to stop dom0 balooning?

  1. #1

    Unhappy opensuse 12.2 How do I set dom0_mem to stop dom0 balooning?

    Hi, My Opensuse 12.2 server running the xen hypervisor opennebula 3.9.80 is showing the following error in my xend.log file.

    Searching the opensuse documentation Chapter I found the following extract but there is no "Additional Xen Hypervisor Parameters" in the boot loader!!

    When I set dom0_mem=1536M in the kernel options and re-boot it does not seem to register.

    2.2.1. Setting a Maximum Amount of Memory



    • Determine the amount of memory to set for Domain0.
    • At Domain0, type xm info to view the amount of memory that is available on the machine. The memory that is currently allocated by Domain0 can be determined with the command xm list.
    • Run YaST+Boot Loader.
    • Select the Xen section.
    • In Additional Xen Hypervisor Parameters, add dom0_mem=mem_amount where mem_amount is the maximum amount of memory to allocate to Domain0. Add K, M, or G, to specify the size, for example,dom0_mem=768M.
    • Restart the computer to apply the changes



    xend.log

    [2013-04-18 00:32:06 2498] DEBUG (XendDomainInfo:2576) XendDomainInfo.constructDomain
    [2013-04-18 00:32:06 2498] DEBUG (balloon:206) Balloon: 35675552 KiB free; need 16384; done.
    [2013-04-18 00:32:06 2498] DEBUG (XendDomain:482) Adding Domain: 1
    [2013-04-18 00:32:06 2498] DEBUG (XendDomainInfo:2917) XendDomainInfo.initDomain: 1 256
    [2013-04-18 00:32:56 2498] ERROR (XendBootloader:60) Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible
    [2013-04-18 00:32:57 2498] ERROR (XendDomainInfo:489) VM start failed
    Traceback (most recent call last):
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 475, in start
    XendTask.log_progress(31, 60, self._initDomain)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendTask.py", line 209, in log_progress
    retval = func(*args, **kwds)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 2919, in _initDomain
    self._configureBootloader()
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 3393, in _configureBootloader
    bootloader_args, kernel, ramdisk, args)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendBootloader.py", line 61, in bootloader
    raise VmError(msg)
    VmError: Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible
    [2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:3164) XendDomainInfo.destroy: domid=1
    [2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:2484) No device model
    [2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:2486) Releasing devices
    [2013-04-18 00:32:57 2498] ERROR (XendDomainInfo:108) Domain construction failed
    Traceback (most recent call last):
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 106, in create
    vm.start()
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 475, in start
    XendTask.log_progress(31, 60, self._initDomain)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendTask.py", line 209, in log_progress
    retval = func(*args, **kwds)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 2919, in _initDomain
    self._configureBootloader()
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 3393, in _configureBootloader
    bootloader_args, kernel, ramdisk, args)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendBootloader.py", line 61, in bootloader
    raise VmError(msg)
    VmError: Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible


    Thanks
    Dennis

  2. #2
    Join Date
    Mar 2010
    Location
    Austin - Texas
    Posts
    10,140
    Blog Entries
    48

    Smile Re: opensuse 12.2 How do I set dom0_mem to stop dom0 balooning?

    Quote Originally Posted by nanite_uk View Post
    Hi, My Opensuse 12.2 server running the xen hypervisor opennebula 3.9.80 is showing the following error in my xend.log file.

    Searching the opensuse documentation Chapter I found the following extract but there is no "Additional Xen Hypervisor Parameters" in the boot loader!!

    When I set dom0_mem=1536M in the kernel options and re-boot it does not seem to register.

    2.2.1. Setting a Maximum Amount of Memory



    • Determine the amount of memory to set for Domain0.
    • At Domain0, type xm info to view the amount of memory that is available on the machine. The memory that is currently allocated by Domain0 can be determined with the command xm list.
    • Run YaST+Boot Loader.
    • Select the Xen section.
    • In Additional Xen Hypervisor Parameters, add dom0_mem=mem_amount where mem_amount is the maximum amount of memory to allocate to Domain0. Add K, M, or G, to specify the size, for example,dom0_mem=768M.
    • Restart the computer to apply the changes



    xend.log

    [2013-04-18 00:32:06 2498] DEBUG (XendDomainInfo:2576) XendDomainInfo.constructDomain
    [2013-04-18 00:32:06 2498] DEBUG (balloon:206) Balloon: 35675552 KiB free; need 16384; done.
    [2013-04-18 00:32:06 2498] DEBUG (XendDomain:482) Adding Domain: 1
    [2013-04-18 00:32:06 2498] DEBUG (XendDomainInfo:2917) XendDomainInfo.initDomain: 1 256
    [2013-04-18 00:32:56 2498] ERROR (XendBootloader:60) Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible
    [2013-04-18 00:32:57 2498] ERROR (XendDomainInfo:489) VM start failed
    Traceback (most recent call last):
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 475, in start
    XendTask.log_progress(31, 60, self._initDomain)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendTask.py", line 209, in log_progress
    retval = func(*args, **kwds)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 2919, in _initDomain
    self._configureBootloader()
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 3393, in _configureBootloader
    bootloader_args, kernel, ramdisk, args)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendBootloader.py", line 61, in bootloader
    raise VmError(msg)
    VmError: Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible
    [2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:3164) XendDomainInfo.destroy: domid=1
    [2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:2484) No device model
    [2013-04-18 00:32:57 2498] DEBUG (XendDomainInfo:2486) Releasing devices
    [2013-04-18 00:32:57 2498] ERROR (XendDomainInfo:108) Domain construction failed
    Traceback (most recent call last):
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 106, in create
    vm.start()
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 475, in start
    XendTask.log_progress(31, 60, self._initDomain)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendTask.py", line 209, in log_progress
    retval = func(*args, **kwds)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 2919, in _initDomain
    self._configureBootloader()
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomainInfo.py", line 3393, in _configureBootloader
    bootloader_args, kernel, ramdisk, args)
    File "/usr/lib64/python2.7/site-packages/xen/xend/XendBootloader.py", line 61, in bootloader
    raise VmError(msg)
    VmError: Disk '/srv/cloud/one/var//datastores/0/13/disk.0' isn't accessible


    Thanks
    Dennis
    My first question is, do you really need to use xen? Have you considered using using VirtuaBox? Next, are you still using Grub2? If so, you can use this to edit your menus: GNU Grub2 Command Listing Helper with --help & Input - Blogs - openSUSE Forums and find the grub2 menu you use to load xen up. With that, we could suggest the location for the dom0_mem=mem_amount which should limit the memory given to your clients. I don't use xen myself, but I think we can help, but are you sure this is the software tool you want to use?

    Thank You,
    My Blog: https://forums.opensuse.org/blogs/jdmcdaniel3/

    Software efficiency halves every 18 months, thus compensating for Moore's Law

    Its James again from Austin, Texas

  3. #3

    Unhappy Re: opensuse 12.2 How do I set dom0_mem to stop dom0 balooning?

    Hi,

    Thank you for your quick response, I will answer your questions in-line.

    My first question is, do you really need to use xen?
    I want a full hypervisor because I am building an Opennebula cloud management platform.
    http://opennebula.org

    Have you considered using using VirtuaBox?
    Opennebula can manage Virtualbox/KVM/VMWare and a few other hypervisors but I do not want the controller node to be Virtualbox.

    Next, are you still using Grub2?
    if so, you can use this to edit your menus: GNU Grub2 Command Listing Helper with --help & Input - Blogs - openSUSE Forums and find the grub2 menu you use to load xen up. With that, we could suggest the location for the dom0_mem=mem_amount which should limit the memory given to your clients. I don't use xen myself, but I think we can help, but are you sure this is the software tool you want to use?

    I have installed vanilla opensuse 12.2, installed the xen virtualization tools and grub2 is used to manage which kernel to use but there is no menu.lst file that I can find to edit to add dom0_mem=mem_amount in the Yast bootloader gui.

    Your blog mentions Grub2 configuration, I have looked at the many files, /etc/grub.d/10_linux/20_linux?_xen etc but not sure which one to update.

    Please advise

    Kind regards
    Dennis


  4. #4
    Join Date
    Mar 2010
    Location
    Austin - Texas
    Posts
    10,140
    Blog Entries
    48

    Smile Re: opensuse 12.2 How do I set dom0_mem to stop dom0 balooning?

    Quote Originally Posted by nanite_uk View Post
    Hi,

    Thank you for your quick response, I will answer your questions in-line.

    My first question is, do you really need to use xen?
    I want a full hypervisor because I am building an Opennebula cloud management platform.
    http://opennebula.org

    Have you considered using using VirtuaBox?
    Opennebula can manage Virtualbox/KVM/VMWare and a few other hypervisors but I do not want the controller node to be Virtualbox.

    Next, are you still using Grub2?
    if so, you can use this to edit your menus: GNU Grub2 Command Listing Helper with --help & Input - Blogs - openSUSE Forums and find the grub2 menu you use to load xen up. With that, we could suggest the location for the dom0_mem=mem_amount which should limit the memory given to your clients. I don't use xen myself, but I think we can help, but are you sure this is the software tool you want to use?

    I have installed vanilla opensuse 12.2, installed the xen virtualization tools and grub2 is used to manage which kernel to use but there is no menu.lst file that I can find to edit to add dom0_mem=mem_amount in the Yast bootloader gui.

    Your blog mentions Grub2 configuration, I have looked at the many files, /etc/grub.d/10_linux/20_linux?_xen etc but not sure which one to update.

    Please advise

    Kind regards
    Dennis

    Both openSUSE 12.2 and 12.3 default to using the new Grub 2 boot loader. Using Grub Legacy was an install option in openSUSE 12.2 and is what used the old menu.lst file. In Grub 2, the file "/etc/default/grub" contains all your default settings, the file "/boot/grub2/grub.cfg" is your active boot menu, equivalent to the old menu.lst file. However, while you can edit the file "/boot/grub2/grub.cfg", any new kernel added to your system will cause this fill to be recreated from scratch based on the bash scripts located in "/etc/grub.d/" and your "/etc/default/grub" file. I suspect a setting may be required in "/etc/default/grub" for xen, but I am not sure. The bash script Grub2Cmd will not run if Grub2 has not been install on your PC and worth a try to see if it can help you find the right files.

    Thank You,
    My Blog: https://forums.opensuse.org/blogs/jdmcdaniel3/

    Software efficiency halves every 18 months, thus compensating for Moore's Law

    Its James again from Austin, Texas

  5. #5
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,448
    Blog Entries
    2

    Default Re: opensuse 12.2 How do I set dom0_mem to stop dom0 balooning?

    When I first saw this and another post about Xen Dom0 memory ballooning, the whole idea seemed preposterous to me since I'd never heard of a similar concern using any other virtualization technology.

    But it seems that this has been raised <very> recently, a Google search turns up two hits, a private blog and a xen wiki
    Managing Xen Dom0′s CPU and Memory - Linux Tutorials - Fclose
    Xen Best Practices - Xen

    It looks like the methods described in both links are easily implemented in any distro including openSUSE.

    But, IMO if you read both the main concern isn't restricting growth, the main concern is shrinkage <after> ballooning and then whether sufficient resources are already available for normal administration and spinning up new Guests.

    So, read <why> and you might find that simply setting some reasonable min limits especially on a "Controller only" node is likely optional and only insurance, and not likely necessary.

    IMO,
    TSU

  6. #6
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,448
    Blog Entries
    2

    Default Re: opensuse 12.2 How do I set dom0_mem to stop dom0 balooning?

    Re your original post.

    Looks like you're pin pointing only a notification, it's not an error.

    Also, according to the links I posted you shouldn't be setting a max memory limit (that's not actually the issue of concern), if anything (again, it <depends> on your node's load if even necessary) you should set a reserved and/or min setting.

    TSU

  7. #7

    Talking Re: opensuse 12.2 How do I set dom0_mem to stop dom0 balooning?

    Quote Originally Posted by tsu2 View Post
    Re your original post.

    Looks like you're pin pointing only a notification, it's not an error.

    Also, according to the links I posted you shouldn't be setting a max memory limit (that's not actually the issue of concern), if anything (again, it <depends> on your node's load if even necessary) you should set a reserved and/or min setting.

    TSU
    I agree with all your comments after more testing it seems I was barking up the wrong tree, I have found a permissions bug in Opennebula 4 RC.

    I now have opensuse 12.2 xen installed on two nodes, one is the controller node and the other is a just a node, the controller node has a NFS share that is mounted on node 2 with the correct share options. I can spawn a vm on the controller node without any issues but if I spawn a vm to node 2 or migrate the vm from node 1 to node 2 I get a disk.0 isn't accessible error.

    The test I did was to change the permission on disk.0 on node 2 while the vm was being generated and the vm started correctly on node 2.

    Thanks for all your help.

    Regards
    Dennis

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •