Results 1 to 9 of 9

Thread: Advice on Possible Virtualization Solution

  1. #1

    Default Advice on Possible Virtualization Solution

    I'm currently running a small Plex server using OpenSUSE 13.1, and am slowly adding things to it here and there. I have an HP desktop machine (retail) that is about 5-7 years old that is currently required for use for a project. Unfortunately, the machine typically maxes out the CPU and RAM quickly and causes significant performance issues while performing the work. The work typically includes MS Office work, Prezi, extension-required browsing (Silverlight, Java, Flash, etc.), and Skype video calls.

    Given this, here is what I'm seeking advice on. Sorry if this has fundamental/noob mistakes. Is there a way to use virtualization to relieve some of the load on the system? Since the CPU and RAM resources max out faster than the intranet bandwidth, would it be a better idea to just run remote desktop (VNC, Teamviewer, etc.) and run browsing that way? Would a virtual machine hosted by a local server and deployed to this HP client relieve any load off of the client machine? Any recommendations for how to utilize networked machine resources to reduce the load on the local client?

    Thanks in advance for the advice.

  2. #2
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,297
    Blog Entries
    1

    Default Re: Advice on Possible Virtualization Solution

    With most common virtualization technologies (VMware, Virtualbox, KVM, Xen, etc) you can expect approx 3%-8% overhead CPU and RAM compared to running on bare metal. If you're running few Guests on a box with plenty of resources, this overhead is usually acceptable. But, I don't know that you're actually asking about a virtualization question.

    You might consider a simple remote desktop connection, in the past I've set this up to substitute for a KVM switch (Keyboard-Video-Machine, not the virtualization tech). That way, from one machine you can access and run anything on another machine. Types of connections with low overhead include VNC and passing X over SSH(there are others).

    Note that you cannot magically gain resources, but you can re-allocate loads, ie run a task on the more appropriate machine.

    HTH,
    TSU

  3. #3
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Advice on Possible Virtualization Solution

    On 2014-10-03 17:06, tsu2 wrote:
    >
    > With most common virtualization technologies (VMware, Virtualbox, KVM,
    > Xen, etc) you can expect approx 3%-8% overhead CPU and RAM compared to
    > running on bare metal.


    IMO, hard disk overload is way higher, unless you use real disks instead.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 13.1 x86_64 "Bottle" at Telcontar)

  4. #4
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Advice on Possible Virtualization Solution

    On 2014-09-30 18:36, Zecht wrote:

    > and Skype
    > video calls.


    For this, I'm experimenting with a cheap (android) tablet device: 50..70
    euros with front and rear cameras. I can move around the house with it.
    It does support skype, and I have done audio test calls. Video test is
    waiting.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 13.1 x86_64 "Bottle" at Telcontar)

  5. #5
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,297
    Blog Entries
    1

    Default Re: Advice on Possible Virtualization Solution

    Quote Originally Posted by robin_listas View Post
    On 2014-10-03 17:06, tsu2 wrote:
    >
    > With most common virtualization technologies (VMware, Virtualbox, KVM,
    > Xen, etc) you can expect approx 3%-8% overhead CPU and RAM compared to
    > running on bare metal.


    IMO, hard disk overload is way higher, unless you use real disks instead.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 13.1 x86_64 "Bottle" at Telcontar)
    Although you'll find many opinions posted that virtual disks are slower than real disks, I can't think of a reason why that might be the case and some others share my observation... That there shouldn't really be any difference and that is because typically virtual disk blocks are aligned when laid on top of real physical disk blocks. There is some obvious mapping overhead from virtual to physical, but if everything is aligned, that's a 1:1 direct mapping (no heterogenous issues) and overhead is nearly non-existent.

    So, alignment is the only real issue and in most cases that wouldn't be an issue, particularly if a disk has never been migrated off the machine the disk was created on. Alignment issues typically are of 2 types:
    - A Windows file system is involved, Windows fs are offset. But non-Windows fs (including all Linux) aren't offset, they're aligned right on the partition boundaries.
    - disk block sizes can vary. The "old" block size was 256k. The "newer, today's standard" is 512k. There is also the WD large capacity block size of 5128k. But, notice that each larger block size is a multiple of the smaller block sizes so that even smaller block sizes overlaid on the larger blocks would still "fit" (be aligned).

    Putting the above in plain words, whenever a virtual disk block is accessed, if it's aligned properly it accesses the physical disk block exactly and as if access were direct... There is no difference. But, if there is an alignment issue, then the virtual disk block might have to access 2 (or more) additional physical blocks to read the same data, and of course that can be a big problem.

    So, although if something were to become mis-aligned it could cause massive I/O problems, it shouldn't happen on a well-working system.

    Given the above, then there remains only "ordinary" fs optimization issues like defragmentation, disk compaction, etc.

    IMO,
    TSU

  6. #6
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Advice on Possible Virtualization Solution

    On 2014-10-03 19:06, tsu2 wrote:
    >
    > robin_listas;2667700 Wrote:


    >> IMO, hard disk overload is way higher, unless you use real disks
    >> instead.


    > Although you'll find many opinions posted that virtual disks are slower
    > than real disks, I can't think of a reason why that might be the case
    > and some others share my observation...


    It is simply my personal observation. I have run disk load tests on some
    virtual machines, and they were much slower that the real hardware on
    the same machine, by a factor of ten or more, some times (depending on
    the test).

    ....

    > That there shouldn't really be
    > any difference and that is because typically virtual disk blocks are
    > aligned when laid on top of real physical disk blocks. There is some
    > obvious mapping overhead from virtual to physical, but if everything is
    > aligned, that's a 1:1 direct mapping (no heterogenous issues) and
    > overhead is nearly non-existent.


    which means that you allocate the whole disk space on machine creation,
    instead of leaving it "dynamic", which is the default (at least with
    vmware). Also, the allocated space must be contiguous, or there will be
    head jumps where none is expected (for instance, when doing "dd
    if=/dev/sda ..." runs).

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 13.1 x86_64 "Bottle" at Telcontar)

  7. #7
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,297
    Blog Entries
    1

    Default Re: Advice on Possible Virtualization Solution

    Quote Originally Posted by robin_listas View Post
    On 2014-10-03 19:06, tsu2 wrote:
    >
    > robin_listas;2667700 Wrote:


    >> IMO, hard disk overload is way higher, unless you use real disks
    >> instead.


    > Although you'll find many opinions posted that virtual disks are slower
    > than real disks, I can't think of a reason why that might be the case
    > and some others share my observation...


    It is simply my personal observation. I have run disk load tests on some
    virtual machines, and they were much slower that the real hardware on
    the same machine, by a factor of ten or more, some times (depending on
    the test).

    ....

    > That there shouldn't really be
    > any difference and that is because typically virtual disk blocks are
    > aligned when laid on top of real physical disk blocks. There is some
    > obvious mapping overhead from virtual to physical, but if everything is
    > aligned, that's a 1:1 direct mapping (no heterogenous issues) and
    > overhead is nearly non-existent.


    which means that you allocate the whole disk space on machine creation,
    instead of leaving it "dynamic", which is the default (at least with
    vmware). Also, the allocated space must be contiguous, or there will be
    head jumps where none is expected (for instance, when doing "dd
    if=/dev/sda ..." runs).

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 13.1 x86_64 "Bottle" at Telcontar)
    I haven't noticed that dynamically growing disks or any type of contiguousness are requirements.
    I've speculated this is because no mapping is done at the file system level and above, it's done at the block level which should be data, file system and file agnostic.

    If you really do have the extreme I/O issues you describe, then I recommend closely inspecting the partition alignment of your virtual disks and volumes on the physical file system(various articles should exist on the Internet).

    Also as I've described, if you create brand new storage containing a Linux file system on a Host Linux file system (the disk isn't being imported from another machine), you should automatically not be affected by alignment issues.

    TSU

  8. #8
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Advice on Possible Virtualization Solution

    On 2014-10-03 21:26, tsu2 wrote:

    > If you really do have the extreme I/O issues you describe, then I
    > recommend closely inspecting the partition alignment of your virtual
    > disks and volumes on the physical file system(various articles should
    > exist on the Internet).


    There is no alignment possible when the virtual disk is dynamic. That
    is, it grows in size as you add files in the guest.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 13.1 x86_64 "Bottle" at Telcontar)

  9. #9
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,297
    Blog Entries
    1

    Default Re: Advice on Possible Virtualization Solution

    Since this is a bit OT to the original post, I hope some will bear this tangent which I think is still useful.

    An attempt to describe Virtual Disk Block mapping to Physical Disk Blocks (as I understand it) and understand Disk Block Alignment

    What is this virtual disk file which is commonly used by various virtualization technologies?
    The following applies to the most common implementation of the virtual file system, which is within what might be called at times a "loopback" or "backing" file. The file is mounted on a loop device for use as an entire disk for a Guest operating system.

    In simplest terms, what is this "alignment" issue?
    When an app reads data in a virtual disk, it will look at a file system that resembles a real, physical file system. But because the file system is virtual, the data will be retrieved not from the actual physical disk blocks but the mapped virtual blocks. If the virtual blocks map exactly to physical blocks , then retrieving the data should be nearly exactly the same as reading directly from the physical disk. But, if the virtual block is mis-aligned, the mapping may straddle multiple physical blocks which would have to be read. If the file is large and fragmented, then this effect can be magnified many, many times causing extreme disk I/O degradation.

    Note that some references will describe "partition" alignment, but in its simplest terms, there is no entire partition that needs to be aligned, it's only to get the disk blocks aligned so the starting edge of the first partition of the virtual disk is all that is important... After that, everything else should be automatically mapped based on that first block.

    A more detailed description of virtual block to physical block mapping(may make your head spin)
    Files are generally "free floating." Unlike partitions which have fixed boundaries (in particular the physical disk) files are generally "self-aligning" -- ie. the HostOS will automatically write any file in a way that it will <start> at the beginning of a block although it may not always fill disk blocks with data completely. So, there is no problem with alignment when you're talking about files as seen by the HostOS.

    The trick then is for virtual disk files to align their internal file systems and partitions so that when a block is written and read within the virtual disk file, the virtual disk blocks should correspond to physical disk blocks, else suffer potentially massive I/O penalties because of the need to read a very large number of additional disk blocks to acquire the same data.

    The way this seems to be accomplished is to map virtual disk blocks 1:1 to physical disk blocks. The simplest scenario to envision is if both virtual and physical disk blocks are the same size. In this case, every virtual disk block is mapped exactly to a physical disk block.

    Now, note that this all works well regardless whether the disk is fully allocated or "dynamically growing." The actual size of the growing disk is whatever number of virtual blocks (which are automatically mapped to physical blocks). If the disk needs more disk blocks up to its specified limit, new blocks are simply mapped. And again, because the overall disk file is automatically already aligned and the internal disk layout and content is irrelevant at the block level, it all works fine.

    So, how does a virtual file system become mis-aligned (ie How to avoid mis-alignment)?
    Cause #1
    Windows file systems are offset, unlike file systems used by any other OS the Windows file system does not start exactly on the beginning partition edge. This should not be a problem if you create a virtual disk file and use that file on the machine where it was created, but if you move and re-deploy that file to another machine you may see mis-alignment. Because all *NIX file systems do not have this offset, you can create and re-deploy virtual disks containing *NIX file systems to any other physical machine and not see an alignment problem.

    Cause #2
    It would be nice if the entire world used the same size physical disk block size, but as disks have grown in size from the floppy to terabytes (and larger) in size, the same block size is not practical to maintain fast disk I/O. Although smaller disk block sizes are still best for efficiency (less waste), most people would not sacrifice efficiency for speed, particularly if you store fewer and larger files. To minimize the possibility of mis-alignment, every block size has been and likely forever will be a multiple of the first size (256kB). When a disk is mounted in an OS, the disk's physical characteristics including block size is automatically managed by the BIOS and OS so that all functions at a higher level don't have to worry about how this all works, and usually all is well. But, when you lay a virtual file system's virtual blocks on the actual physical blocks, it all <usually> works without an issue. Because there is not an exact 1:1 virtual to physical block mapping, there can be corner scenarios where multiple additional physical blocks might have to be read to retrieve the same amount of data, so a Virtual Admin needs to be aware of this possibility.

    I suspect disk block mis-alignment, what are some common ways to resolve?
    The following are only quick recommendations which resolve common issues but will not necessarily resolve a particular issue (ie Google your exact symptoms if necessary)
    - Defrag(both within the disk file and the HostOS file system), zero empty space and compact your disk file.
    - Move the diskfile off the existing HostOS partition and then move it back again. This is the common, well-known method to defrag files without running a defrag utility.
    - Again, Windows file systems are different. There should be specific instructions for your virtual technology and virtual disk format. So, when possible avoid Windows file systems and use only Linux file systems instead to just avoid these issues.

    HTH,
    TSU

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •