New installation in VirtualBox doesn't start GUI (Xorg segmentation fault caused by vboxvideo)

I want to try to do tumbleweed and have installed the latest snapshot (2015/12/17) on VirtualBox.

When I finish the installation and restart the virtual machine I get this:

http://i63.tinypic.com/25au8mf.jpg

The system stop at this point and I can’t login.

I restarted the VM in text mode and have seen the following errors in the Xorg log:


    33.341] (EE) Backtrace:
    33.341] (EE) 0: /usr/bin/Xorg (xorg_backtrace+0x41) [0x58d451]
    33.341] (EE) 1: /usr/bin/Xorg (0x400000+0x191619) [0x591619]
    33.341] (EE) 2: /lib64/libc.so.6 (0x7f58bb793000+0x34dc0) [0x7f58bb7c7dc0]
    33.341] (EE) 3: /usr/lib64/xorg/modules/drivers/vboxvideo_drv.so (0x7f58b718e000+0x7819) [0x7f58b7195819]
    33.341] (EE) 4: /usr/lib64/xorg/modules/drivers/vboxvideo_drv.so (0x7f58b718e000+0x636d) [0x7f58b719436d]
    33.341] (EE) 5: /usr/bin/Xorg (InitOutput+0xa93) [0x47c773]
    33.341] (EE) 6: /usr/bin/Xorg (0x400000+0x3cdf4) [0x43cdf4]
    33.341] (EE) 7: /lib64/libc.so.6 (__libc_start_main+0xf0) [0x7f58bb7b3610]
    33.341] (EE) 8: /usr/bin/Xorg (_start+0x29) [0x428539]
    33.341] (EE) 
    33.341] (EE) Segmentation fault at address 0x0
    33.341] (EE) 
Fatal server error:
    33.341] (EE) Caught signal 11 (Segmentation fault). Server aborting

I haven’t installed VirtualBox modules, I guess they are installed by Tumbleweed.

This is the list of packages installed on the VirtualBox host machine. The host machine is a OpenSuse Leap 42.1. I have other virtual machines running properly on it.


virtualbox-5.0.10-10.1.x86_64
virtualbox-host-kmp-default-5.0.10_k4.1.12_1-10.1.x86_64
virtualbox-guest-desktop-icons-5.0.10-10.1.noarch
virtualbox-guest-tools-5.0.10-10.1.x86_64
virtualbox-guest-kmp-default-5.0.10_k4.1.12_1-10.1.x86_64
virtualbox-qt-5.0.10-10.1.x86_64
virtualbox-guest-x11-5.0.10-10.1.x86_64

I found this bug related to my problem. It seems that only happens when you install Tumbleweed with Gnome as default desktop in VirtualBox, as it happens to me.

Yes, it’s a “bug” in GDM, it doesn’t work with video drivers that don’t support KMS (e.g. vboxvideo, vesa, fbdev), because it runs Xorg as unprivileged user so it has insufficient rights to actually access the graphics card.

Use a different display manager (e.g. lightdm, kdm, sddm) and it should work.

I installed LightDM and now works fine. Thank you for the support :wink:

On Sat, 19 Dec 2015 18:16:02 +0000, wolfi323 wrote:

> feliciano28;2743889 Wrote:
>> I found this ‘bug’
>> (https://bugzilla.opensuse.org/show_bug.cgi?id=929697) related to my
>> problem. It seems that only happens when you install Tumbleweed with
>> Gnome as default desktop in VirtualBox, as it happens to me.
>
> Yes, it’s a “bug” in GDM, it doesn’t work with video drivers that don’t
> support KMS (e.g. vboxvideo, vesa, fbdev), because it runs Xorg as
> unprivileged user so it has insufficient rights to actually access the
> graphics card.
>
> Use a different display manager (e.g. lightdm, kdm, sddm) and it should
> work.

Another option would probably be to do what Sebastien added to the ATI
driver package - modifying the permissions on /usr/bin/X. Looking at the
script, the fix he uses is:


chmod ug+s /usr/bin/X

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

The workaround I’ve been using is to boot to runlevel 3 and then sudo -s and startx which then lets me in. Will see if the
chmod ug+s /usr/bin/X has the same effect

Yes, this works… how does this fix get incorporated into the main build?

It is not broken it is a security measure. So nothing to fix.

You should probably set that in /etc/permissions.local though.
Otherwise your change will be overwritten when installing updates.

OK, but the install doesn’t work out of the box on a popular free virtualisation system - surely if we want users to try and use linux, there needs to be some form of capturing of this, either at install time, or when it occurs advising what a user can do to bypass the security measure and actually use the product they’ve downloaded?

Thanks, will do.

Just to be clear here: this affects only Tumbleweed, and only if you use gdm (i.e. a default GNOME installation).
For Leap 42.1 gdm has been downgraded to 3.14 shortly before the release to prevent this problem.

A discussion about how to fix the actual problem (and not workaround it by reopening a security hole that was closed years ago) is going on here:
http://lists.opensuse.org/opensuse-gnome/2016-01/msg00010.html
In the special case of VirtualBox/vboxvideo, work is going on by the VirtualBox developers to “port” vboxvideo to support KMS which would make this work again out of the box without running Xorg as root.