Mod_perl app causes Apache to segfault

I am writing a Catalyst app and deploying on a server running openSUSE 11.4, everything was running perfectly up until a couple of days ago when I applied updates to both the server and my app. Now every time I try to start Apache it goes through the start up process, even gets as far as printing “done” on the terminal and it doesn’t show up from a “ps -axf”. When I check dmesg I can see the following:

httpd2-prefork[21582]: segfault at 7f1cac9f84c0 ip 00007f1cac9f84c0 sp 00007fffd98d11d8 error 14 in librt-2.11.3.so[7f1cae229000+8000]

There is also nothing in /var/log/apache2/error_log or /var/log/apache2/rcapache2.out

I know that the virtual host that loads my catalyst app is at fault because if I move the vhost file out of /etc/apache2/vhosts.d and start Apache everything loads fine. I’m sure that my app is not solely to blame for this because it loads just fine using the development server that Catalyst ships with.

The vhost file for my Catalyst App can be viewed here http://cloud.vendion.net/apps/files_sharing/get.php?token=9e6836a8d9e633e2c81390d73423beb92df489aa

Various useful information:
openSUSE 11.4 (Yes I know a week away from 12.1)
Apache2 2.2.17
mod_perl 2.0.5
Catalyst runtime 5.90006

If I left something out or more information is needed I would be glad to give more information.

On Wed, 09 Nov 2011 01:36:03 +0000, vendion wrote:

> Various useful information:
> openSUSE 11.4 (Yes I know a week away from 12.1)
> Apache2 2.2.17 mod_perl 2.0.5 Catalyst runtime 5.90006
>
> If I left something out or more information is needed I would be glad to
> give more information.

What you might do is run apache in the foreground under gdb to determine
more precisely what it is that caused the problem.

I ran into something similar with mod_php5 on 12.1 RC1 a week or so ago,
and found that if I manually started apache thusly:

gdb httpd-prefork

And then within gdb:

run -f /etc/apache2/hosts.conf -X

That’ll run it in the foreground and use a single worker thread so it can
be debugged.

You might also want to install the relevant -debug packages so you get
symbolic information back.

When it bombs, type:

bt

In gdb and that’ll give you a stack trace that should get you going in
the right direction (or be of use in a bug you submit on the issue).

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

vendion wrote:
> Code:
> --------------------
> httpd2-prefork[21582]: segfault at 7f1cac9f84c0 ip 00007f1cac9f84c0 sp 00007fffd98d11d8 error 14 in librt-2.11.3.so[7f1cae229000+8000]
> --------------------

Good suggestions from Jim; follow those!

Also, have you asked on the Catalyst list or IRC?

Do you, or anybody else, have any idea why librt was running or even linked?

Cheers, Dave

No haven’t asked on the Catalyst list yet, I want to eliminate the possibility of the problem being in Apache/mod_perl first.

I don’t know why librt is being loaded and or what it really does…

I’ll try running it through gdb later today

I ran Apache through gdb and got this error from it:


Syntax error on line 7 of /etc/apache2/vhosts.d/www.vendion.net.conf:
Invalid command 'PerlSwitches', perhaps misspelled or defined by a module not included in the server configuration

Program exited with code 01.

Unless something has changed that is correct syntax and spelling on line 7 of my virtual host file, even the Catalyst docs has this Catalyst::Manual:: Deployment

vendion wrote:
> Code:
> --------------------
>
> Syntax error on line 7 of /etc/apache2/vhosts.d/www.vendion.net.conf:
> Invalid command ‘PerlSwitches’, perhaps misspelled or defined by a module not included in the server configuration
>
> Program exited with code 01.
>
> --------------------

It sounds like you don’t have mod_perl loaded.

But I don’t see any connection between this error and the previous
segfault. Perhaps its just my lack of knowledge, or perhaps there’s some
other factor at work as well. Did anything else change?

Other than this line:

 <Location />
	SetHandler	modperl
	PerlResponseHandler	Ickarus
 </Location>

how do I explicitly load mod_perl in my host?

The only thing that has changed recently that I can remember is PAM updated which required Apache to restart and updating the Catalyst::Runtime engine from 5.8 to 5.9.

vendion wrote:
> how do I explicitly load mod_perl in my host?

YaST / HTTP Server / Expert Configuration / Server modules

:stuck_out_tongue: I know how to enable it, and it is already enabled in Apache, I just wasn’t sure if there was something I could add to the virtual host file to make sure it is being loaded, I’m also sure it is loading because if I enable debugging in my Catalyst app and try to start Apache normally I see were it renders the app and loads it just before crashing, although it doesn’t show as crashing here is the output of rcapache2 start with debugging enabled in the app -> #2843329 - Pastie](http://pastie.org/2843329). I don’t think Apache would get that far without mod_perl

I am getting this same segfault EXACTLY with OpenSuSE 11.4 OR 12.1 and the “Freeside” ISP billing software program. There must be some issue in a common module that they both use.

Anybody made any headway on this issue?

ducharme wrote:
> I am getting this same segfault EXACTLY with OpenSuSE 11.4 OR 12.1 and
> the “Freeside” ISP billing software program. There must be some issue in
> a common module that they both use.
>
> Anybody made any headway on this issue?

The conversation continued on the Catalyst list, starting here:
<http://www.mail-archive.com/catalyst@lists.scsys.co.uk/msg12326.html>

I think the OP switched to fastcgi, but I’m not entirely certain what
the outcome was. If there is an issue with an opensuse RPM, it would be
useful for somebody to nail down exactly what it is, so please post back
with whatever you discover.