Mission critical OpenSuse server?

English Open Chat
1

Perhaps this is a bit silly question, but I was wondering if anyone is deploying OpenSuse for “a certain level of mission critical” web server, and would like to share experience?

By “certain level of mission critical” I assume:

  • Updates as patches to currently installed packages only, ie. no package upgrades unless manually requested
  • Occasional quick reboots for critical kernel updates allowed (say, once a month).
  • Specific software selection required: lighttpd (1.4.19 or newer), PHP 5.2(.10) and 5.3, Postgres 8.3.4 (or newer), Sun’s Java 6.
  • Server used to host professional services, ie. not a home, or intranet/office web/file/samba/blah server.

Having used OpenSuse before, but never deployed on such a server, I know that the requirements ARE there, especially in that official software selection is not as ancient as on Debian or CentOS, which is very important, and that there are patch only updates via zypper.

Still, as far as I gather, OpenSuse is not really a favorite server OS among (dedicated) web server users, and I was wondering why is that?

Is anyone here deploying OpenSuse in such a “critical” environment successfully willing to share their experience?

I understand I can probably get more “mission critical” with SLES, and I am considering it, but I’d like to start with OpenSuse and see how it goes, because I am quite capable of maintaining a server.

Thanks.

2

> Still, as far as I gather, OpenSuse is not really a favorite server OS
> among (dedicated) web server users, and I was wondering why is that?

CAVEAT: I have never deployed or maintained “a certain level of
mission critical” web server…so, i have no experience in doing that
BUT, there are some things (imo) you need to consider:

  • the normal lifetime of any particular openSUSE version is TWO years
    from release date to no more security patches or updates!
    cite: http://en.opensuse.org/SUSE_Linux_Lifetime

  • since about 10.0 it has been very difficult to mostly impossible to
    upgrade from one version to the next…most all of the ‘gurus’ here
    recommend a format reinstall with /home on a separate partition so
    your ‘stuff’ gets kept…and the upgrade method is officially
    unsupported! cite: http://en.opensuse.org/Upgrade

  • on any day, at anytime an update might be released which causes your
    machine to no longer work as expected!
    cite: http://forums.opensuse.org/search.php?searchid=1164737

  • there was a time, long ago that SuSE was a ‘company’ in Germany
    which released software when it was ready to be released…and, that
    meant as stable and dependable as a small group of folks could make
    it…and, when released it was ‘done’ until the next time a new
    version was “ready”…as far as i can tell the last of those releases
    was either 9.3 or 10.0, since then openSUSE has been ‘sponsored’ by
    Novell and versions have been released on a SCHEDULE that had nothing
    to do (imo) with stability or dependability…it is just a date
    certain that a new number would be used…the experience of new
    version problems got progressively worse from 10.1 though to 11.1

  • today, openSUSE is the bug smashing grounds where Novell allows us
    of the “openSUSE community” to hammers out the kinks and then, when
    something is ‘ready’ they release a new, commercial SLED/SLES version,
    which is then supported for (what?) five or more years…(sorry, i
    can’t find the support map on the Novell Site…heck, i can’t find
    ANYTHING on the Novell site in a reasonable amount of time)

i’m sitting on a home machine connected to the internet and no other
boxes, and i feel my machine is important enough to me that i want it
both stable and dependable, and so i’m still on 10.3 and considering
what i’m gonna do when i must move in just a few months because it
will be no longer supported…i’ll probably go to SLED 11 or CentOS,
or debian, or slack, or or or

on the other hand, YOUR millage may vary…you may like the idea of
being forced to either update every six months to a version ‘seasoned’
for more than six months, or every two years to an, unseasoned, brand
new experience…

AND, if you wait a bit eventually someone will come along and tell you
they operate a huge server farm and have had a 99.999% uptime rate
with openSUSE 11.1, when they do, get their street address and go have
a look for yourself.

–
goldie

3

Thanks for your input.

I’d very much like to deploy OpenSuse because I think it has the optimal age of officially supported packages in stable. The lifetime of 2 years is more than acceptable.

For our needs, we require Java 1.6, PHP 5.2, soon 5.3, and Postgres 8.3 as mission critical services. In addition to that, I’d like to harden the server with AppArmor. SELinux is disgustingly difficult to manage, and I’m not sure how good a friend I’d find in grsecurity. Also Xen, in the beginning our server will be deployed as a Xen guest, and I believe OpenSuse is at home here.

This and many other things accumulate for me to prefer OpenSuse above CentOS/RHEL or Debian. Ubuntu is really out of the question, and I am counting days before our current Gentoo powered server says kthnxbye because glibc and some other packages are increasingly becoming required by other updates which I’m holding off even on the staging server. If anyone ever used gentoo, they know how big the can of worms called Glibc Update is, especially when handed a can opener called New Python version.

So I guess to me it boils down to using one of:

  • CentOS hacked with unofficial PHP 5.2/5.3, Postgres 8.3 packages and hope nothing breaks.
  • OpenSuse with officially supported apps we need.
  • SLES as the more enterprise friendly variant of OpenSuse.
4

So long as you have stable package selections to begin with, and you don’t update (or if you do, check through the update list very carefully), I don’t see why you couldn’t run openSUSE on a mission critical server indefinitely (that is to say, until the server is no longer needed or becomes obsolete).

A lot would rely on your expertise in managing a server though. The community forums, while good for most people, certainly aren’t designed to provide mission-critical support. Online documentation is sporadic in quality (in some areas it is excellent, while in others it is non-existent).

If you did want some kind of “official” support for the server, you could buy theboxed version of openSUSE from Novell. That actually provides you with 90-day installation support from Novell. You could even think of it as a test bed for possibly running SLES in the future; see how well their support is, how well the product holds up, etc.

5

If you are really serious about ‘mission critical’ you should be going for the Enterprise Server and an appropriate support package.

I don’t think openSUSE will let you down but you won’t get access to the support you might need if something went wrong. If you look at the monthly statistics you will see that the average time to answer queries on most subforums on this forum is over 24 hours - not really what you want in a crisis.

6

Depends on how critical is “mission-critical”. An e-commerce site’s idea of critical is different from a information site’s idea of critical. You haven’t told us how long the site can afford to be down in case of problems. If it’s in the matter of minutes rather than hours, perhaps you need hardware redundancy.

There’s no doubt that Linux will chock up uptimes of months or even years if there are no disrupting factors, like kernel updates. If the support period of 2 years suits you, and you are capable of patching and resolving sysadmin issues on your own, then openSUSE makes a fine server.

I understand how you feel about RHEL/CentOS. I have no problems with their update support. Only problem is for some packages the versions are so prehistoric. I too am using third-party provided PHP packages.

@john_hudson: I don’t know that the average time to answer on these forums is a good measure for mission support. First you have to remove all the queries relating to setup, new hardware, copying to new system, programming questions, etc. Presumably those kinds of issues would have been resolved before deployment. As for the real emergencies, like RAID1 broken how do I reassemble, that depends on the expertise of the maintainer. And I’ve seen that some of those questions never get answered on the forum, which makes it far worse than 24 hours. But for sure if you pay Novell, or a qualified geek, to be on hand to help, if it is out of the depth of the maintainer, then you can lower the time to resolve.

7

Thanks for all your input, folks! :slight_smile:

I guess I cannot know whether OpenSuse is stable enough for me until I deploy it and try it out. At least, I can afford doing that, and switch to SLES if I am continually unable to fix potential issues myself. I’m not new to server administration.

Perhaps I should have asked a different question:

Are you deploying OpenSuse in situations that you consider “mission critical”, and can you describe your experience?

8

Possibly the best qualified to answer that question is user @Chrysantine
If she see this!?

9

Why’s that :stuck_out_tongue:

Well I run a few hundred servers of which about 20 can be classified as “ultra mission critical” (where-as them going down would cost considerable amount of money) and another 50 or so can be considered “really bad” if they did (customers would effectively lose the service completely or at least disrupt as some segments would fall).

They’re all in a closed network and running a mix of openSuSE 10.3 and 11.1 on HP’s DL series of rack servers, most of them are 2.33, 2.66 and 2.93 Xeon Dual/Quad Core setups - most them still running x86 for certain hardware and software compatibility reasons. There’s not much magic to them, they run a pretty simple high-availability setup that reports their status, SNMP for gathering information (as it’s a closed network, it’s quite safe) and I have a commercial program that compiles the information for me and alerts when something breaks (which 99 out of 100 times means a hardware failure)

The only trouble so far I’ve had with openSuSE as a server has been that on some machines a kernel update has failed to create a new initrd that contains all the necessary drivers (in this case the raid driver) - fixing it was pretty trivial and didn’t take much time. Since it was during an allocated maintenance period, it wasn’t even that bad.

I can only suggest that if you roll out mission critical platforms, make a VM out of the machine after you’ve installed it, snapshot it and drop any updates/changes on it - then see how they perform and roll out the same changes on the actual production machine if you need to do so at all.

Oh and we run openSuSE on all our web servers and company file servers too without a hitch. Never been broken into so far, although we’ve hardened them a bit.

If you got any questions, just ask - and buy HP servers :>

10

I manage some servers running openSUSE. Stability is not an issue at all. As with pretty much any distro, Linux is rock solid. I have encountered issues with one or two things breaking due to updates. For example once a squirrelmail update contained syntactic errors, which were quickly fixed, I must say. I have had to be careful with ocfs2 updates because they once pushed out an update which required updating all the servers in the cluster simultaneously and then rebooting, something they would not have done in SLES. And the latest 11.1 release contained a show-stopper kernel bug so I had to hold off the upgrade to 11.1 until that was fixed. But all in all, I would continue to use openSUSE. Partly because openSUSE packages are modern enough to cope with software I am asked to install (these are education servers), partly because of the wide range of software available, reducing the need for me to roll my own and then having to track security fixes for those.

11

Great answers, thanks!

Other than the usual (public keys, port changes, chroots, intrusion detection, etc…) are you using AppArmor?

12

I don’t use AppArmor.

On RHEL servers I maintain, I have left SELinux enabled. I have worked with it and not tried to defeat it.

AppArmor/SELinux protect against a certain class of bugs. I regard it as extra security but not sufficiently compelling for the kind of mission my servers handle. Plus most of the current exploits tend to be against faulty web apps. AA/SEL cannot protect against those.

13

Excellent advice here thus far. I’ll just poke in my 2 cents’ worth. And that’s all it’s worth. Someone like Chrysantine, who maintains a farm of the bloomin’ things, or ken_yap, who has apparently been doing this since the Big Bang, definitely has an opinion that carries more weight. :slight_smile:

That said, I’ve had no real problems with OpenSUSE – with ONE exception. If there is a kernel update, I always cringe. About 1 time out of 5 (that’s a guess), it will break something. Since I’m administering the Web server (which is in Denver) remotely (from Birmingham, AL), this naturally causes a wee bit of distress. :slight_smile:

Also (someone here can correct me if I’m wrong), it seems that OpenSUSE will NOT automatically update all of your servers (in particular, Apache). Other “enterprise” systems will.

OpenSUSE’s big, big strength, in my opinion, is Yast. That thing continues to set THE standard for usable, excellent system configuration – even over SSH in a text terminal. It just can’t be beat.

However, we did have occasional, weird, unexplainable hangs with our mail server (primarily with Apache Tomcat) under Opensuse 10.3 and Opensuse 11. When I switched to CentOS, those hangs went away. I miss Yast with CentOS, but by installing Webmin, I’m able to do a lot of it in a remote Web browser (including adding and removing virtual hosts, which is one of Yast’s big strengths).

Ergo and therefore, if you can swing the bucks (and it’s really not that expensive), go ahead and get Suse Enterprise. You get Yast WITH excellent support. If you’re on a budget, but want Enterprise features (including automatic updates of servers such as Apache), CentOS is a good choice.

Just my 2 cents’ worth. Tape a dollar bill to it, and it’ll get you a (small) cup of coffee at your nearest fast food joint. :slight_smile:

14

I run 35 internal servers at work, some of them openSUSE 11.0 and some 11.1. None of these are considered mission critical but one, which needs to be up constantly and if HW breaks, it needs to be up ASAP. Since this one is also an internal server, I never apply kernel updates to it so rebooting virtually never happens during the lifetime of the OS version. Our public web server runs Debian though, but I’m not responsible for it (my colleague is who’s also my senior (I’m a junior :P)) but we’re currently investigating, actually he is not me, in trying to add ksplice support to all servers so we won’t have to reboot them when kernel updates come. Upgrade of the internel openSUSE servers is in the form of backing up important config files and running zypper dup which has worked without a problem for us so far.

15

i am, by the way, VERY happy to hear all the good experiences the pros
have had with openSUSE…

i guess i’ve just been reading far to many problems here which don’t
really apply to folks who (hmmmmm, trying to figure out how to say
this without hurting any feelings) … well, professional system
administrators certainly seem to have better luck with openSUSE than
do new folks … HUH??

i’ve been with this lizard since 9.x (maybe 8.x, i’m not sure—i
hopped over about the time RH went to subscription) anyway, maybe it
is time for me to stop thinking about bolting and install 11.1 over my
10.3 (but it will NOT have KDE4 as prime)

–
goldie

16

The only thing I’d add … and I’m not trying to dismiss another distro, or start a flame war … is NOT to use Fedora. It’s just unsuitable for a serious, publicly-exposed server.

I’m not dismissing them because to be fair, that’s not Fedora’s purpose. Its purpose is to serve as a testbed for RHEL. But you might use OpenSuse (which likewise serves as a testbed for Suse Enterprise), say, “hey, that’s pretty good!” … and decide that Fedora must likewise be similar to the RHEL product.

Ummm … just my opinion, but no, it’s not. Not by a long shot.

17

Yes but in a very limited manner by running the almost-out-of-the-box settings.

Since many of the public web servers run PHP+MySQL+Tomcat, taking care of their protection is more important than anything else. PHP hardening with various tools like Suhosin and making sure the SQL code itself is sane goes a long way.

Some of the public web services are virtualised so even if someone did break in, the damage would be insignificant as I can roll it back in a few minutes.

18

CanOfWires wrote:
> Perhaps this is a bit silly question, but I was wondering if anyone is
> deploying OpenSuse for “a certain level of mission critical” web server,
> and would like to share experience?
>
> By “certain level of mission critical” I assume:
>
> - Updates as patches to currently installed packages only, ie. no
> package upgrades unless manually requested

Mostly true with regards to openSUSE. The only exceptions (IMHO) are when a massive flaw in a package is found which may force a move to something
different.

> - Occasional quick reboots for critical kernel updates allowed (say,
> once a month).

Quick ONLY if you do NOT have proprietary drivers on your system. They yes… mostly true, but again, if the kernel needs updating, it will get
updated… and might mean more than one a month. Security is important to many poeple…

> - Specific software selection required: lighttpd (1.4.19 or newer), PHP
> 5.2(.10) and 5.3, Postgres 8.3.4 (or newer), Sun’s Java 6.

No… that could lead to a lack of ‘mission critical’ operation. The packages that come with openSUSE will receive patches for security issues. But
no (in general) feature releases are done. However, a community repository (outside of any openSUSE support) may have newer packages available, but
probably not something that fits the ‘mission critical’ requirement.

> - Server used to host professional services, ie. not a home, or
> intranet/office web/file/samba/blah server.

Certainly.

>
> Having used OpenSuse before, but never deployed on such a server, I
> know that the requirements ARE there, especially in that official
> software selection is not as ancient as on Debian or CentOS, which is
> very important, and that there are patch only updates via zypper.

And again, patches are usually NOT feature oriented… mostly security updates triggered directly or indirectly. Do NOT expect all things that are
‘broken’ to be fixed via a patch. The fixes many not happen until the next full release of openSUSE.

>
> Still, as far as I gather, OpenSuse is not really a favorite server OS
> among (dedicated) web server users, and I was wondering why is that?
>

Uh… well… that’s a lie. Where on earth did you hear that? If you’re going to use a community based distro, openSUSE is BETTER in most cases than
the other community based distros. It has a wiser understanding of security and interoperability than most.

> Is anyone here deploying OpenSuse in such a “critical” environment
> successfully willing to share their experience?

No. Why? Because openSUSE IS a community based distro. For ‘mission critical’ deployments you need to consider Novell’s SUSE Linux Enterprise
Server. Why? Because it is VERY well supported (long term support).

>
> I understand I can probably get more “mission critical” with SLES, and
> I am considering it, but I’d like to start with OpenSuse and see how it
> goes, because I am quite capable of maintaining a server.

Your ‘mission critical’ server WILL REQUIRE upgrade and possible full reinstall about every 2 years. If that’s acceptable, openSUSE may fit your
needs. In the process of that upgrade, it is possible that some packages will be removed from the distribution. It is also possible that feature
level upgrades of certain packages will render your existing configurations unusable without modification. These are some of the things to consider
when using a more bleeding edge community distribution like openSUSE. The currently well maintained SLES 9 SP4 dates originated from SUSE Prof. 9.1,
the well supported SLES 10 SP2 come from openSUSE 10.1 days and SLES 11 comes from the openSUSE 11.1 time period. In all cases, the product is well
maintained for newer hardware additions, but you might find SLES 9 to not have all of the features you need. SLES 10 probably does. If you need very
recent features, then SLES 11 is key. I would advocate that anything ‘mission critical’ use the enterprise product line… with that said, I have
administered environments for many years that used the free stuff. There WILL be occasions when you’ll HAVE to compile packages (possible large ones)
on your own due to security issues or features that are not perceived as critical by the community.

If you have NO MONEY… AND you need a top notch Linux distro, I think openSUSE is best.

If you’re ‘mission critical’ needs are for a revenue producing business (for example), then go with the enterprise line.

19

Uh… well… that’s a lie. Where on earth did you hear that?

It is my experience from hosting various (custom) web applications with various hosting companies over the years. OpenSuse is rarely found, it’s usually CentOS, Ubuntu, Debian, and Fedora (sic!), and some offer Arch, Gentoo and FreeBSD. Of course, you can ask for custom installation and get OpenSuse, but not all of them will do that.

I am particularly confused by the fact that Archlinux, which is a fine minimalist desktop OS, is being offered for servers, which is the last role that distro should be taking, given the speed and quality of updates – and OpenSuse is not.

It is my observation that hosting companies see only CentOS and RHEL in the same sentence with “mission critical” or “enterprise grade”.

However, there are levels of “mission critical”, naturally. That’s why I corrected myself and asked if anyone is deploying OpenSuse in situations which they consider “mission critical”, and what that is. I’ve got my answer. :slight_smile:

20

You wouldn’t happen to be living in US/Canada Can’o’Wires?

Because SuSE is vastly more popular here in ol’ Yurop(R).