Novell/SUSE Login Outage

Just as a heads up, we had an issue with the user database today that affected logins for all of the Novell and SUSE sites, including forums. The issue lasted about an hour, during which it was not possible to authenticate. This did not affect public users or those who were already authenticated.

The admins for the database are still looking into the root cause. Sorry about the inconvenience.

It hit me earlier

We had another smaller version of it about an hour ago. There are still some syncing/replication issues from what I’m told, so it could continue to happen intermittently throughout the afternoon. The admin is working on getting it stable.

Thanks for the warning and an update.

is the outage in any way related to performance hit the forums have taken this evening (my time)?
For the past couple of hours performance has reverted to old.
Trying to login in Chrome I got - server took too long to respond - message.
When tried to log in again got a very simple message - server error.

Some pages I tried to load got the message - invalid page

On 11/05/2012 05:46 PM, farcusnz wrote:
>
> is the outage in any way related to performance hit the forums have
> taken this evening (my time)?

about 17.15 UTC

yep, something is going wrong…chrome reports “Waiting for
forums.opensuse.org…” until it times out…

while isup.me reports: “It’s just you. http://forums.opensuse.org is up.”

and
traceroute forums.opensuse.org
traceroute to forums.opensuse.org (130.57.66.6), 30 hops max, 40 byte
packets using UDP
1 192.168.1.1 (192.168.1.1) 0.255 ms 0.271 ms 0.249 ms
2 ge-1-3-0-1103.hsnqu1.dk.ip.tdc.net (87.55.151.65) 21.362 ms
21.176 ms 21.186 ms
3 as0-0.nyk2nqp1.us.ip.tdc.net (83.88.29.23) 115.751 ms 115.782 ms
115.243 ms
4 146.82.35.201 (146.82.35.201) 115.216 ms 115.992 ms 115.825 ms
5 0.ae11.BR3.NYC4.ALTER.NET (204.255.168.193) 131.392 ms 132.022
ms 131.268 ms
6 0.so-5-1-0.XT2.DEN4.ALTER.NET (152.63.1.138) 178.458 ms 178.276
ms 178.144 ms
7 POS7-0.GW3.DEN4.ALTER.NET (152.63.72.73) 180.577 ms 179.554 ms
179.735 ms
8 unknown.customer.alter.net (65.206.183.22) 188.664 ms 188.811 ms
188.649 ms
9 192.94.118.247 (192.94.118.247) 191.226 ms 192.978 ms 191.226 ms
10 forums.opensuse.org (130.57.66.6) 192.371 ms 191.468 ms 192.402 ms


dd http://goo.gl/PUjnL
http://tinyurl.com/DD-Caveat http://tinyurl.com/DD-Hardware
http://tinyurl.com/DD-Software

Yes, we’re still experiencing some of the problem Matt explained above.
It’s being worked on.


Kim - 11/5/2012 1:09:56 PM

Most of problems I had at the weekend seemed to be with the novell front-end - login not achieved and errors, although I could still browse the forum.

Good luck with repairs. :slight_smile:

> Yes, we’re still experiencing some of the problem Matt explained above.
> It’s being worked on.

wow…that is a shame…i guess i got too happy about everything being
smooth, i thought finally we had turned that corner…

well, pat Matt on the back for the smooth days, and encourage him to
keep trying…we are all pulling for him, you, Jim and all else who are
trying to provide success…


dd

On 2012-11-05 21:10, kgroneman wrote:
> Yes, we’re still experiencing some of the problem Matt explained above.
> It’s being worked on.

I can not log to Bugzilla, all day I think.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))

On Mon, 05 Nov 2012 21:09:05 +0000, Carlos E. R. wrote:

> On 2012-11-05 21:10, kgroneman wrote:
>> Yes, we’re still experiencing some of the problem Matt explained above.
>> It’s being worked on.
>
> I can not log to Bugzilla, all day I think.

Yes, bugzilla would have been affected as well.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On Mon, 05 Nov 2012 23:43:09 +0000, Jim Henderson wrote:

> On Mon, 05 Nov 2012 21:09:05 +0000, Carlos E. R. wrote:
>
>> On 2012-11-05 21:10, kgroneman wrote:
>>> Yes, we’re still experiencing some of the problem Matt explained
>>> above.
>>> It’s being worked on.
>>
>> I can not log to Bugzilla, all day I think.
>
> Yes, bugzilla would have been affected as well.

I should mention that I was in bugzilla a lot this afternoon for work-
related tasks and at least this afternoon, it didn’t seem to be having
any issues. (Of course, I’m talking afternoon local time for me, which
is Utah).

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

Hey guys, quick update…

As you know, we had a pretty rough weekend and Monday. However, I think we are pretty stable now. I am not aware of any outages since Monday morning, although logins may still be a little slow.

I have been working closely with the directory administrator on the issue. We have found that once a day, some process is touching all the users and therefore causing the synchronization to be horribly backed up. We think that we have identified the process doing this, and I turned it off. We will see tonight if this has worked. It’s possible that we may take one more outage for a few minutes for a server move, but I don’t think it will be necessary.

For those looking who want to look at the bright side, there is some good that is coming of this. The fact that our user directory is huge and is mostly filled with junk (spam accounts, etc.) has not helped with the synchronization issues. This has put a lot of priority on the task of cleaning out the spam accounts and putting in a more aggressive process for keeping those accounts from being created in the first place. In the long run, I think this will be good for the forums.

On 2012-11-07 23:36, MatthewEhle wrote:

> For those looking who want to look at the bright side, there is some
> good that is coming of this. The fact that our user directory is huge
> and is mostly filled with junk (spam accounts, etc.) has not helped with
> the synchronization issues. This has put a lot of priority on the task
> of cleaning out the spam accounts and putting in a more aggressive
> process for keeping those accounts from being created in the first
> place. In the long run, I think this will be good for the forums.

which is one of the reasons why some of us think that, although having
the same login as for bugzilla is convenient, maybe the forums should
have a different login database or whatever.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))

On Thu, 08 Nov 2012 02:28:12 +0000, Carlos E. R. wrote:

> which is one of the reasons why some of us think that, although having
> the same login as for bugzilla is convenient, maybe the forums should
> have a different login database or whatever.

Thank you (again) for your input.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 2012-11-08 03:33, Jim Henderson wrote:
> On Thu, 08 Nov 2012 02:28:12 +0000, Carlos E. R. wrote:
>
>> which is one of the reasons why some of us think that, although having
>> the same login as for bugzilla is convenient, maybe the forums should
>> have a different login database or whatever.
>
> Thank you (again) for your input.

Welcome :slight_smile:
I’m aware that you think differently and you have your reasons, so I’m
not trying to push; I’m just expressing an opinion (without internal
knowledge).

:slight_smile:


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))

On Thu, 08 Nov 2012 02:43:34 +0000, Carlos E. R. wrote:

> On 2012-11-08 03:33, Jim Henderson wrote:
>> On Thu, 08 Nov 2012 02:28:12 +0000, Carlos E. R. wrote:
>>
>>> which is one of the reasons why some of us think that, although having
>>> the same login as for bugzilla is convenient, maybe the forums should
>>> have a different login database or whatever.
>>
>> Thank you (again) for your input.
>
> Welcome :slight_smile:
> I’m aware that you think differently and you have your reasons, so I’m
> not trying to push; I’m just expressing an opinion (without internal
> knowledge).
>
> :slight_smile:

Which is why I simply thanked you. :slight_smile: No need to rehash an old
discussion. :slight_smile:

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 2012-11-08 03:46, Jim Henderson wrote:
> Which is why I simply thanked you. :slight_smile: No need to rehash an old
> discussion. :slight_smile:

But he does not know, that’s why…


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))

On Thu, 08 Nov 2012 03:44:13 +0000, Carlos E. R. wrote:

> On 2012-11-08 03:46, Jim Henderson wrote:
>> Which is why I simply thanked you. :slight_smile: No need to rehash an old
>> discussion. :slight_smile:
>
> But he does not know, that’s why…

Well, he does know, because we’ve discussed it in the past.

So, no, no need to rehash the old discussion. :slight_smile:

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 2012-11-08 05:59, Jim Henderson wrote:
> On Thu, 08 Nov 2012 03:44:13 +0000, Carlos E. R. wrote:

> Well, he does know, because we’ve discussed it in the past.

A sorry, I did not know that. Or have forgotten if I knew O:-)


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” (Minas Tirith))