Welcome to emergency mode!

Since upgrading to openSUSE 12.3 (Dartmouth) (x86_64) with Linux 3.7.10-1.1-desktop and KDE 4.10.00 “release1” on an AMD Athlon 64 Processor 3200+, I usually boot into emergency mode! I don’t think systemctl has ever failed me, but would be glad to escape this diversion, so would be grateful to anybody who can understand the following system boot log and advise me what I have to do to return to normal:
Apr 02 10:43:50 UsrFriendesktop.Lisheen systemd-journal[227]: Allowing runtime journal files to grow to 100.3M.
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Initializing cgroup subsys cpuset
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Initializing cgroup subsys cpu
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Linux version 3.7.10-1.1-desktop (geeko@buildhost) (gcc version 4.7.2 20130108 [gcc-4_7-branch revision 195012] (SUSE Linux) ) #1 SMP PREEMPT Thu Feb 28 15:06:29 UTC 2013 (82d3f21)
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-3.7.10-1.1-desktop root=UUID=7417c0e6-33e0-4f84-ad65-1a84ce56c0d7 resume=/dev/disk/by-id/ata-ST3160827AS_4MT0BNMF-part5 splash=silent quiet showopts
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: e820: BIOS-provided physical RAM map:
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: BIOS-e820: [mem 0x00000000000e4000-0x00000000000fffff] reserved
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: BIOS-e820: [mem 0x0000000000100000-0x000000007ff2ffff] usable
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: BIOS-e820: [mem 0x000000007ff30000-0x000000007ff3ffff] ACPI data
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: BIOS-e820: [mem 0x000000007ff40000-0x000000007ffeffff] ACPI NVS
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: BIOS-e820: [mem 0x000000007fff0000-0x000000007fffffff] reserved
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: BIOS-e820: [mem 0x00000000fff80000-0x00000000ffffffff] reserved
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: NX (Execute Disable) protection: active
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: SMBIOS 2.3 present.
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: DMI: To Be Filled By O.E.M. To Be Filled By O.E.M./K8V-X, BIOS 1012.003 09/12/2005
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: e820: update [mem 0x00000000-0x0000ffff] usable ==> reserved
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: e820: remove [mem 0x000a0000-0x000fffff] usable
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: AGP bridge at 00:00:00
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Aperture from AGP @ f4000000 old size 32 MB
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Aperture from AGP @ f4000000 size 64 MB (APSIZE f30)
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: e820: last_pfn = 0x7ff30 max_arch_pfn = 0x400000000
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: MTRR default type: uncachable
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: MTRR fixed ranges enabled:
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: 00000-9FFFF write-back
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: A0000-EFFFF uncachable
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: F0000-FFFFF write-protect
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: MTRR variable ranges enabled:
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: 0 base 0000000000 mask FF80000000 write-back
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: 1 disabled
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: 2 disabled
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: 3 disabled
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: 4 disabled
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: 5 disabled
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: 6 disabled
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: 7 disabled
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: found SMP MP-table at [mem 0x000ff780-0x000ff78f] mapped at [ffff8800000ff780]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: initial memory mapped: [mem 0x00000000-0x1fffffff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Base memory trampoline at [ffff880000099000] 99000 size 24576
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: init_memory_mapping: [mem 0x00000000-0x7ff2ffff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: [mem 0x00000000-0x7fdfffff] page 2M
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: [mem 0x7fe00000-0x7ff2ffff] page 4k
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: kernel direct mapping tables up to 0x7ff2ffff @ [mem 0x1fffc000-0x1fffffff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: RAMDISK: [mem 0x340e8000-0x3606bfff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: ACPI: RSDP 00000000000faa60 00014 (v00 ACPIAM)
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: ACPI: RSDT 000000007ff30000 00030 (v01 A M I OEMRSDT 09000512 MSFT 00000097)
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: ACPI: FACP 000000007ff30200 00081 (v01 A M I OEMFACP 09000512 MSFT 00000097)
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: ACPI: DSDT 000000007ff303e0 03632 (v01 A0091 A0091006 00000006 MSFT 0100000D)
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: ACPI: FACS 000000007ff40000 00040
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: ACPI: APIC 000000007ff30390 0004A (v01 A M I OEMAPIC 09000512 MSFT 00000097)
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: ACPI: OEMB 000000007ff40040 0003F (v01 A M I OEMBIOS 09000512 MSFT 00000097)
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: ACPI: Local APIC address 0xfee00000
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Scanning NUMA topology in Northbridge 24
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: No NUMA configuration found
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Faking a node at [mem 0x0000000000000000-0x000000007ff2ffff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Initmem setup node 0 [mem 0x00000000-0x7ff2ffff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: NODE_DATA [mem 0x7ff1c000-0x7ff2ffff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: [ffffea0000000000-ffffea0001bfffff] PMD -> [ffff88007d600000-ffff88007f1fffff] on node 0
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Zone ranges:
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: DMA [mem 0x00010000-0x00ffffff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: DMA32 [mem 0x01000000-0xffffffff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Normal empty
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Movable zone start for each node
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: Early memory node ranges
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: node 0: [mem 0x00010000-0x0009efff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: node 0: [mem 0x00100000-0x7ff2ffff]
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: On node 0 totalpages: 523967
Apr 02 10:43:50 UsrFriendesktop.Lisheen kernel: DMA zone: 56 pages used for memmap

On 2013-04-02 12:06, peterichardavis wrote:

> Since upgrading to openSUSE 12.3 (Dartmouth) (x86_64) with Linux
> 3.7.10-1.1-desktop and KDE 4.10.00 “release1” on an AMD Athlon 64
> Processor 3200+, I usually boot into emergency mode! I don’t think
> systemctl has ever failed me, but would be glad to escape this
> diversion, so would be grateful to anybody who can understand the
> following system boot log and advise me what I have to do to return to
> normal:

Please use code tags for printouts and commands. Advanced editor, ‘#’
button. Posting in Code Tags - A Guide. Or upload to susepaste.

Sorry, I don’t see anything of interest in your log. According to the
developers, you look in “systemd-journalctl” to find out what happened.

This lack of information by systemd, by the way, is “Bug 782904”.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

This is simply journalctl in 12.3. And in 12.3 it even says you to call it when entering emergency mode.

On 2013-04-02 16:36, arvidjaar wrote:
>
> robin_listas;2543748 Wrote:
>> “systemd-journalctl”
> This is simply journalctl in 12.3.

Ah.

> And in 12.3 it even says you to call
> it when entering emergency mode.

The problem is, as explained in that bugzilla, that log is useless to
find what was the actual problem that provoked emergency mode to start.

I may have to repeat the test in 12.3 and report again as/if necessary.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

Newer systemd became better in this respect, “journalctl -b” in emergency mode usually gives enough information which mandatory service failed to start. Of course, to find why it failed to start is more challenging.

Upstream systemd now finally preserves current services state when going in emergency mode, and this patch is queued to be released as update for 12.3 as well. It means you also will be able to use standard systemctl to check services state, see error output etc.

On 2013-04-02 17:16, arvidjaar wrote:
>
> robin_listas;2543782 Wrote:
>>
>> The problem is, as explained in that bugzilla, that log is useless to
>> find what was the actual problem that provoked emergency mode to start.
>>
>>
>> I may have to repeat the test in 12.3 and report again as/if necessary.
>>
>
> Newer systemd became better in this respect, “journalctl -b” in
> emergency mode usually gives enough information which mandatory service
> failed to start. Of course, to find why it failed to start is more
> challenging.

I’ll try this when my time permits… I’m interested.

What happens if the emergency mode fails badly and I have to use a
separate rescue system, is the information still available :-?

> Upstream systemd now finally preserves current services state when
> going in emergency mode, and this patch is queued to be released as
> update for 12.3 as well. It means you also will be able to use standard
> systemctl to check services state, see error output etc.

Mmm… :slight_smile:


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

Is this a question?

In addition to start and stop systemd supports “isolate” operation. Comparing to “start” in addition to starting all dependencies it also stops everything that is currently started but is not in list of dependencies. This is normally used to switch “run-levels” (doing “init 3” will stop X as example because X is not in list of services that would be started by multi-user.target).

Historically systemd performed “isolate emergency.target” when fatal errors were encountered during system startup. The effect was, almost all services that had failed were stopped. So when you tried to see failed services with “systemctl --failed” no services were listed. Because they no more were “failed”.

This patch changes it into “start emergency.target” (or .service, do not remember), so all state is preserved. You see immediately what failed and to some extent why.

What happens if the emergency mode fails badly and I have to use a
separate rescue system, is the information still available :-?

By default on openSUSE journal is volatile, so it is lost. You can make it non-volatile, but I’m not sure how often it is flushed to disk. Of course, if system is failed so badly that even emergency mode does not start, there is no guarantee journal could be started or it could write anything to disk.

On 2013-04-02 19:56, arvidjaar wrote:
>
> robin_listas;2543787 Wrote:
>>
>>
>>> Upstream systemd now finally preserves current services state when
>>> going in emergency mode, and this patch is queued to be released as
>>> update for 12.3 as well. It means you also will be able to use
>> standard
>>> systemctl to check services state, see error output etc.
>>
>> Mmm… :slight_smile:
>
> Is this a question?

My neurones trying to digest some difficult information and grumbling a
bit :slight_smile:

> In addition to start and stop systemd supports “isolate” operation.
> Comparing to “start” in addition to starting all dependencies it also
> stops everything that is currently started but is not in list of
> dependencies. This is normally used to switch “run-levels” (doing “init
> 3” will stop X as example because X is not in list of services that
> would be started by multi-user.target).
>
> Historically systemd performed “isolate emergency.target” when fatal
> errors were encountered during system startup. The effect was, almost
> all services that had failed were stopped. So when you tried to see
> failed services with “systemctl --failed” no services were listed.
> Because they no more were “failed”.
>
> This patch changes it into “start emergency.target” (or .service, do
> not remember), so all state is preserved. You see immediately what
> failed and to some extent why.

Ah! I understand now, that is very interesting. Thank you for the
expanded explanation :slight_smile:

>> What happens if the emergency mode fails badly and I have to use a
>> separate rescue system, is the information still available :-?
> By default on openSUSE journal is volatile, so it is lost. You can make
> it non-volatile, but I’m not sure how often it is flushed to disk. Of
> course, if system is failed so badly that even emergency mode does not
> start, there is no guarantee journal could be started or it could write
> anything to disk.

Ah. I see, it is that journal.

There is a package, not installed by default, that does the writing to
file. There have been reports of people with that file growing to 200MB
or so, with the side effect of making boot very slow. The maximum size
is adjustable. I made notes of that somewhere :slight_smile:

Someday I will have to collect all those dispersed notes in a single
file, but as I’m still using systemv, I feel lazy about it :slight_smile:


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

> My neurones trying to digest some difficult information and grumbling a
> bit :slight_smile:

It’sreassuring, in a way, to know that a Flux Capacitor Penguin can be so afflicted. I, as a Newcomer, feel a bit less inadequate.

My printout was produced by journalctl -b.

I take it that my current bottom line is that I have to live with the diversion and the consequent slow boot. I hope that I’ll be made aware when the patch is released, in the hope that somebody will be able to help me back to normal.

Peter Davis

Maybe I misunderstood what is volatile or not?

AFAIK
The journal is not volatile, and is historically searchable. But, if you want to inspect only what occurred during your current session, then apply the “-b” flag.

Unit status is volatile, and currently I do not know of any way to preserve that information except by writing to file which is easy enough to do.

“isolate” is merely the systemd syntax for specifying the init level, typically changing from default.

So far, I have not yet found any limitation to running systemctl in emergency mode although listing failed/stopped services sounds interesting. I would expect that if I’m successfully booted into emergency mode (not rebooted) because of a failure, “status” and maybe grepping for “Failed” should return the relevant information if state has changed.

AFAIK and still learning,
TSU

Hi peterichardavis,

Try following:
Log in as root and then use an editor to modify the fstab file.In /etc/fstab put:
nofail
at the end of line /home just after defaults (two empty spaces) and then save the file and reboot.

See what happens. [size=4]

Another workaround might be to set a new user account during installation. This m[size=4]a[size=4]y need a r[size=4]eboot after installation to make N[size=4]M working.[/size][/size][/size][/size][/size]

I don’t really understand how going back before the current session can explain its emergency mode, but, if you can tell me exactly the command to enter, I’ll be happy to post the printout.

Peter Davis

Thanks; I’m happy to try, but would just like to make sure I’m understanding you correctly. Defaults isn’t actually at the end of my /home line. Should I put two spaces between defaults and nofail instead of a comma?
Are you suggesting that I create a user account specifically for use when installing, or are you referring to installation of something in particular, related, I suppose, to booting? (I hope you don’t mean that I should reinstall the operating system with a new user account.)

On 2013-04-03 23:26, peterichardavis wrote:
>
> AlmostAWhisper;2544194 Wrote:
>>
>> Try following:
>> Log in as root and then use an editor to modify the fstab file.In
>> /etc/fstab put:
>> nofail
>> at the end of line /home just after defaults (two empty spaces) and
>> then save the file and reboot.
>>
>> See what happens.
>>
>> Another workaround might be to set a new user account during
>> installation. This may need a reboot after
>> installation to make NM
>> working.
>
> Thanks; I’m happy to try, but would just like to make sure I’m
> understanding you correctly. Defaults isn’t actually at the end of my
> /home line. Should I put two spaces between defaults and nofail instead
> of a comma?

Sorry, I see no justification to put the nofail option on your home mount.

> Are you suggesting that I create a user account specifically for use
> when installing, or are you referring to installation of something in
> particular, related, I suppose, to booting? (I hope you don’t mean that
> I should reinstall the operating system with a new user account.)

He talks of a new user during installation, but I don’t see a
justification of that either.

Maybe there is a reason, but I don’t see it. :-?


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

I don’t notice anything in the log you posted that provides any clue something is amiss.

Recommend during boot hit “Esc” to view background messages during bootup, not hidden by the splash screen. Try to notice last few lines before being dumped into Emergency Mode. And, assuming that your system does regularly get to the graphical screen where you login with Username and password (aka Plymouth).

My early speculation might be a mis-configured or wrong graphical driver.
BTW- what GPU do you have installed and which driver? And Desktop?

TSU

To make sure that you won’t loss anything, back up the original fstab file first and then modify it. Normally defaults will be automatically put behind /home. Maybe you selected other parameters during installation. Nevertheless put
defaults nofail
behind /home.
The line should look like this somehow:
/dev/disk/by-id/ata-XXX_XXX_XXX-partN /home ext4 defaults nofail 1 2
Try it again and see what happens.

I’ll try your recommendation, thank you.
I have a GeForce 6200 and a nouveau driver. If, by Desktop, you mean something other than the KDE that I specified at the beginning, please let me know exactly what you’d like to know.
Peter

I actually have ext3 on that partition, but have inserted nofail, and hope robin_listas’ doubts are misplaced.
Peter Davis

Also,
And this is a reason why I posted earlier on what kind of information is volatile or not…

Since it sounds you were able to boot <directly> into Emergency mode and not encountering an error during boot which then might have caused a <reboot>,

The following should work

systemctl | grep failed

Then, for each of the failed units, do a “systemctl status” eg

systemctl status* failedunit*

For each of these results, you should see information about the Unit plus relevant entries from the journal exactly what and where the failure happened.

Again, because “failed” and “status” are volatile, they will work only if you run them from within the same session as when the failures occurred. If your system failed, then rebooted these steps won’t work.

As for running the nouveau driver, it seems to be the most flexible and problem-free driver for now.
Recommend disabling KMS (See 12.3 release notes), “on” is the default. Also, perhaps try the “nomodeset” setting.

TSU

Did you read what I wrote about emergency mode? This will not work currently because status of all failed units is reset when emergency.target is entered.