Systemd targets related questions

English Install/Boot/Login
#1

At boot time, when systemd starts the services, it reports reached targets like this in the journal:

journalctl -b --facility=daemon | fgrep 'systemd[1]: Reached'
May 01 10:17:05 xs systemd[1]: Reached target Initrd /usr File System.
May 01 10:17:05 xs systemd[1]: Reached target Slice Units.
May 01 10:17:05 xs systemd[1]: Reached target Swaps.

I created an ordered list of all active targets on my system by manually adding line numbers:

$ journalctl -b --facility=daemon | fgrep 'systemd[1]: Reached' | cut -c46- | tr -d '.'
01 Initrd /usr File System
02 Slice Units
03 Swaps
04 Timer Units
05 Socket Units
06 Path Units
07 Initrd Root Device
08 Preparation for Local File Systems
09 Local File Systems
10 System Initialization
11 Basic System
12 Preparation for Remote File Systems
13 Remote File Systems
14 Initrd Root File System
15 Initrd File Systems
16 Initrd Default Target
17 Switch Root
18 Preparation for Local File Systems = 08
19 Local File Systems = 09
20 Swaps = 03
21 Sound Card
22 System Initialization = 10
23 Path Units = 06
24 Socket Units = 05
25 Basic System = 11
26 Preparation for Network
27 Network
28 Network is Online
29 Login Prompts
30 Multi-User System
31 System Time Synchronized
32 Timer Units = 04

Question 1: Why do a couple of targets get reported twice?

For your convenience, I’ve added the line number of the first occurence at the end, e.g. “18 Preparation for Local File Systems = 08” means that this target was first reached at position 8 and then again at position 18 of all reported reached goals.

Next, I produced a list of all available targets (the output is shortened at * for better readability):

$ systemctl list-units --type target --all
  UNIT                        LOAD      ACTIVE   SUB    DESCRIPTION
  basic.target                loaded    active   active Basic System
  blockdev@dev-disk*.target   loaded    inactive dead   Block Device Preparation
  blockdev@dev-disk*.target   loaded    inactive dead   Block Device Preparation
  blockdev@dev-sda2.target    loaded    inactive dead   Block Device Preparation for /dev/sda2
  blockdev@dev-sda3.target    loaded    inactive dead   Block Device Preparation for /dev/sda3
  cryptsetup-pre.target       loaded    inactive dead   Local Encrypted Volumes (Pre)
  cryptsetup.target           loaded    active   active Local Encrypted Volumes
  emergency.target            loaded    inactive dead   Emergency Mode
  first-boot-complete.target  loaded    inactive dead   First Boot Complete
  getty-pre.target            loaded    inactive dead   Preparation for Logins
  getty.target                loaded    active   active Login Prompts
  graphical.target            loaded    inactive dead   Graphical Interface
  initrd-fs.target            loaded    inactive dead   Initrd File Systems
  initrd-root-device.target   loaded    inactive dead   Initrd Root Device
  initrd-root-fs.target       loaded    inactive dead   Initrd Root File System
  initrd-switch-root.target   loaded    inactive dead   Switch Root
  initrd-usr-fs.target        loaded    inactive dead   Initrd /usr File System
  initrd.target               loaded    inactive dead   Initrd Default Target
  local-fs-pre.target         loaded    active   active Preparation for Local File Systems
  local-fs.target             loaded    active   active Local File Systems
  multi-user.target           loaded    active   active Multi-User System
  network-online.target       loaded    active   active Network is Online
  network-pre.target          loaded    active   active Preparation for Network
  network.target              loaded    active   active Network
  nss-lookup.target           loaded    inactive dead   Host and Network Name Lookups
  nss-user-lookup.target      loaded    inactive dead   User and Group Name Lookups
● openvpn.target              not-found inactive dead   openvpn.target
  paths.target                loaded    active   active Path Units
  remote-cryptsetup.target    loaded    inactive dead   Remote Encrypted Volumes
  remote-fs-pre.target        loaded    inactive dead   Preparation for Remote File Systems
  remote-fs.target            loaded    active   active Remote File Systems
  remote-veritysetup.target   loaded    inactive dead   Remote Verity Protected Volumes
  rescue.target               loaded    inactive dead   Rescue Mode
  shutdown.target             loaded    inactive dead   System Shutdown
  slices.target               loaded    active   active Slice Units
  sockets.target              loaded    active   active Socket Units
  sound.target                loaded    active   active Sound Card
  swap.target                 loaded    active   active Swaps
  sysinit.target              loaded    active   active System Initialization
  time-set.target             loaded    active   active System Time Set
  time-sync.target            loaded    active   active System Time Synchronized
  timers.target               loaded    active   active Timer Units
  umount.target               loaded    inactive dead   Unmount All Filesystems
  veritysetup-pre.target      loaded    inactive dead   Local Verity Protected Volumes (Pre)
  veritysetup.target          loaded    active   active Local Verity Protected Volumes
● wg-quick.target             not-found inactive dead   wg-quick.target

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
46 loaded units listed.
To show all installed unit files use 'systemctl list-unit-files'.

As you may see, the description in the last column is the same like the one which systemd reports as reached target in the journal. So I looked up each of the reported reached targets and entered its line number at the beginning of the line. For all remaining targets, I entered non-numeric line numbers to be able to sort the table. For better readability, I splitted the table into two parts:

a) targets reported as reached in the journal (no surprise here):

00 UNIT                        LOAD      ACTIVE   SUB    DESCRIPTION
01 initrd-usr-fs.target        loaded    inactive dead   Initrd /usr File System
02 slices.target               loaded    active   active Slice Units
03 swap.target                 loaded    active   active Swaps
04 timers.target               loaded    active   active Timer Units
05 sockets.target              loaded    active   active Socket Units
06 paths.target                loaded    active   active Path Units
07 initrd-root-device.target   loaded    inactive dead   Initrd Root Device
08 local-fs-pre.target         loaded    active   active Preparation for Local File Systems
09 local-fs.target             loaded    active   active Local File Systems
10 sysinit.target              loaded    active   active System Initialization
11 basic.target                loaded    active   active Basic System
12 remote-fs-pre.target        loaded    inactive dead   Preparation for Remote File Systems
13 remote-fs.target            loaded    active   active Remote File Systems
14 initrd-root-fs.target       loaded    inactive dead   Initrd Root File System
15 initrd-fs.target            loaded    inactive dead   Initrd File Systems
16 initrd.target               loaded    inactive dead   Initrd Default Target
17 initrd-switch-root.target   loaded    inactive dead   Switch Root
21 sound.target                loaded    active   active Sound Card
26 network-pre.target          loaded    active   active Preparation for Network
27 network.target              loaded    active   active Network
28 network-online.target       loaded    active   active Network is Online
29 getty.target                loaded    active   active Login Prompts
30 multi-user.target           loaded    active   active Multi-User System
31 time-sync.target            loaded    active   active System Time Synchronized

b) targets which have not been reported during startup:

00 UNIT                        LOAD      ACTIVE   SUB    DESCRIPTION
0a blockdev@dev-disk*.target   loaded    inactive dead   Block Device Preparation
0b blockdev@dev-disk*.target   loaded    inactive dead   Block Device Preparation
0c blockdev@dev-sda2.target    loaded    inactive dead   Block Device Preparation for /dev/sda2
0d blockdev@dev-sda3.target    loaded    inactive dead   Block Device Preparation for /dev/sda3
0e cryptsetup-pre.target       loaded    inactive dead   Local Encrypted Volumes (Pre)
0f cryptsetup.target           loaded    active   active Local Encrypted Volumes
1a emergency.target            loaded    inactive dead   Emergency Mode
1b rescue.target               loaded    inactive dead   Rescue Mode
2a getty-pre.target            loaded    inactive dead   Preparation for Logins
3b time-set.target             loaded    active   active System Time Set
3c remote-cryptsetup.target    loaded    inactive dead   Remote Encrypted Volumes
3d nss-lookup.target           loaded    inactive dead   Host and Network Name Lookups
3e nss-user-lookup.target      loaded    inactive dead   User and Group Name Lookups
3f veritysetup-pre.target      loaded    inactive dead   Local Verity Protected Volumes (Pre)
3g veritysetup.target          loaded    active   active Local Verity Protected Volumes
3h remote-veritysetup.target   loaded    inactive dead   Remote Verity Protected Volumes
3i wg-quick.target             not-found inactive dead   wg-quick.target
3j openvpn.target              not-found inactive dead   openvpn.target
3k graphical.target            loaded    inactive dead   Graphical Interface
3l first-boot-complete.target  loaded    inactive dead   First Boot Complete
9a umount.target               loaded    inactive dead   Unmount All Filesystems
9b shutdown.target             loaded    inactive dead   System Shutdown

Obviously, the graphical.target is in this list because of the default runlevel (3) of my system.

There are three targets in this list which are listed as loaded and active but cannot be found in the journal:

  • cryptsetup.target
  • time-set.target
  • veritysetup.target

Question 2: Why do these get reported as loaded and active? I neither have set up encryption nor did I install any Verity software.

Question 3: Why do targets get delivered for software that is not even installed like Verity, OpenVPN, and WireGuard? (And why only for those products and not for other common software?)

As a last remark, the only target that is being mentioned as reached when shutting down the machine for systemd[1] is the shutdown.target; the umount.target is not being reported.

#2

Because initrd is also using systemd as its /sbin/init.

Because these targets are activated by default. You should really start with reading systemd documentation, at least man 7 bootup and man 7 systemd.special.

openvpn.target and wg-quick.target are not installed, they are mentioned by some other unit as dependency. Verity is kernel module (dm-verity) so it is most certainly present, systemd just provides framework to set it up and integrate into boot process like it does for cryptsetup.

#3

Targets reached:

erlangen:~ # journalctl -b -u init.scope -g Reached -o short-monotonic 
[    3.064443] erlangen systemd[1]: Reached target System Initialization.
[    3.065169] erlangen systemd[1]: Reached target Path Units.
[    3.065182] erlangen systemd[1]: Reached target Basic System.
[    3.136334] erlangen systemd[1]: Reached target Initrd Root Device.
[    5.901592] erlangen systemd[1]: Reached target Preparation for Remote File Systems.
[    5.901607] erlangen systemd[1]: Reached target Remote File Systems.
[    6.009386] erlangen systemd[1]: Reached target Initrd Root File System.
[    6.040068] erlangen systemd[1]: Reached target Initrd File Systems.
[    6.040083] erlangen systemd[1]: Reached target Initrd Default Target.
[    6.082892] erlangen systemd[1]: Reached target Switch Root.
[    7.414631] erlangen systemd[1]: Reached target Preparation for Local File Systems.
[    7.801004] erlangen systemd[1]: Reached target Local File Systems.
[    7.801048] erlangen systemd[1]: Reached target Sound Card.
[    8.031752] erlangen systemd[1]: Reached target System Initialization.
[    8.032893] erlangen systemd[1]: Reached target Path Units.
[    8.034608] erlangen systemd[1]: Reached target Socket Units.
[    8.034642] erlangen systemd[1]: Reached target Basic System.
[    8.311330] erlangen systemd[1]: Reached target Network.
[    8.311380] erlangen systemd[1]: Reached target Network is Online.
[    8.470244] erlangen systemd[1]: Reached target System Time Synchronized.
[    8.471627] erlangen systemd[1]: Reached target Timer Units.
[    8.602283] erlangen systemd[1]: Reached target Login Prompts.
[   15.278276] erlangen systemd[1]: Reached target Multi-User System.
[   15.278359] erlangen systemd[1]: Reached target Graphical Interface.
[ 2734.131883] erlangen systemd[1]: Reached target Sleep.
[ 2736.278321] erlangen systemd[1]: Reached target Suspend.
erlangen:~ #

Active targets:

erlangen:~ # systemctl list-units --type target 
  UNIT                  LOAD   ACTIVE SUB    DESCRIPTION                       
  basic.target          loaded active active Basic System
  cryptsetup.target     loaded active active Local Encrypted Volumes
  getty.target          loaded active active Login Prompts
  graphical.target      loaded active active Graphical Interface
  integritysetup.target loaded active active Local Integrity Protected Volumes
  local-fs-pre.target   loaded active active Preparation for Local File Systems
  local-fs.target       loaded active active Local File Systems
  multi-user.target     loaded active active Multi-User System
  network-online.target loaded active active Network is Online
  network.target        loaded active active Network
  paths.target          loaded active active Path Units
  remote-fs.target      loaded active active Remote File Systems
  rpcbind.target        loaded active active RPC Port Mapper
  slices.target         loaded active active Slice Units
  sockets.target        loaded active active Socket Units
  sound.target          loaded active active Sound Card
  swap.target           loaded active active Swaps
  sysinit.target        loaded active active System Initialization
  time-set.target       loaded active active System Time Set
  time-sync.target      loaded active active System Time Synchronized
  timers.target         loaded active active Timer Units
  veritysetup.target    loaded active active Local Verity Protected Volumes

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
22 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
erlangen:~ #
#4

I C. So basically systemd is being executed twice, first when init has mounted the initrd image in RAM as root, and second when init mounts the “real” root file system from disk? And both systemd instances are being executed with the same process ID.

Thank you, Captain Obvious. :smile:

Actually I found and read (ok, not everything, but the relevant sections) in the systemd.special(7) man page when I did a Web search for the emergency.target. However, the descriptions for the cryptsetup and veritysetup targets are not that meaningful: “A target that pulls in setup services for all [encrypted|verity integrity protected] block devices.”

Does that mean these targets have to be active because one cannot tell in advance if there are encrypted or protected block devices being set up?

The time-set.target is described in more detail. I interpreted the first sentence such that this target is only needed for local time sources: “Services responsible for setting the system clock (CLOCK_REALTIME) from a local source (such as a maintained timestamp file or imprecise real-time clock) should pull in this target and order themselves before it.”

Reading the description more attentively I now understand that dependencies are automatically being added because of the OnCalendar= directive in the timer units.

Only nftables.service has [Wants|After]=[openvpn|wg-quick].target and if I hadn’t installed and enabled this service, these targets would not even be listed. Understood. Thanks for the clarification.

#6

Why raven is black? Why elephant is grey? Because they just happened to develop this way. If you are really interested why and whether any alternatives were considered, you need to dig in systemd list archives/git history.

They do not have to. But it is easier to have some well-known point during boot where you can claim “now I have my encrypted device” than to maintain dependencies for each individual filesystem.

#7

Example: Running Services After the Network Is Up

#8

It seems that the init.scope does include only the second instance of systemd after switching root:

$ journalctl -b -u init.scope -g Reached | wc -l
27

$ journalctl -b --facility=daemon | fgrep 'systemd[1]: Reached' | wc -l
32

$ journalctl -b -u init.scope -g Reached > /tmp/r_by_scope
$ journalctl -b --facility=daemon | fgrep 'systemd[1]: Reached' > /tmp/r_by_pid
$ diff /tmp/r_by_scope /tmp/r_by_pid
0a1,5
> May 02 08:48:19 xs systemd[1]: Reached target Initrd /usr File System.
> May 02 08:48:19 xs systemd[1]: Reached target Slice Units.
> May 02 08:48:19 xs systemd[1]: Reached target Swaps.
> May 02 08:48:19 xs systemd[1]: Reached target Timer Units.
> May 02 08:48:19 xs systemd[1]: Reached target Socket Units.

So, when you’re filtering with -u init.scope, you won’t see the double targets of the initrd systemd.

I wonder why the reached targets of the systemd --user daemon are not being reported for -u init.scope–actually this instance is also reported as being part of the init.scope by the cgroups lister tool:

$ systemd-cgls -u user.slice
Unit user.slice (/user.slice):
└─user-1000.slice
  ├─user@1000.service
  │ └─init.scope
  │   ├─ 1884 /usr/lib/systemd/systemd --user
  │   └─ 1885 (sd-pam)
  └─session-2.scope
    ├─ 1881 ...
    ├─ 1892 ...
	...

$  journalctl -b --facility=daemon | fgrep 'systemd[1884]: Reached' | cut -c50- | tr -d '.'
Paths
Timers
Sockets
Basic System
Main User Target

This is also the reason why I didn’t filter using journalctl -b /usr/lib/systemd/systemd in the first place.

#9

That makes sense. Thanks.

#10

User targets:

karl@erlangen:~> journalctl --user -b -u init.scope -g Reached -o short-monotonic 
[ 1232.853593] erlangen systemd[2824]: Reached target Paths.
[ 1232.853729] erlangen systemd[2824]: Reached target Timers.
[ 1232.854028] erlangen systemd[2824]: Reached target Sockets.
[ 1232.854085] erlangen systemd[2824]: Reached target Basic System.
[ 1232.854220] erlangen systemd[2824]: Reached target Main User Target.
[ 1233.212377] erlangen systemd[2824]: Reached target Session services which should run early before the graphical session is brought up.
[ 1233.658329] erlangen systemd[2824]: Reached target KDE Plasma Workspace Core.
[ 1233.871821] erlangen systemd[2824]: Reached target KDE Plasma Workspace.
[ 1233.871900] erlangen systemd[2824]: Reached target Current graphical user session.
[ 1233.871993] erlangen systemd[2824]: Reached target plasma-workspace-x11.target.
[ 1233.885923] erlangen systemd[2824]: Reached target Startup of XDG autostart applications.
karl@erlangen:~> 

karl@erlangen:~> systemctl list-units  --user --type target
  UNIT                         LOAD   ACTIVE SUB    DESCRIPTION                                                                       
  basic.target                 loaded active active Basic System
  default.target               loaded active active Main User Target
  graphical-session-pre.target loaded active active Session services which should run early before the graphical session is brought up
  graphical-session.target     loaded active active Current graphical user session
  paths.target                 loaded active active Paths
  plasma-core.target           loaded active active KDE Plasma Workspace Core
  plasma-workspace-x11.target  loaded active active plasma-workspace-x11.target
  plasma-workspace.target      loaded active active KDE Plasma Workspace
  sockets.target               loaded active active Sockets
  timers.target                loaded active active Timers
  xdg-desktop-autostart.target loaded active active Startup of XDG autostart applications

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
11 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
karl@erlangen:~>