LXD dose not start

seven_skies · November 7, 2019, 5:49pm

Hello there, i just installed lxd on my opensuse 15.1 using zypper;

sudo zypper in lxd;
The following 4 NEW packages are going to be installed:
  lxcfs lxcfs-hooks-lxc lxd lxd-bash-completion
sudo systemctl status lxd;
 t=2019-11-07T19:41:33+0300 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
lxd.service: Main process exited, code=killed, status=11/SEGV
sudo systemctl stop lxd
sudo systemctl start lxd, hangis for 3 minutes, then same result.

adding logs here;

tail -50 /var/log/lxd/lxd.log 
t=2019-11-07T19:39:50+0300 lvl=info msg=" - uevent injection: no" 
t=2019-11-07T19:39:50+0300 lvl=info msg=" - seccomp listener: no" 
t=2019-11-07T19:39:50+0300 lvl=info msg=" - unprivileged file capabilities: no" 
t=2019-11-07T19:39:50+0300 lvl=info msg=" - shiftfs support: no" 
t=2019-11-07T19:39:50+0300 lvl=info msg="Initializing local database" 
t=2019-11-07T19:39:50+0300 lvl=info msg="Starting /dev/lxd handler:" 
t=2019-11-07T19:39:50+0300 lvl=info msg=" - binding devlxd socket" socket=/var/lib/lxd/devlxd/sock
t=2019-11-07T19:39:50+0300 lvl=info msg="REST API daemon:" 
t=2019-11-07T19:39:50+0300 lvl=info msg=" - binding Unix socket" socket=/var/lib/lxd/unix.socket
t=2019-11-07T19:39:50+0300 lvl=info msg="Initializing global database" 
t=2019-11-07T19:40:33+0300 lvl=info msg="LXD 3.18 is starting in normal mode" path=/var/lib/lxd
t=2019-11-07T19:40:33+0300 lvl=info msg="Kernel uid/gid map:" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - u 0 0 4294967295" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - g 0 0 4294967295" 
t=2019-11-07T19:40:33+0300 lvl=info msg="Configured LXD uid/gid map:" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - u 0 100000 65536" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - g 0 100000 65536" 
t=2019-11-07T19:40:33+0300 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored." 
t=2019-11-07T19:40:33+0300 lvl=info msg="Kernel features:" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - netnsid-based network retrieval: no" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - uevent injection: no" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - seccomp listener: no" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - unprivileged file capabilities: no" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - shiftfs support: no" 
t=2019-11-07T19:40:33+0300 lvl=info msg="Initializing local database" 
t=2019-11-07T19:40:33+0300 lvl=info msg="Starting /dev/lxd handler:" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - binding devlxd socket" socket=/var/lib/lxd/devlxd/sock
t=2019-11-07T19:40:33+0300 lvl=info msg="REST API daemon:" 
t=2019-11-07T19:40:33+0300 lvl=info msg=" - binding Unix socket" socket=/var/lib/lxd/unix.socket
t=2019-11-07T19:40:33+0300 lvl=info msg="Initializing global database" 
t=2019-11-07T19:41:33+0300 lvl=info msg="LXD 3.18 is starting in normal mode" path=/var/lib/lxd
t=2019-11-07T19:41:33+0300 lvl=info msg="Kernel uid/gid map:" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - u 0 0 4294967295" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - g 0 0 4294967295" 
t=2019-11-07T19:41:33+0300 lvl=info msg="Configured LXD uid/gid map:" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - u 0 100000 65536" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - g 0 100000 65536" 
t=2019-11-07T19:41:33+0300 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored." 
t=2019-11-07T19:41:33+0300 lvl=info msg="Kernel features:" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - netnsid-based network retrieval: no" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - uevent injection: no" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - seccomp listener: no" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - unprivileged file capabilities: no" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - shiftfs support: no" 
t=2019-11-07T19:41:33+0300 lvl=info msg="Initializing local database" 
t=2019-11-07T19:41:33+0300 lvl=info msg="Starting /dev/lxd handler:" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - binding devlxd socket" socket=/var/lib/lxd/devlxd/sock
t=2019-11-07T19:41:33+0300 lvl=info msg="REST API daemon:" 
t=2019-11-07T19:41:33+0300 lvl=info msg=" - binding Unix socket" socket=/var/lib/lxd/unix.socket
t=2019-11-07T19:41:33+0300 lvl=info msg="Initializing global database"

am i missing something? how do i get it to start please?

malcolmlewis · November 7, 2019, 5:58pm

Hi
I see;


zypper in lxd

The following 11 NEW packages are going to be installed:
  criu liblxc1 libnet9 libuv1 lxcfs lxcfs-hooks-lxc lxd lxd-bash-completion python2-ipaddr python2-protobuf squashfs

Additional rpm output:
==>>> setting /usr/lib/lxc/lxc-user-nic to root:kvm 4750. (wrong permissions 0750)

systemctl status lxd

● lxd.service - LXD Container Hypervisor
   Loaded: loaded (/usr/lib/systemd/system/lxd.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:lxd(1)

systemctl start lxd
systemctl status lxd

● lxd.service - LXD Container Hypervisor
   Loaded: loaded (/usr/lib/systemd/system/lxd.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-11-07 10:53:38 CST; 1s ago
     Docs: man:lxd(1)
  Process: 5640 ExecStartPost=/usr/bin/lxd waitready --timeout=600 (code=exited, status=0/SUCCESS)
 Main PID: 5639 (lxd)
    Tasks: 23
   Memory: 43.2M
      CPU: 446ms
   CGroup: /system.slice/lxd.service
           └─5639 /usr/bin/lxd --group=lxd --logfile=/var/log/lxd/lxd.log

Nov 07 10:53:36 grover-os151.homelinux.org systemd[1]: Starting LXD Container Hypervisor...
Nov 07 10:53:36 grover-os151.homelinux.org lxd[5639]: t=2019-11-07T10:53:36-0600 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
Nov 07 10:53:38 grover-os151.homelinux.org systemd[1]: Started LXD Container Hypervisor.

I image your system and CPU supports virtualization and is enabled in the BIOS?

seven_skies · November 7, 2019, 6:26pm

malcolmlewis:

Hi
I see;


zypper in lxd

The following 11 NEW packages are going to be installed:
  criu liblxc1 libnet9 libuv1 lxcfs lxcfs-hooks-lxc lxd lxd-bash-completion python2-ipaddr python2-protobuf squashfs

Additional rpm output:
==>>> setting /usr/lib/lxc/lxc-user-nic to root:kvm 4750. (wrong permissions 0750)

systemctl status lxd

● lxd.service - LXD Container Hypervisor
   Loaded: loaded (/usr/lib/systemd/system/lxd.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:lxd(1)

systemctl start lxd
systemctl status lxd

● lxd.service - LXD Container Hypervisor
   Loaded: loaded (/usr/lib/systemd/system/lxd.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-11-07 10:53:38 CST; 1s ago
     Docs: man:lxd(1)
  Process: 5640 ExecStartPost=/usr/bin/lxd waitready --timeout=600 (code=exited, status=0/SUCCESS)
 Main PID: 5639 (lxd)
    Tasks: 23
   Memory: 43.2M
      CPU: 446ms
   CGroup: /system.slice/lxd.service
           └─5639 /usr/bin/lxd --group=lxd --logfile=/var/log/lxd/lxd.log

Nov 07 10:53:36 grover-os151.homelinux.org systemd[1]: Starting LXD Container Hypervisor...
Nov 07 10:53:36 grover-os151.homelinux.org lxd[5639]: t=2019-11-07T10:53:36-0600 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
Nov 07 10:53:38 grover-os151.homelinux.org systemd[1]: Started LXD Container Hypervisor.

I image your system and CPU supports virtualization and is enabled in the BIOS?

Yes sir, virtualization is enabled.

malcolmlewis · November 7, 2019, 6:56pm

Hi
Disk space ok? Try starting again and wait, does it finally start?

My test Leap 15.1 is actually a qemu instance, so virtualized running a virtualization container…

I also see more info in the log output, does yours stop at global, or other lines setup additional items?

tsu2 · November 7, 2019, 8:18pm

Your post caught my eye just as I’m running… So what I’m posting is going to be brief…

I don’t know why a hypervisor is mentioned, it’s not required for Linux containers of course, and I can’t think of any reason why it’d be required for a container **manager **(no actual use for self-deployment). Documentation also says hardware CPU extensions not required.
Recommend you collect some information.
Open an elevated console running the following command and leave it running

journalctl -f

Now, with the above console running which will display system log events, start your lxd daemon and leave it running as long as you need.

I don’t use lxd, so can’t know for sure but suspect that it might have just timed out and stopped.
If you don’t see any significant system events, just use lxd to do what you need… It’s actually lxc which is needed for your actual containers.

TSU

seven_skies · November 7, 2019, 10:16pm

tsu2:

Your post caught my eye just as I’m running… So what I’m posting is going to be brief…

I don’t know why a hypervisor is mentioned, it’s not required for Linux containers of course, and I can’t think of any reason why it’d be required for a container **manager **(no actual use for self-deployment). Documentation also says hardware CPU extensions not required.

Recommend you collect some information.
Open an elevated console running the following command and leave it running
journalctl -f
Now, with the above console running which will display system log events, start your lxd daemon and leave it running as long as you need.

I don’t use lxd, so can’t know for sure but suspect that it might have just timed out and stopped.
If you don’t see any significant system events, just use lxd to do what you need… It’s actually lxc which is needed for your actual containers.

TSU

Hi,
with logs on;

systemctl start lxd,; hangs....
Starting LXD Container Hypervisor...
t=2019-11-08T00:12:52+0300 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."

lxd.service: Main process exited, code=killed, status=11/SEGV

till i ctl + c;
systemctl stop lxc;

Stopped LXD Container Hypervisor.

lxd.service: Unit entered failed state.
lxd.service: Failed with result 'signal'.

seven_skies · November 7, 2019, 10:17pm

Disk space is ok, have more than 100 GB empty, i do use kvm too, infact some vms are running without issue.
edit: disk format is ext4.

seven_skies · November 7, 2019, 10:45pm

additional log that i missed out:

Error: LXD still not running after 600s timeout (Get http://unix.socket/1.0: dial unix /var/lib/lxd/unix.socket: connect: connection refused)

malcolmlewis · November 7, 2019, 10:57pm

Hi
Have you run lxd init command?

seven_skies · November 7, 2019, 11:08pm

sudo lxd init
Error: Failed to connect to local LXD: Get http://unix.socket/1.0: dial unix /var/lib/lxd/unix.socket: connect: connection refused

malcolmlewis · November 7, 2019, 11:13pm

Hi
Can you switch to root user instead of using sudo and try and start the lxd service.

seven_skies · November 7, 2019, 11:27pm

i did remove lxd, open terminal as

root, su -;
zypper rm lxd criu liblxc1 libnet9 libuv1 lxcfs lxcfs-hooks-lxc lxd lxd-bash-completion python2-ipaddr python2-protobuf squashfs
zypper in lxd;
The following 11 NEW packages are going to be installed:  criu liblxc1 libnet9 libuv1 lxcfs lxcfs-hooks-lxc lxd lxd-bash-completion python2-ipaddr python2-protobuf squashfs

 systemctl start lxd, hangs;

here are full logs;

Nov 08 01:21:40 seven [RPM][4583]: Transaction ID 5dc498f4 started
Nov 08 01:21:40 seven [RPM][4583]: install liblxc1-3.2.1-lp151.4.5.1.x86_64: success
Nov 08 01:21:40 seven [RPM][4583]: install liblxc1-3.2.1-lp151.4.5.1.x86_64: success
Nov 08 01:21:40 seven [RPM][4583]: Transaction ID 5dc498f4 finished: 0
Nov 08 01:21:41 seven [RPM][4587]: Transaction ID 5dc498f5 started
Nov 08 01:21:41 seven systemd[1]: Reloading.
Nov 08 01:21:41 seven systemd[1]: nss-lookup.target: Dependency Before=nss-lookup.target dropped
Nov 08 01:21:41 seven systemd[1]: Reloading.
Nov 08 01:21:42 seven systemd[1]: nss-lookup.target: Dependency Before=nss-lookup.target dropped
Nov 08 01:21:42 seven [RPM][4587]: install lxcfs-3.1.2-lp151.2.3.1.x86_64: success
Nov 08 01:21:42 seven [RPM][4587]: install lxcfs-3.1.2-lp151.2.3.1.x86_64: success
Nov 08 01:21:42 seven [RPM][4587]: Transaction ID 5dc498f5 finished: 0
Nov 08 01:21:42 seven [RPM][4625]: Transaction ID 5dc498f6 started
Nov 08 01:21:42 seven [RPM][4625]: install lxcfs-hooks-lxc-3.1.2-lp151.2.3.1.noarch: success
Nov 08 01:21:42 seven [RPM][4625]: install lxcfs-hooks-lxc-3.1.2-lp151.2.3.1.noarch: success
Nov 08 01:21:42 seven [RPM][4625]: Transaction ID 5dc498f6 finished: 0
Nov 08 01:21:42 seven [RPM][4626]: Transaction ID 5dc498f6 started
Nov 08 01:21:43 seven [RPM][4626]: install libnet9-1.2~rc3-lp151.2.2.x86_64: success
Nov 08 01:21:43 seven [RPM][4626]: install libnet9-1.2~rc3-lp151.2.2.x86_64: success
Nov 08 01:21:43 seven [RPM][4626]: Transaction ID 5dc498f6 finished: 0
Nov 08 01:21:43 seven [RPM][4628]: Transaction ID 5dc498f7 started
Nov 08 01:21:43 seven [RPM][4628]: install libuv1-1.18.0-lp151.2.3.x86_64: success
Nov 08 01:21:43 seven [RPM][4628]: install libuv1-1.18.0-lp151.2.3.x86_64: success
Nov 08 01:21:43 seven [RPM][4628]: Transaction ID 5dc498f7 finished: 0
Nov 08 01:21:44 seven [RPM][4636]: Transaction ID 5dc498f8 started
Nov 08 01:21:44 seven [RPM][4636]: install python2-ipaddr-2.1.11-lp151.2.2.noarch: success
Nov 08 01:21:44 seven [RPM][4636]: install python2-ipaddr-2.1.11-lp151.2.2.noarch: success
Nov 08 01:21:44 seven [RPM][4636]: Transaction ID 5dc498f8 finished: 0
Nov 08 01:21:44 seven [RPM][4637]: Transaction ID 5dc498f8 started
Nov 08 01:21:44 seven [RPM][4637]: install python2-protobuf-3.5.0-lp151.4.7.x86_64: success
Nov 08 01:21:44 seven [RPM][4637]: install python2-protobuf-3.5.0-lp151.4.7.x86_64: success
Nov 08 01:21:44 seven [RPM][4637]: Transaction ID 5dc498f8 finished: 0
Nov 08 01:21:45 seven [RPM][4638]: Transaction ID 5dc498f9 started
Nov 08 01:21:45 seven [RPM][4638]: install squashfs-4.3-lp151.2.3.x86_64: success
Nov 08 01:21:45 seven [RPM][4638]: install squashfs-4.3-lp151.2.3.x86_64: success
Nov 08 01:21:45 seven [RPM][4638]: Transaction ID 5dc498f9 finished: 0
Nov 08 01:21:45 seven [RPM][4639]: Transaction ID 5dc498f9 started
Nov 08 01:21:45 seven [RPM][4639]: install criu-3.8.1-lp151.2.3.x86_64: success
Nov 08 01:21:45 seven [RPM][4639]: install criu-3.8.1-lp151.2.3.x86_64: success
Nov 08 01:21:45 seven [RPM][4639]: Transaction ID 5dc498f9 finished: 0
Nov 08 01:21:46 seven [RPM][4640]: Transaction ID 5dc498fa started
Nov 08 01:21:46 seven nscd[1214]: 1214 monitored file `/etc/passwd` was written to
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring file `/etc/passwd` (1)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring directory `/etc` (2)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring file `/etc/group` (3)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring directory `/etc` (2)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring file `/etc/passwd` (1)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring directory `/etc` (2)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring file `/etc/group` (3)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring directory `/etc` (2)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitored file `/etc/passwd` was written to
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring file `/etc/passwd` (1)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring directory `/etc` (2)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring file `/etc/group` (3)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring directory `/etc` (2)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring file `/etc/passwd` (1)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring directory `/etc` (2)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring file `/etc/group` (3)
Nov 08 01:21:46 seven nscd[1214]: 1214 monitoring directory `/etc` (2)
Nov 08 01:21:49 seven systemd[1]: Reloading.
Nov 08 01:21:49 seven systemd[1]: nss-lookup.target: Dependency Before=nss-lookup.target dropped
Nov 08 01:21:49 seven systemd[1]: Reloading.
Nov 08 01:21:49 seven systemd[1]: nss-lookup.target: Dependency Before=nss-lookup.target dropped
Nov 08 01:21:49 seven [RPM][4640]: install lxd-3.18-lp151.11.1.x86_64: success
Nov 08 01:21:49 seven [RPM][4640]: install lxd-3.18-lp151.11.1.x86_64: success
Nov 08 01:21:49 seven [RPM][4640]: Transaction ID 5dc498fa finished: 0
Nov 08 01:21:50 seven [RPM][4693]: Transaction ID 5dc498fe started
Nov 08 01:21:50 seven [RPM][4693]: install lxd-bash-completion-3.18-lp151.11.1.noarch: success
Nov 08 01:21:50 seven [RPM][4693]: install lxd-bash-completion-3.18-lp151.11.1.noarch: success
Nov 08 01:21:50 seven [RPM][4693]: Transaction ID 5dc498fe finished: 0
Nov 08 01:22:03 seven systemd[1]: Started FUSE filesystem for LXC.
Nov 08 01:22:03 seven systemd[1]: Reached target Network is Online.
Nov 08 01:22:03 seven systemd[1]: Starting LXD Container Hypervisor...
Nov 08 01:22:03 seven kernel: new mount options do not match the existing superblock, will be ignored
Nov 08 01:22:03 seven lxcfs[4709]: mount namespace: 5
Nov 08 01:22:03 seven lxcfs[4709]: hierarchies:
Nov 08 01:22:03 seven lxcfs[4709]:   0: fd:   6: rdma
Nov 08 01:22:03 seven lxcfs[4709]:   1: fd:   7: perf_event
Nov 08 01:22:03 seven lxcfs[4709]:   2: fd:   8: devices
Nov 08 01:22:03 seven lxcfs[4709]:   3: fd:   9: freezer
Nov 08 01:22:03 seven lxcfs[4709]:   4: fd:  10: blkio
Nov 08 01:22:03 seven lxcfs[4709]:   5: fd:  11: memory
Nov 08 01:22:03 seven lxcfs[4709]:   6: fd:  12: net_cls,net_prio
Nov 08 01:22:03 seven lxcfs[4709]:   7: fd:  13: hugetlb
Nov 08 01:22:03 seven lxcfs[4709]:   8: fd:  14: cpuset
Nov 08 01:22:03 seven lxcfs[4709]:   9: fd:  15: cpu,cpuacct
Nov 08 01:22:03 seven lxcfs[4709]:  10: fd:  16: pids
Nov 08 01:22:03 seven lxcfs[4709]:  11: fd:  17: name=systemd
Nov 08 01:22:03 seven lxcfs[4709]:  12: fd:  18: unified
Nov 08 01:22:03 seven lxd[4710]: t=2019-11-08T01:22:03+0300 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
Nov 08 01:22:03 seven systemd[1]: lxd.service: Main process exited, code=killed, status=11/SEGV

EDIT:
i did some googling some people has done the followings so i did, BUT STILL NOT WORKING;

chmod u+s /usr/lib/lxc/lxc-user-nic
[FONT=Helvetica]/etc/pam.d/common-session:session optional pam_cgfs.so -c all ## is supposed to make cgroup handler

[/FONT]

edit so i let it hanging till it self terminated with this output and logs;


systemctl start lxd
Job for lxd.service failed because a fatal signal was delivered to the control process.
See "systemctl  status lxd.service" and "journalctl  -xe" for details.

Nov 08 01:32:03 seven lxd[4711]: Error: LXD still not running after 600s timeout (Get http://unix.socket/1.0: dial unix /var/lib/lxd/unix.socket: connect: connection refused)
Nov 08 01:32:03 seven systemd[1]: lxd.service: Start-post operation timed out. Stopping.
Nov 08 01:32:03 seven systemd[1]: lxd.service: Control process exited, code=exited status=1
Nov 08 01:32:03 seven systemd[1]: Failed to start LXD Container Hypervisor.
Nov 08 01:32:03 seven systemd[1]: lxd.service: Unit entered failed state.
Nov 08 01:32:03 seven systemd[1]: lxd.service: Failed with result 'signal'.
Nov 08 01:32:03 seven systemd[1]: lxd.service: Service RestartSec=100ms expired, scheduling restart.
Nov 08 01:32:03 seven systemd[1]: Stopped LXD Container Hypervisor.
Nov 08 01:32:03 seven systemd[1]: Starting LXD Container Hypervisor...
Nov 08 01:32:03 seven lxd[5373]: t=2019-11-08T01:32:03+0300 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
Nov 08 01:32:03 seven systemd[1]: lxd.service: Main process exited, code=killed, status=11/SEGV
Nov 08 01:33:26 seven systemd[1]: Starting Cleanup of Temporary Directories...

malcolmlewis · November 7, 2019, 11:38pm

Hi
Is apparmor running? Not sure, all a bit strange…

seven_skies · November 7, 2019, 11:47pm

Yes sir, its runing;

systemctl status apparmor.service 
● apparmor.service - Load AppArmor profiles
   Loaded: loaded (/usr/lib/systemd/system/apparmor.service; enabled; vendor preset: enabled)
   Active: active (exited) since Fri 2019-11-08 01:45:28 +03; 18min 3s ago
  Process: 6321 ExecStop=/bin/true (code=exited, status=0/SUCCESS)
  Process: 6380 ExecStart=/lib/apparmor/apparmor.systemd reload (code=exited, status=0/SUCCESS)
 Main PID: 6380 (code=exited, status=0/SUCCESS)
      CPU: 962ms


Nov 08 01:45:27 seven systemd[1]: Starting Load AppArmor profiles...
Nov 08 01:45:27 seven apparmor.systemd[6380]: Restarting AppArmor
Nov 08 01:45:27 seven apparmor.systemd[6380]: Reloading AppArmor profiles
Nov 08 01:45:28 seven systemd[1]: Started Load AppArmor profiles.

malcolmlewis · November 7, 2019, 11:52pm

seven_skies:

Yes sir, its runing;

systemctl status apparmor.service 
● apparmor.service - Load AppArmor profiles
   Loaded: loaded (/usr/lib/systemd/system/apparmor.service; enabled; vendor preset: enabled)
   Active: active (exited) since Fri 2019-11-08 01:45:28 +03; 18min 3s ago
  Process: 6321 ExecStop=/bin/true (code=exited, status=0/SUCCESS)
  Process: 6380 ExecStart=/lib/apparmor/apparmor.systemd reload (code=exited, status=0/SUCCESS)
 Main PID: 6380 (code=exited, status=0/SUCCESS)
      CPU: 962ms


Nov 08 01:45:27 seven systemd[1]: Starting Load AppArmor profiles...
Nov 08 01:45:27 seven apparmor.systemd[6380]: Restarting AppArmor
Nov 08 01:45:27 seven apparmor.systemd[6380]: Reloading AppArmor profiles
Nov 08 01:45:28 seven systemd[1]: Started Load AppArmor profiles.

Hi
If you stop it temporarily and then try starting the service (all as root, no sudo ), does it start?

If not, then restart apparmor, then (again as root user) start manually with some debug;


lxd -d

seven_skies · November 8, 2019, 12:00am

malcolmlewis:

Hi
If you stop it temporarily and then try starting the service (all as root, no sudo ), does it start?

If not, then restart apparmor, then (again as root user) start manually with some debug;
lxd -d

i did restart apparmor, start lxd hangs i ctr+c then start again hangs as always.
here are debugs for lxd;

lxd --debug --group lxd
INFO[11-08|01:55:42] LXD 3.18 is starting in normal mode      path=/var/lib/lxd
INFO[11-08|01:55:42] Kernel uid/gid map: 
INFO[11-08|01:55:42]  - u 0 0 4294967295 
INFO[11-08|01:55:42]  - g 0 0 4294967295 
INFO[11-08|01:55:42] Configured LXD uid/gid map: 
INFO[11-08|01:55:42]  - u 0 400000000 500000001 
INFO[11-08|01:55:42]  - g 0 400000000 500000001 
WARN[11-08|01:55:42] CGroup memory swap accounting is disabled, swap limits will be ignored. 
INFO[11-08|01:55:42] Kernel features: 
INFO[11-08|01:55:42]  - netnsid-based network retrieval: no 
INFO[11-08|01:55:42]  - uevent injection: no 
INFO[11-08|01:55:42]  - seccomp listener: no 
INFO[11-08|01:55:42]  - unprivileged file capabilities: no 
INFO[11-08|01:55:42]  - shiftfs support: no 
INFO[11-08|01:55:42] Initializing local database 
DBUG[11-08|01:55:42] Initializing database gateway 
DBUG[11-08|01:55:42] Start database node                      address= id=1
DBUG[11-08|01:55:42] Detected stale unix socket, deleting 
INFO[11-08|01:55:42] Starting /dev/lxd handler: 
INFO[11-08|01:55:42]  - binding devlxd socket                 socket=/var/lib/lxd/devlxd/sock
INFO[11-08|01:55:42] REST API daemon: 
INFO[11-08|01:55:42]  - binding Unix socket                   socket=/var/lib/lxd/unix.socket
INFO[11-08|01:55:42] Initializing global database 
DBUG[11-08|01:55:42] Dqlite: connected address=1 attempt=0 
Segmentation fault (core dumped)

malcolmlewis · November 8, 2019, 12:19am

Hi
Here is my output for comparison;


lxd -d
INFO[11-07|16:51:06] LXD 3.18 is starting in normal mode      path=/var/lib/lxd
INFO[11-07|16:51:06] Kernel uid/gid map: 
INFO[11-07|16:51:06]  - u 0 0 4294967295 
INFO[11-07|16:51:06]  - g 0 0 4294967295 
INFO[11-07|16:51:06] Configured LXD uid/gid map: 
INFO[11-07|16:51:06]  - u 0 400000000 500000001 
INFO[11-07|16:51:06]  - g 0 400000000 500000001 
WARN[11-07|16:51:06] CGroup memory swap accounting is disabled, swap limits will be ignored. 
INFO[11-07|16:51:06] Kernel features: 
INFO[11-07|16:51:06]  - netnsid-based network retrieval: no 
INFO[11-07|16:51:06]  - uevent injection: no 
INFO[11-07|16:51:06]  - seccomp listener: no 
INFO[11-07|16:51:06]  - unprivileged file capabilities: no 
INFO[11-07|16:51:06]  - shiftfs support: no 
INFO[11-07|16:51:06] Initializing local database 
DBUG[11-07|16:51:06] Initializing database gateway 
DBUG[11-07|16:51:06] Start database node                      id=1 address=
INFO[11-07|16:51:06] Starting /dev/lxd handler: 
INFO[11-07|16:51:06]  - binding devlxd socket                 socket=/var/lib/lxd/devlxd/sock
INFO[11-07|16:51:06] REST API daemon: 
INFO[11-07|16:51:06]  - binding Unix socket                   socket=/var/lib/lxd/unix.socket
INFO[11-07|16:51:06] Initializing global database 
DBUG[11-07|16:51:06] Dqlite: connected address=1 attempt=0 
INFO[11-07|16:51:06] Initializing storage pools 
DBUG[11-07|16:51:06] No existing storage pools detected 
INFO[11-07|16:51:06] Initializing networks 
DBUG[11-07|16:51:06] New task Operation: f2f5075f-5b97-4f3f-9e14-d2f67869b900 
INFO[11-07|16:51:06] Pruning leftover image files 
DBUG[11-07|16:51:06] Started task operation: f2f5075f-5b97-4f3f-9e14-d2f67869b900 
INFO[11-07|16:51:06] Done pruning leftover image files 
INFO[11-07|16:51:06] Loading daemon configuration 
DBUG[11-07|16:51:06] Initialized inotify with file descriptor 23 
DBUG[11-07|16:51:06] Success for task operation: f2f5075f-5b97-4f3f-9e14-d2f67869b900 
DBUG[11-07|16:51:06] New task Operation: 3da1c9f5-a0d3-4da8-8646-d6ed445fbcfb 
INFO[11-07|16:51:06] Pruning expired images 
DBUG[11-07|16:51:06] Started task operation: 3da1c9f5-a0d3-4da8-8646-d6ed445fbcfb 
INFO[11-07|16:51:06] Done pruning expired images 
DBUG[11-07|16:51:06] New task Operation: ff9f8e5e-1b30-4648-a212-aa415be1ac03 
DBUG[11-07|16:51:06] Success for task operation: 3da1c9f5-a0d3-4da8-8646-d6ed445fbcfb 
INFO[11-07|16:51:06] Pruning expired container backups 
DBUG[11-07|16:51:06] Started task operation: ff9f8e5e-1b30-4648-a212-aa415be1ac03 
DBUG[11-07|16:51:06] Success for task operation: ff9f8e5e-1b30-4648-a212-aa415be1ac03 
INFO[11-07|16:51:06] Done pruning expired container backups 
DBUG[11-07|16:51:06] New task Operation: 404cbf64-4823-4e63-98f9-7d843f283e1f 
DBUG[11-07|16:51:06] New task Operation: a890790f-c9f0-4bbe-93d5-cbdcc8b6fe70 
INFO[11-07|16:51:06] Updating instance types 
DBUG[11-07|16:51:06] Started task operation: 404cbf64-4823-4e63-98f9-7d843f283e1f 
INFO[11-07|16:51:06] Expiring log files 
DBUG[11-07|16:51:06] Started task operation: a890790f-c9f0-4bbe-93d5-cbdcc8b6fe70 
INFO[11-07|16:51:06] Done updating instance types 
INFO[11-07|16:51:06] Done expiring log files 
DBUG[11-07|16:51:06] Success for task operation: a890790f-c9f0-4bbe-93d5-cbdcc8b6fe70 


DBUG[11-07|16:51:11] Success for task operation: 404cbf64-4823-4e63-98f9-7d843f283e1f 
^CINFO[11-07|16:51:14] Received 'interrupt signal', exiting 
INFO[11-07|16:51:14] Starting shutdown sequence 
INFO[11-07|16:51:14] Stopping REST API handler: 
INFO[11-07|16:51:14]  - closing socket                        socket=/var/lib/lxd/unix.socket
INFO[11-07|16:51:14] Stopping /dev/lxd handler: 
INFO[11-07|16:51:14]  - closing socket                        socket=/var/lib/lxd/devlxd/sock
INFO[11-07|16:51:14] Closing the database 
DBUG[11-07|16:51:14] Stop database gateway 
INFO[11-07|16:51:14] Unmounting temporary filesystems 
INFO[11-07|16:51:14] Done unmounting temporary filesystems

Do you have some customization’s for tmpfs perhaps? Your core dump occurs at creating the storage pool as that’s what I see after your system core dumps.

seven_skies · November 8, 2019, 12:26am

malcolmlewis:

Hi
Here is my output for comparison;


lxd -d
INFO[11-07|16:51:06] LXD 3.18 is starting in normal mode      path=/var/lib/lxd
INFO[11-07|16:51:06] Kernel uid/gid map: 
INFO[11-07|16:51:06]  - u 0 0 4294967295 
INFO[11-07|16:51:06]  - g 0 0 4294967295 
INFO[11-07|16:51:06] Configured LXD uid/gid map: 
INFO[11-07|16:51:06]  - u 0 400000000 500000001 
INFO[11-07|16:51:06]  - g 0 400000000 500000001 
WARN[11-07|16:51:06] CGroup memory swap accounting is disabled, swap limits will be ignored. 
INFO[11-07|16:51:06] Kernel features: 
INFO[11-07|16:51:06]  - netnsid-based network retrieval: no 
INFO[11-07|16:51:06]  - uevent injection: no 
INFO[11-07|16:51:06]  - seccomp listener: no 
INFO[11-07|16:51:06]  - unprivileged file capabilities: no 
INFO[11-07|16:51:06]  - shiftfs support: no 
INFO[11-07|16:51:06] Initializing local database 
DBUG[11-07|16:51:06] Initializing database gateway 
DBUG[11-07|16:51:06] Start database node                      id=1 address=
INFO[11-07|16:51:06] Starting /dev/lxd handler: 
INFO[11-07|16:51:06]  - binding devlxd socket                 socket=/var/lib/lxd/devlxd/sock
INFO[11-07|16:51:06] REST API daemon: 
INFO[11-07|16:51:06]  - binding Unix socket                   socket=/var/lib/lxd/unix.socket
INFO[11-07|16:51:06] Initializing global database 
DBUG[11-07|16:51:06] Dqlite: connected address=1 attempt=0 
INFO[11-07|16:51:06] Initializing storage pools 
DBUG[11-07|16:51:06] No existing storage pools detected 
INFO[11-07|16:51:06] Initializing networks 
DBUG[11-07|16:51:06] New task Operation: f2f5075f-5b97-4f3f-9e14-d2f67869b900 
INFO[11-07|16:51:06] Pruning leftover image files 
DBUG[11-07|16:51:06] Started task operation: f2f5075f-5b97-4f3f-9e14-d2f67869b900 
INFO[11-07|16:51:06] Done pruning leftover image files 
INFO[11-07|16:51:06] Loading daemon configuration 
DBUG[11-07|16:51:06] Initialized inotify with file descriptor 23 
DBUG[11-07|16:51:06] Success for task operation: f2f5075f-5b97-4f3f-9e14-d2f67869b900 
DBUG[11-07|16:51:06] New task Operation: 3da1c9f5-a0d3-4da8-8646-d6ed445fbcfb 
INFO[11-07|16:51:06] Pruning expired images 
DBUG[11-07|16:51:06] Started task operation: 3da1c9f5-a0d3-4da8-8646-d6ed445fbcfb 
INFO[11-07|16:51:06] Done pruning expired images 
DBUG[11-07|16:51:06] New task Operation: ff9f8e5e-1b30-4648-a212-aa415be1ac03 
DBUG[11-07|16:51:06] Success for task operation: 3da1c9f5-a0d3-4da8-8646-d6ed445fbcfb 
INFO[11-07|16:51:06] Pruning expired container backups 
DBUG[11-07|16:51:06] Started task operation: ff9f8e5e-1b30-4648-a212-aa415be1ac03 
DBUG[11-07|16:51:06] Success for task operation: ff9f8e5e-1b30-4648-a212-aa415be1ac03 
INFO[11-07|16:51:06] Done pruning expired container backups 
DBUG[11-07|16:51:06] New task Operation: 404cbf64-4823-4e63-98f9-7d843f283e1f 
DBUG[11-07|16:51:06] New task Operation: a890790f-c9f0-4bbe-93d5-cbdcc8b6fe70 
INFO[11-07|16:51:06] Updating instance types 
DBUG[11-07|16:51:06] Started task operation: 404cbf64-4823-4e63-98f9-7d843f283e1f 
INFO[11-07|16:51:06] Expiring log files 
DBUG[11-07|16:51:06] Started task operation: a890790f-c9f0-4bbe-93d5-cbdcc8b6fe70 
INFO[11-07|16:51:06] Done updating instance types 
INFO[11-07|16:51:06] Done expiring log files 
DBUG[11-07|16:51:06] Success for task operation: a890790f-c9f0-4bbe-93d5-cbdcc8b6fe70 


DBUG[11-07|16:51:11] Success for task operation: 404cbf64-4823-4e63-98f9-7d843f283e1f 
^CINFO[11-07|16:51:14] Received 'interrupt signal', exiting 
INFO[11-07|16:51:14] Starting shutdown sequence 
INFO[11-07|16:51:14] Stopping REST API handler: 
INFO[11-07|16:51:14]  - closing socket                        socket=/var/lib/lxd/unix.socket
INFO[11-07|16:51:14] Stopping /dev/lxd handler: 
INFO[11-07|16:51:14]  - closing socket                        socket=/var/lib/lxd/devlxd/sock
INFO[11-07|16:51:14] Closing the database 
DBUG[11-07|16:51:14] Stop database gateway 
INFO[11-07|16:51:14] Unmounting temporary filesystems 
INFO[11-07|16:51:14] Done unmounting temporary filesystems

Do you have some customization’s for tmpfs perhaps? Your core dump occurs at creating the storage pool as that’s what I see after your system core dumps.

Hi,
no customizations am aware of.
here are logs;

df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs         16G     0   16G   0% /dev
tmpfs            16G   87M   16G   1% /dev/shm
tmpfs            16G   10M   16G   1% /run
tmpfs            16G     0   16G   0% /sys/fs/cgroup
/dev/sdd3        95G   34G   56G  38% /
/dev/sdd1       511M   29M  482M   6% /boot/efi
/dev/sdb1       916G  468G  403G  54% /home
tmpfs           3.2G   32K  3.2G   1% /run/user/457
tmpfs           3.2G   44K  3.2G   1% /run/user/1000
tmpfs           100K     0  100K   0% /var/lib/lxd/shmounts
tmpfs           100K     0  100K   0% /var/lib/lxd/devlxd

malcolmlewis · November 8, 2019, 12:48am

Hi
If you have these then lxd is running, can you check for stale lxd running, if not, can you remove those two directories and try starting lxd again.


ps -ef |grep lxd
root      3882     1  0 17:45 ?        00:00:00 /usr/bin/lxd --group=lxd --logfile=/var/log/lxd/lxd.log
root      3920  3508  0 17:47 pts/2    00:00:00 grep --color=auto lxd

systemctl stop lxd

ps -ef |grep lxd
root      3926  3508  0 17:47 pts/2    00:00:00 grep --color=auto lxd

seven_skies · November 8, 2019, 12:55am

malcolmlewis:

Hi
If you have these then lxd is running, can you check for stale lxd running, if not, can you remove those two directories and try starting lxd again.
ps -ef |grep lxd
root      3882     1  0 17:45 ?        00:00:00 /usr/bin/lxd --group=lxd --logfile=/var/log/lxd/lxd.log
root      3920  3508  0 17:47 pts/2    00:00:00 grep --color=auto lxd

systemctl stop lxd

ps -ef |grep lxd
root      3926  3508  0 17:47 pts/2    00:00:00 grep --color=auto lxd

still no luck

systemctl status lxd
● lxd.service - LXD Container Hypervisor
   Loaded: loaded (/usr/lib/systemd/system/lxd.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:lxd(1)

systemctl start lxd
hangs.....
ps -ef |grep lxd
root      3930  3805  0 02:53 pts/2    00:00:00 systemctl start lxd
root      3934     1  0 02:53 ?        00:00:00 /usr/bin/lxd waitready --timeout=600
root      3973  3714  0 02:54 pts/0    00:00:00 grep --color=auto lxd

ls -all /var/lib/lxd/
total 56
drwx--x--x 14 root root 4096 Nov  8 02:53 .
drwxr-xr-x 63 root root 4096 Nov  8 01:21 ..
drwx------  2 root root 4096 Nov  8 01:05 backups
drwx--x--x  2 root root 4096 Nov  8 01:05 containers
drwx------  3 root root 4096 Nov  8 01:05 database
drwx--x--x  2 root root 4096 Nov  8 01:05 devices
drwxr-xr-x  2 root root   60 Nov  8 02:53 devlxd
drwx------  2 root root 4096 Nov  8 01:05 disks
drwx------  2 root root 4096 Nov  8 01:05 images
drwx--x--x  2 root root 4096 Nov  8 01:05 networks
drwx------  2 root root 4096 Nov  8 01:05 security
-rw-r--r--  1 root root  757 Nov  8 01:05 server.crt
-rw-------  1 root root  288 Nov  8 01:05 server.key
drwx--x--x  2 root root   40 Nov  8 02:53 shmounts
drwx------  2 root root 4096 Nov  8 01:05 snapshots
drwx--x--x  2 root root 4096 Nov  8 01:05 storage-pools
srw-rw----  1 root lxd     0 Nov  8 02:53 unix.socket