/srv empty after booting, recovering folders after reboot

[Voronamanga]

We progress ! lol! I ran journalctl as soon as the problem appeared, and …

erik@linux-5n3o:~> sudo journalctl | grep srv
[sudo] Mot de passe de root : 
août 21 19:46:40 linux-5n3o systemd[1]: Mounting /srv...
août 21 19:46:40 linux-5n3o systemd[1]: Mounted /srv.
août 21 19:46:47 linux-5n3o systemd[1]: Unmounting /srv...
août 21 19:46:47 linux-5n3o systemd[1]: srv.mount: Succeeded.
août 21 19:46:47 linux-5n3o systemd[1]: Unmounted /srv.

So I just have to mount /srv, no need to reboot !

Any idea about what could make /srv being unmounted during boot up ?:sarcastic:

(edit)
At the same time, I found apache2 inactive, though it is normally launched at start up and I had to start it … wow …

[hcvv]

I see higher up in this thread that /srv is a Btrfs subvolume.
I have no Btrfs knowledge (and do not use it). Thus others may try to explain this.

[Voronamanga]

@Henk : thanks anyway !

I wonder why I limited journalctl to grep /srv. Next time I’ll have to look at messages just preceding “unmount /srv”…

[arvidjaar]

@Voronamanga

Any idea about what could make /srv being unmounted during boot up ?

Boot with “systemd.log_level=debug printk.devkmsg=on log_buf_len=8M” and provide full output of “journalctl -b”.

[Voronamanga]

@arvidjaar

Boot with “systemd.log_level=debug printk.devkmsg=on log_buf_len=8M” and provide full output of “journalctl -b”.

Hi arvidjaar, thank you for replying.

I booted this evening and encountered the issue which is the matter of this thread. Unfortunately I could’nt have read your post before. Then I rebooted the way you suggested but as usually, the problem did’nt show off again - anyway here is the result of journalctl -b :
https://susepaste.org/46695941

I noticed that many directories, and not only /srv, were told as unmounted in the former log. Being declared as unmounted sounds weird to me, because they are present in the system tree, the problem is that they look empty except hidden files. They all figure in my fstab :

UUID=63d518af-52ff-4179-b186-340a7abd02b8  /                       btrfs  defaults                      0  0
UUID=63d518af-52ff-4179-b186-340a7abd02b8  /.snapshots             btrfs  subvol=/@/.snapshots          0  0
UUID=7f106034-fa4d-4db0-985e-d6cac4d28178  swap                    swap   defaults                      0  0
UUID=63d518af-52ff-4179-b186-340a7abd02b8  /var                    btrfs  subvol=/@/var                 0  0
UUID=63d518af-52ff-4179-b186-340a7abd02b8  /usr/local              btrfs  subvol=/@/usr/local           0  0
UUID=63d518af-52ff-4179-b186-340a7abd02b8  /tmp                    btrfs  subvol=/@/tmp                 0  0
UUID=63d518af-52ff-4179-b186-340a7abd02b8  /srv                    btrfs  subvol=/@/srv                 0  0
UUID=63d518af-52ff-4179-b186-340a7abd02b8  /root                   btrfs  subvol=/@/root                0  0
UUID=63d518af-52ff-4179-b186-340a7abd02b8  /opt                    btrfs  subvol=/@/opt                 0  0
UUID=72aa2929-b98f-449b-935c-a833d98d5655  /home                   ext4   data=ordered                  0  2
UUID=63d518af-52ff-4179-b186-340a7abd02b8  /boot/grub2/x86_64-efi  btrfs  subvol=/@/boot/grub2/x86_64-efi  0  0
UUID=63d518af-52ff-4179-b186-340a7abd02b8  /boot/grub2/i386-pc     btrfs  subvol=/@/boot/grub2/i386-pc  0  0
UUID=58ED-09A0                             /boot/efi               vfat   defaults                      0  0
LABEL=Media                                /home/erik/Media        ext4   user_xattr,data=ordered       0  0

What suggested me to look into fstab was the fact that directories (which appeared to be the ones listed in fstab) were successively unmounted an mounted, and unmounted, in the journal report. I have no idea of the reason for that.

The former output of journalctl, a couple of hours ago when I booted with unsuccessful mount of /srv (and probably other directories), showed errors with freedesktop and others, and asked for a fsck on dev/sdc1. I give it below in the case it would be useful.
https://susepaste.org/64601347

Thanks for anything that could make me wiser than today !

[Voronamanga]

Still having the same issue. Not that much ennoying because it’s easy to work around, but it makes me wonder if it could be a hardware problem, since no other hint came out.

And sometimes, my system just does’nt boot at all - no beep, the fan is active, and no screen … after reboot everything’s working fine.

Weird.:sarcastic:

Any idea ?

[malcolmlewis]

@Voronamanga

Still having the same issue. Not that much annoying because it’s easy to work around, but it makes me wonder if it could be a hardware problem, since no other hint came out.

And sometimes, my system just doesn’t boot at all - no beep, the fan is active, and no screen … after reboot everything’s working fine.

Weird.:sarcastic:

Any idea ?

Hi
And the journal output shows it being unmounted again?

Normally the likes of /srv and especially databases should be off on their own xfs (or other preferred filesystem) partitions if setting up a web server…

[Voronamanga]

Hello malcolmlewis, thanks for your answer.

@malcolmlewis

Hi
And the journal output shows it being unmounted again?

I did’nt check it lastly, but I assume it’s the same as before, and, yes, the journal said it was mounted, and then unmounted. It’s not that it was never mounted.

@malcolmlewis

Normally the likes of /srv and especially databases should be off on their own xfs (or other preferred filesystem) partitions if setting up a web server…

My understanding, either of english or of technical stuff regarding web servers, seams a bit too weak … :\ I admit I did’nt really understand that… could you please explain a little bit more ? Thanks !

[malcolmlewis]

@Voronamanga

Hello malcolmlewis, thanks for your answer.

I did’nt check it lastly, but I assume it’s the same as before, and, yes, the journal said it was mounted, and then unmounted. It’s not that it was never mounted.

My understanding, either of english or of technical stuff regarding web servers, seams a bit too weak … :\ I admit I did’nt really understand that… could you please explain a little bit more ? Thanks !

Hi
For databases which tend to read/write a lot then btrfs is not really suitable, during install one would normally create a /var/lib/pgsql partition for the database, at this point would also put /srv onto it’s own partition it all makes it easier to backup as well… or even use LVM… But it’s something that needs to be considered before install/setup as to the final use of the system. Perhaps consider a virtual machine on your host operating system?

[Voronamanga]

@malcolmlewis

Hi
For databases which tend to read/write a lot then btrfs is not really suitable, during install one would normally create a /var/lib/pgsql partition for the database, at this point would also put /srv onto it’s own partition it all makes it easier to backup as well… or even use LVM… But it’s something that needs to be considered before install/setup as to the final use of the system. Perhaps consider a virtual machine on your host operating system?

OK, thanks for those precisions. My postgresql database here, and the server as well, are just for development purpose, so the apache web server does’nt face significant transaction loads (for production I’ll totally rely on my web host). I’ll remember that btrfs is’nt maybe the most suitable file system anyway, so actually why not put all the database-related stuff on a different partition with a fs such as ext4 for instance. By the way, how could it explain that other directories are unmounted ? I’d have to come back upon that, but I’m pretty sure that other subdirectories of / are impacted. I confess my lack of knowledge, also regarding the use of a virtual machine, that I always used only for embedding an OS into another one.

What would you think of a reinstall of opensuse with ext4 fs ? Though the amount of work makes me a bit reluctant, but when one has to do it …

[malcolmlewis]

@Voronamanga

OK, thanks for those precisions. My postgresql database here, and the server as well, are just for development purpose, so the apache web server does’nt face significant transaction loads (for production I’ll totally rely on my web host). I’ll remember that btrfs is’nt maybe the most suitable file system anyway, so actually why not put all the database-related stuff on a different partition with a fs such as ext4 for instance. By the way, how could it explain that other directories are unmounted ? I’d have to come back upon that, but I’m pretty sure that other subdirectories of / are impacted. I confess my lack of knowledge, also regarding the use of a virtual machine, that I always used only for embedding an OS into another one.

What would you think of a reinstall of opensuse with ext4 fs ? Though the amount of work makes me a bit reluctant, but when one has to do it …

Hi
Well it’s a bug somewhere for sure as your not the only one having the issue. I have SSD/NVMe here and working fine with btrfs/xfs across multiple partitions. Not sure if it’s a rotating HDD issue with any filesystems as some folks having issues use ext4. Combination of SSD and Rotating HDD…

https://forums.opensuse.org/showthread.php/537920-Yet-another-mount-at-boot-problem-sda1-not-mounting-at-boot

[malcolmlewis]

@malcolmlewis

Hi
Well it’s a bug somewhere for sure as your not the only one having the issue. I have SSD/NVMe here and working fine with btrfs/xfs across multiple partitions. Not sure if it’s a rotating HDD issue with any filesystems as some folks having issues use ext4. Combination of SSD and Rotating HDD…

https://forums.opensuse.org/showthread.php/537920-Yet-another-mount-at-boot-problem-sda1-not-mounting-at-boot

Hi
And the other current thread…
https://forums.opensuse.org/showthread.php/537966-Partitions-are-unmounted-during-boot

[Voronamanga]

Hi MalcolmLewis,

very interesting ! Those threads have begun after I reported the problem here for the first time, and I had’nt made a research about that recently, so at least the idea that it’s probably a “bug somewhere” is comforting.

Just for information I have exactly the same OS (Tumbleweed) on a machine at my work but with a SSD HDD and I never had that kind of issue. But there’s no apache server on it, which is maybe important too.

And I really have a hardware problem too, probably my PSU getting a bit old. Hope it’s not the motherboard, crossing fingers.

Thanks again, cheers !