# snapper ls; ll /.snapshots/; ll /var/lib/overlay/
# | Type | Pre # | Date | User | Used Space | Cleanup | Description | Userdata
---+--------+-------+--------------------------+------+------------+---------+-----------------------+---------
0 | single | | | root | | | current |
1 | single | | Tue Jun 25 00:15:37 2024 | root | 768.00 KiB | | first root filesystem |
2 | single | | Thu Jul 25 13:56:02 2024 | root | 7.43 MiB | number | Snapshot Update of #1 |
3 | single | | Wed Aug 21 04:57:24 2024 | root | 896.00 KiB | number | Snapshot Update of #1 |
4 | single | | Tue Oct 8 04:03:53 2024 | root | 1.14 MiB | number | Snapshot Update of #3 |
5* | single | | Tue Oct 8 04:05:55 2024 | root | 93.14 MiB | number | Snapshot Update of #4 |
total 4
drwxr-xr-x. 1 root root 66 Jul 25 13:56 1
drwxr-xr-x. 1 root root 66 Aug 21 04:57 2
drwxr-xr-x. 1 root root 66 Oct 8 04:03 3
drwxr-xr-x. 1 root root 66 Oct 8 04:05 4
drwxr-xr-x. 1 root root 32 Oct 8 04:06 5
-rw-r-----. 1 root root 616 Oct 8 04:06 grub-snapshot.cfg
total 0
drwxr-xr-x. 1 root root 30 Oct 8 04:03 1
drwxr-xr-x. 1 root root 22 Jul 25 13:56 2
drwxr-xr-x. 1 root root 30 Oct 8 04:05 3
drwxr-xr-x. 1 root root 22 Oct 8 04:03 4
drwxr-xr-x. 1 root root 22 Oct 8 04:05 5
drwxr-xr-x. 1 root root 8 Aug 21 04:54 work-etc
# find / -name check-certificates.service
find: File system loop detected; ‘/.snapshots/5/snapshot’ is part of the same file system loop as ‘/’.
/var/lib/overlay/4/etc/systemd/system/check-certificates.service
/etc/systemd/system/check-certificates.service
find: ‘/proc/2450’: No such file or directory
Then I run:
# transactional-update pkg install wget
Checking for newer version.
transactional-update 4.6.6 started
Options: pkg install wget
Separate /var detected.
2024-10-11 13:43:32 tukit 4.6.6 started
2024-10-11 13:43:32 Options: -c5 open
2024-10-11 13:43:33 Using snapshot 5 as base for new snapshot 6.
2024-10-11 13:43:33 /var/lib/overlay/5/etc
2024-10-11 13:43:33 Syncing /etc of previous snapshot 4 as base into new snapshot "/.snapshots/6/snapshot"
2024-10-11 13:43:33 SELinux is enabled.
ID: 6
2024-10-11 13:43:35 Transaction completed.
Calling zypper install
2024-10-11 13:43:39 tukit 4.6.6 started
2024-10-11 13:43:39 Options: callext 6 zypper -R {} install wget
2024-10-11 13:43:41 Executing `zypper -R /tmp/transactional-update-ctt581 install wget`:
Refreshing service 'openSUSE'.
Loading repository data...
Reading installed packages...
Resolving package dependencies...
The following 3 NEW packages are going to be installed:
libcares2 libmetalink3 wget
3 new packages to install.
Overall download size: 543.9 KiB. Already cached: 0 B. After the operation, additional 985.0 KiB will be used.
Continue? [y/n/v/...? shows all options] (y): y
Checking for file conflicts: [...done]
Warning: 3 packages had to be excluded from file conflicts check because they are not yet downloaded.
Note: Checking for file conflicts requires not installed packages to be downloaded in advance in
order to access their file lists. See option '--download-in-advance / --dry-run --download-only'
in the zypper manual page for details.
Retrieving: libcares2-1.27.0-1.7.x86_64 (repo-main (6.0)) (1/3), 111.1 KiB
Retrieving: libcares2-1.27.0-1.7.x86_64.rpm [............done (111.1 KiB/s)]
(1/3) Installing: libcares2-1.27.0-1.7.x86_64 [..done]
Retrieving: libmetalink3-0.1.3-1.5.x86_64 (repo-main (6.0)) (2/3), 30.2 KiB
Retrieving: libmetalink3-0.1.3-1.5.x86_64.rpm [............done (30.2 KiB/s)]
(2/3) Installing: libmetalink3-0.1.3-1.5.x86_64 [...done]
Retrieving: wget-1.24.5-1.1.x86_64 (repo-main (6.0)) (3/3), 402.6 KiB
Retrieving: wget-1.24.5-1.1.x86_64.rpm [...................done (402.6 KiB/s)]
(3/3) Installing: wget-1.24.5-1.1.x86_64 [..done]
2024-10-11 13:43:49 Application returned with exit status 0.
2024-10-11 13:43:49 Transaction completed.
Trying to rebuild kdump initrd
2024-10-11 13:43:52 tukit 4.6.6 started
2024-10-11 13:43:52 Options: close 6
2024-10-11 13:43:55 New default snapshot is #6 (/.snapshots/6/snapshot).
2024-10-11 13:43:55 Transaction completed.
Please reboot your machine to activate the changes and avoid data loss.
New default snapshot is #6 (/.snapshots/6/snapshot).
transactional-update finished
In this I can clearly see that:
btrfs snapshot is done like this: Using snapshot 5 as base for new snapshot 6.
but /etc is copied from last but one snapshot (overlay): Syncing /etc of previous snapshot 4 as base into new snapshot "/.snapshots/6/snapshot"
This causes that if my /etc changes were in snapshot 5 it is not visible in snapshot 6 (and I have to wait until snapshot 7).
Could you please answer me these questions?:
Why is it done like this?
How to deal with this? Can I somehow manually create an empty snapshot (with transactional update) to have btrfs and /etc from last snapshot?
Does this mean that I want always have not only last snapshot but 2 last snapshots?
Thanks for reply. You’re right. I think I found a solution. When I wanted to create data for the question I was proceeding from a mistaken belief about the behavior of the system and therefore I was asserting something that was not true and on the wrong examples. I have to apologize for that.
My problem is not this procedure but what I did before because I didn’t know about that. My effort was to cleanup all the snapshots so I deleted all of them and left only last one.
# snapper ls; ll /.snapshots/; ll /var/lib/overlay/; findmnt -u /etc
# | Type | Pre # | Date | User | Used Space | Cleanup | Description | Userdata
---+--------+-------+--------------------------+------+------------+---------+-----------------------+---------
0 | single | | | root | | | current |
5* | single | | Tue Oct 8 04:05:55 2024 | root | 1.45 GiB | number | Snapshot Update of #4 |
total 4
drwxr-xr-x. 1 root root 66 Oct 12 04:56 5
-rw-r-----. 1 root root 292 Oct 12 04:56 grub-snapshot.cfg
total 0
drwxr-xr-x. 1 root root 30 Oct 8 04:03 1
drwxr-xr-x. 1 root root 22 Jul 25 13:56 2
drwxr-xr-x. 1 root root 30 Oct 8 04:05 3
drwxr-xr-x. 1 root root 22 Oct 8 04:03 4
drwxr-xr-x. 1 root root 22 Oct 8 04:05 5
drwxr-xr-x. 1 root root 8 Aug 21 04:54 work-etc
TARGET
SOURCE FSTYPE OPTIONS
/etc overlay overlay rw,relatime,seclabel,lowerdir=/sysroot/var/lib/overlay/4/etc:/sysroot/etc,upperdir=/sysroot/var/lib/overlay/5/etc,workdir=/sysroot/var/lib/overlay/5/work-etc
# transactional-update pkg install wget
Checking for newer version.
transactional-update 4.6.6 started
Options: pkg install wget
Separate /var detected.
2024-10-12 04:58:54 tukit 4.6.6 started
2024-10-12 04:58:54 Options: -c5 open
2024-10-12 04:58:54 Using snapshot 5 as base for new snapshot 6.
2024-10-12 04:58:54 /var/lib/overlay/5/etc
2024-10-12 04:58:54 Parent snapshot 4 does not exist any more - skipping rsync
ID: 6
2024-10-12 04:58:56 Transaction completed.
Calling zypper install
2024-10-12 04:58:59 tukit 4.6.6 started
2024-10-12 04:58:59 Options: callext 6 zypper -R {} install wget
2024-10-12 04:59:01 Executing `zypper -R /tmp/transactional-update-sVMY8h install wget`:
Refreshing service 'openSUSE'.
Loading repository data...
Reading installed packages...
Resolving package dependencies...
The following 3 NEW packages are going to be installed:
libcares2 libmetalink3 wget
3 new packages to install.
Overall download size: 543.9 KiB. Already cached: 0 B. After the operation, additional 985.0 KiB will be used.
Continue? [y/n/v/...? shows all options] (y):
After deleting all snapshots, transactional-update is skipping the rsyncing of some data in etc because the previous snapshot does not exist (although overlay exists). This caused this strange behavior because Parent snapshot 4 does not exist any more - skipping rsync . So I had to wait for that until next snapshot.
I think I can now pretty well understand how btrfs snapshots work but one thing I still do not understand is how these overlays work. I don’t mean the concept of lower, uppper and work dir, this is quiet clear. But I don’t know what exactly happens with those overlays when transactional update is run. Could you please explain it to me?
Thank you for your time in pushing me in the right direction.
Please, drop this habit. It makes it near to impossible to match command output with the command itself or recognize where one command output ends and another command output starts.
The content of /etc in the current snapshot (5 in your example) may change after new snapshot (6 in your example) has been created. You are not forced to reboot immediately. If transactional-update made a copy of the current state of (combined) /etc in the 5 and you changed something, new snapshot 6 would miss these changes. So, transactional-update coalesces the content of the /etc in the previous snapshot (4 in your example) which is assumed to be immutable, effectively combining the content in the snapshot itself and the content in the writable directory /var/lib/overlay/4/etc, writes it as base into the immutable part of the new snapshot (6) and configures snapshot 6 to also use additional lowerdir with the changed content of the current snapshot (upperdir of the 5). After reboot into the snapshot 6 the base content of the /etc will be the content of all snapshots up to and including the “grandparent” (4) plus the changes in the 5 itself. As if you rebooted immediately after having created the snapshot.
Please tell me if understand it well:
There is always snapshot (with /etc), lowerdir and upperdir. Each snapshot has its own overlay of /etc and its number is the same like number of the snapshot.
In my case I am running snapshot 5 and using /etc overlay associated with snapshot 4 (let’s call it overlay 4) as lowerdir and overlay 5 as upperdir.
When I run transactional-update this happens:
the new btrfs snapshot with number 6 is created
data from /etc in snapshot 4 and overlay 4 are combined and copied to the snapshot 6 - this is the base of new /etc folder in snapshot 6
after reboot overlay 5 is mounted as lowerdir and newly created overlay 6 as upperdir
this way you can have “rock stable” data in /etc from snapshot/overlay 4 (in snapshot 6), combined with overlay 5.
