Page 1 of 2 12 LastLast
Results 1 to 10 of 18

Thread: Impossibly large file

  1. #1

    Default Impossibly large file

    I was tracing why a tar was creating an absurdly large backup. For a directory tree that "du" claims has about 780MB, it had made 7.9GB file and was continuing to add to it.

    I finally found the issue.
    Code:
    $ ll -h /var/lib/dhcp/proc/kcore
    -r-------- 1 root root 128T Aug  1 21:10 /var/lib/dhcp/proc/kcore
    A 128 terabyte (140,737,477,885,952 bytes) file. On a volume that has about 300GB free. tar valiantly tried to back it up.

    An attempt to remove it did not succeed: "Read-only file system". Probably just as well.

    How is this file size possible? What is this file that is a file system?

    Adding an --exclude option fixed the backup issue.
    Code:
    $ tar --exclude=/var/lib/dhcp/proc/* -czf /data01/backups/root/var-lib.tar.gz /var/lib/

  2. #2
    Join Date
    May 2012
    Location
    Finland
    Posts
    2,005

    Default Re: Impossibly large file

    It represents the physical memory of your system - it's there because dhcp runs in chroot.

    You should never backup proc directories as those are virtual and are re-created on every system startup (as well as /dev, /sys, /tmp, /run and a few other locations)
    .: miuku #suse @ irc.freenode.net
    :: miuku@opensuse.org

    .: h​ttps://download.opensuse.org/repositories/home:/Miuku/

  3. #3

    Default Re: Impossibly large file

    Along with this, you may also want to consider the --one-file-system
    option to the tar command if you are doing backups like this. It should
    prevent tar from wandering across filesystem boundaries, which in your
    case would probably stpo it from going into /var/lib/dhcp/proc in the
    first place. Keep in mind some applications MAY stretch across
    filesystems, but it is more common for all data to be in one.


    --
    Good luck.

    If you find this post helpful and are logged into the web interface,
    show your appreciation and click on the star below.

    If you want to send me a private message, please let me know in the
    forum as I do not use the web interface often.

  4. #4
    Join Date
    Aug 2010
    Location
    Chicago suburbs
    Posts
    12,794
    Blog Entries
    3

    Default Re: Impossibly large file

    I normally don't backup the root file system, because I would prefer to reinstall rather than recover from backup.

    However, the last time that I did a backup of the root file system, I used something like:

    Code:
    mount --bind / /mnt
    and then I backed up "/mnt" which was really the root partition but without any mounts on it.

    When done with the backup, I used
    Code:
    umount /mnt
    to undo that bind mount.
    openSUSE Leap 15.1; KDE Plasma 5;
    testing Leap 15.2Alpha

  5. #5

    Default Re: Impossibly large file

    Quote Originally Posted by Miuku View Post
    It represents the physical memory of your system - it's there because dhcp runs in chroot.
    But Ihave only 8GB RAM, not 128TB.

    You should never backup proc directories as those are virtual and are re-created on every system startup
    Lesson learned.

  6. #6

    Default Re: Impossibly large file

    Quote Originally Posted by nrickert View Post
    I normally don't backup the root file system, because I would prefer to reinstall rather than recover from backup.
    I only backup /etc/ and /var/lib/. Which had proven to be wise for the LEAP 15.0 upgrade from 42.3; the upgrade did not go well for non-default services, most of them being either not enabled or reset to default values. Having a backup of /etc/ and /var/lib saved me some grief.

    However, the last time that I did a backup of the root file system, I used something like:
    The suggestion would do nothing to prevent the original problem: backing up the mysteriously huge 128TB file.

  7. #7
    Join Date
    Nov 2009
    Location
    West Virginia Sector 13
    Posts
    15,762

    Default Re: Impossibly large file

    Well no mystery, That is a virtual file that represents the whole address space available to the processor. It is created at boot. Normally it is in /proc. Not in /var but apparently some program decided it needed a link. dhcp??? any how you have no need and should never backup anything in any directory called /proc

  8. #8
    Join Date
    Aug 2010
    Location
    Chicago suburbs
    Posts
    12,794
    Blog Entries
    3

    Default Re: Impossibly large file

    Quote Originally Posted by jimoe666 View Post
    The suggestion would do nothing to prevent the original problem: backing up the mysteriously huge 128TB file.
    Did you try it?

    That 128TB file is not present in the "/mnt" tree that I created with that bind mount. So it won't be backed up.
    openSUSE Leap 15.1; KDE Plasma 5;
    testing Leap 15.2Alpha

  9. #9

    Default Re: Impossibly large file

    Quote Originally Posted by nrickert View Post
    Did you try it?
    I did.
    Apparently I did not look in the correct directory when I was verifying the result earlier. Doing this again, there is nothing under .../var/lib/dhcp/proc/, as you noted.

  10. #10
    Join Date
    Feb 2010
    Location
    Germany
    Posts
    2,570

    Question Re: Impossibly large file

    Quote Originally Posted by jimoe666 View Post
    I only backup /etc/ and /var/lib/. Which had proven to be wise for the LEAP 15.0 upgrade from 42.3; the upgrade did not go well for non-default services, most of them being either not enabled or reset to default values. Having a backup of /etc/ and /var/lib saved me some grief.
    After the upgrade, did you execute the following?
    • 'rpmconfigcheck' -- normally indicates which configuration files need to be checked for changes after the upgrade …
    • 'zypper verify --details --recommends'
    • 'rpm --verify --all'

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •