Page 1 of 2 12 LastLast
Results 1 to 10 of 12

Thread: Disk space not freed up after deletion of big file (caused by virtiofs)

  1. #1

    Question Disk space not freed up after deletion of big file (caused by virtiofs)

    The title says the problem but I'd like to understand why this happens.

    So, when I delete file (<40..50GB) the space is freed up immediately but when I'm deleting big file (>50GB) then the space is not freed up. File system is EXT4. Specific folder from where the deletion happens is also shared via virtiofs to virtual machine (qemu). No-one in virtual machine, nor in host is using the files, locking it, hard linking etc. LSOF cannot find anything. And strange thing is that this happens only to big files - not smaller.

    When I reboot the virtual machine then space is freed up on the host side. I would like to avoid regular rebooting because of this.

    This was not an issue earlier. Don't recall exactly when this started but several months ago anyhow. I have reinstalled the virtual machine so the mounting is done differently (older method was obsoleted) - maybe this is the reason? If so, why and how to change to the old behavior where space was freed up immediate after deletion on host side.

    Mounting on virtual machine side (currently):
    Code:
    /data     /data    virtiofs rw,noatime,_netdev 0 2
    Older mounting way:
    Code:
    /data    /data    9p auto,x-systemd.automount,x-systemd.device-timeout=10,timeo=14,x-systemd.idle-timeout=0,trans=virtio,version=9p2000.L,rw 0 0

  2. #2
    Join Date
    Sep 2012
    Posts
    8,042

    Default Re: Disk space not freed up after deletion of big file (caused by virtiofs)

    Quote Originally Posted by paju-21 View Post
    Specific folder from where the deletion happens is also shared via virtiofs to virtual machine (qemu).
    Does it also happen when virtual machine is stopped?
    LSOF cannot find anything.
    It is rather vague. How exactly are you using lsof to check?
    Code:
    /data     /data    virtiofs rw,noatime,_netdev 0 2
    Does it happen if you comment out this line in /etc/fstab and start VM?
    Older mounting way:
    Did it happen when you mounted this way?

  3. #3
    Join Date
    Apr 2011
    Location
    Texas USA
    Posts
    519

    Default Re: Disk space not freed up after deletion of big file (caused by virtiofs)

    I am not sure from your question what you are seeing related to deletion of large files but:

    From past experience, look in all system .Trash directories. Many root directories have/create separate .Trash subdirectories. Files get deleted from the filesystem but still occupy disk space when retained in .Trash. Manually delete the files in these .Trash directories.

    I had df saying that system was full and out of root (/) disk space due to 70 gig of files in .Trash when it was really only 30% full. The deleted files retained in .Trash were from deletion of vms when I allowed vm storage in root directories.

    tom kosvic

  4. #4

    Default Re: Disk space not freed up after deletion of big file (caused by virtiofs)

    Quote Originally Posted by arvidjaar View Post
    Does it also happen when virtual machine is stopped?
    That one I have not tried yet. To be checked. VM needs to be running so not much use for this approach but can be tried.

    It is rather vague. How exactly are you using lsof to check?
    Various ways what I could find from net. Plain lsof, lsof +L1, lsof -nP +L1.. probably more, don't recall anymore which I used. Tried many ways but nothing was shown.

    Does it happen if you comment out this line in /etc/fstab and start VM?
    Haven't tried. This would be rather useless for me since I need VM to access that folder and by commenting it out VM would be useless.

    Did it happen when you mounted this way?
    No. Old way of mounting didn't cause this. Is it related to mounting change, cannot say for sure but a good candidate for it.

  5. #5

    Default Re: Disk space not freed up after deletion of big file (caused by virtiofs)

    Quote Originally Posted by tckosvic View Post
    I am not sure from your question what you are seeing related to deletion of large files but:

    From past experience, look in all system .Trash directories. Many root directories have/create separate .Trash subdirectories. Files get deleted from the filesystem but still occupy disk space when retained in .Trash. Manually delete the files in these .Trash directories.

    I had df saying that system was full and out of root (/) disk space due to 70 gig of files in .Trash when it was really only 30% full. The deleted files retained in .Trash were from deletion of vms when I allowed vm storage in root directories.

    tom kosvic
    Not the case. I'm deleting files. Not moving to trash. Trash would not clear out when VM is stopped but in my case the space gets freed up once VM is shutdown. Affects only large files.

  6. #6

    Default Re: Disk space not freed up after deletion of big file (caused by virtiofs)

    Umounting the shared folder from VM is enough to free up the space after deletion. Nothing on VM side is keeping those files open in any way. And only big files are affected.. very strange.

  7. #7

    Default Re: Disk space not freed up after deletion of big file (caused by virtiofs)

    Hello,

    My response is rather anecdotal, but I am almost certain that I have observed similar "delay?" with ext4 on luks on mdraid. For whatever it may be worthy ... the space eventually shows up.


    ???? ... as to why,how,what .... I have not a clue!

  8. #8

    Default Re: Disk space not freed up after deletion of big file (caused by virtiofs)

    Quote Originally Posted by oxWrongAgain View Post
    Hello,

    My response is rather anecdotal, but I am almost certain that I have observed similar "delay?" with ext4 on luks on mdraid. For whatever it may be worthy ... the space eventually shows up.


    ???? ... as to why,how,what .... I have not a clue!
    Any idea about the time? I waited several hours (did something else in the meanwhile). Previously I'd say longer period of time without freeing up.

    Freeing up of the space started immediately when I unmounted the folder. And it took several minutes before unmounting completed. This time roughly 200GB got freed up - maybe more (free space was shown in TB hence the inaccuracy).

  9. #9
    Join Date
    Apr 2011
    Location
    Texas USA
    Posts
    519

    Default Re: Disk space not freed up after deletion of big file (caused by virtiofs)

    Quote Originally Posted by paju-21 View Post
    Not the case. I'm deleting files. Not moving to trash. Trash would not clear out when VM is stopped but in my case the space gets freed up once VM is shutdown. Affects only large files.
    I deleted vm files from their storage llocation in the root directory using vm manager and libvirt put the "deleted" files into .Trash directories in several root directories. I did nothing specific to send the files to .Trash, I deleted the storage through vm manager. The files were deleted from the file system but not deleted from the disk and still took space. May not be your problem.

  10. #10

    Default Re: Disk space not freed up after deletion of big file (caused by virtiofs)

    Quote Originally Posted by paju-21 View Post
    Any idea about the time? I waited several hours (did something else in the meanwhile). Previously I'd say longer period of time without freeing up.

    Freeing up of the space started immediately when I unmounted the folder. And it took several minutes before unmounting completed. This time roughly 200GB got freed up - maybe more (free space was shown in TB hence the inaccuracy).

    Oh boy!

    Please keep in mind the anecdotal nature of my response. I do NOT completely trust the accuracy of my own memory! That said, when I have noticed this behavior ( I have only noticed it seat of the pants moving files from a full partition to another because I want to put something there ), it's somewhere in the plus or minus ten minute neighborhood. Until I saw your post, I have never given it much thought, as the space does "reappear". But I do distinctly remember being a confused by the output of "df" and "du -sh" on numerous occasions.

    The point of my response was to caution you not be so certain that the culprit is a vm share. It may be the culprit, or perhaps the vm share is a contributing factor, but I suspect, based on my extremely human imperfect memory that something more inherent to file system tracking MAY be a play.

    I have a bunch of video files in my que to review/edit. Most will be either be deleted or "cut" to a fraction of there original size. So I am actually in a good position to watch for for the phenomena as I expect to gain a full TB of capacity over the next week or two, in pretty BIG chunks. If I get lucky and come up with something less anecdotal, I will certainly reply here.

    Happy camping ... (or something like that )

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •