Results 1 to 6 of 6

Thread: Disaster! Entire /var directory disappeared!

  1. #1
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Disaster! Entire /var directory disappeared!


    The entire /var directory of my work system has disappeared.

    I don't know how.

    My backup was done after the disaster, so no /var in there.



    I was working on a test partition, where I was trying 12.1 before upgrading my main partition from
    11.4 to 12.1. As part of the procedure, I was doing a backup with dd for imaging partitions, and
    rsync for files. I think I might have done an rsync line that erased things, but I was trying it on
    an empty directory... the line contained a --del that I later removed. It might be that... :-?


    Code:
    
    > bombadillo:~ # grep -e "--del" .bash_history
    > time nice rsync --archive --acls --xattrs --hard-links --del --stats --human-readable /other/main/usr copia_usr_main
    > time nice rsync --archive --acls --xattrs --hard-links --del --stats --human-readable /other/main/usr copia_usr_main
    > time nice rsync --archive --acls --xattrs --hard-links --del --stats --human-readable  copia_usr_main/usr/local/ /other/main/
    > time nice rsync --archive --acls --xattrs --hard-links --del --stats --human-readable /other/main/opt/ copia_opt_main
    I think it has been the third line. I stopped it because it was supposed to be a quick test of a
    restore and it was taking a long time... and it deleted "/other/main/var" and who knows what more...
    so **** fast!

    I should have mounted the main system RO.

    I may have an older backup. [...] Yep, from June 2011... Better than nothing, I guess. No... it
    would place an antique rpm database, which is what matters for the upgrade procedure. I will have to
    recover the entire 2011 backup for consistency, then upgrade from there.


    As the saying goes, the question is not whether disaster will strike, but when.
    And as the Spanish saying says, the ironmonger is using a wooden spoon.

    :-(

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 12.1 "Asparagus" GM (bombadillo))

  2. #2
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Disaster! Entire /var directory disappeared!

    On 2012-08-11 04:38, Carlos E. R. wrote:
    >
    > The entire /var directory of my work system has disappeared.


    Well, I managed to restore, with a lot of work, the last good backup was too old.


    I found a good tool for undeleting files in ext3/4, but it was not good enough.

    This post in the factory mail list (from Greg Freemyer) was useful:

    Code:
    The developer of ext4magic (a ext4 deleted file recovery tool) has
    created a simple text based menu'ing front-end.  He's used SuseStudio
    to create a simple rescue CD.
    
    You can see some screenshots as well as a link to the downloadable image at:
    http://openfacts2.berlios.de/wikien/index.php/BerliosProject:Ext4magic-e4m-tui
    
    The author  made a post about it here:
    http://developer.berlios.de/forum/forum.php?forum_id=35219
    
    He's looking for feedback if anyone has time to try it out.
    --
    Cheers / Saludos,

    Carlos E. R.
    (from 11.4 x86_64 "Celadon" at Telcontar)

  3. #3
    josephkk NNTP User

    Default Re: Disaster! Entire /var directory disappeared!

    On Sun, 12 Aug 2012 22:23:06 GMT, "Carlos E. R."
    <robin_listas@no-mx.forums.opensuse.org> wrote:

    >On 2012-08-11 04:38, Carlos E. R. wrote:
    >>
    >> The entire /var directory of my work system has disappeared.

    >
    >Well, I managed to restore, with a lot of work, the last good backup wastoo old.
    >
    >
    >I found a good tool for undeleting files in ext3/4, but it was not good enough.
    >
    >This post in the factory mail list (from Greg Freemyer) was useful:
    >
    >
    Code:
    >The developer of ext4magic (a ext4 deleted file recovery tool) has
    >created a simple text based menu'ing front-end.  He's used SuseStudio
    >to create a simple rescue CD.
    >
    >You can see some screenshots as well as a link to the downloadable imageat:
    >http://openfacts2.berlios.de/wikien/index.php/BerliosProject:Ext4magic-e4m-tui
    >
    >The author  made a post about it here:
    >http://developer.berlios.de/forum/forum.php?forum_id=35219
    >
    >He's looking for feedback if anyone has time to try it out.
    >


    Please consider that the entire /var tree is very temporal and normally
    transient in existence. And yes it gets abused a LOT. Salt shaker
    optional.

    ?-)

  4. #4
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Disaster! Entire /var directory disappeared!

    On 2012-08-13 08:53, josephkk wrote:

    > Please consider that the entire /var tree is very temporal and normally
    > transient in existence. And yes it gets abused a LOT. Salt shaker
    > optional.
    >
    > ?-)


    It holds several crucial databases, like for example, the rpm database. Without that YaST
    doesn't know what is installed. I also lost my mysql database, the hylafax database... I could
    have lost the entire nntp cache, except that it resides on a different partition and was safe.

    But also several directories in /etc and /root were lost. Files everywhere were lost, but only
    /var was evident.

    System would not boot.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 11.4 x86_64 "Celadon" at Telcontar)

  5. #5
    josephkk NNTP User

    Default Re: Disaster! Entire /var directory disappeared!

    On Mon, 13 Aug 2012 16:48:06 GMT, "Carlos E. R."
    <robin_listas@no-mx.forums.opensuse.org> wrote:

    >On 2012-08-13 08:53, josephkk wrote:
    >
    >> Please consider that the entire /var tree is very temporal and normally
    >> transient in existence. And yes it gets abused a LOT. Salt shaker
    >> optional.
    >>
    >> ?-)

    >
    >It holds several crucial databases, like for example, the rpm database. Without that YaST
    >doesn't know what is installed. I also lost my mysql database, the hylafax database... I could
    >have lost the entire nntp cache, except that it resides on a different partition and was safe.
    >
    >But also several directories in /etc and /root were lost. Files everywhere were lost, but only
    >/var was evident.
    >
    >System would not boot.


    Big time ouch. Was fsck run at that point?

    ??-)

  6. #6
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: Disaster! Entire /var directory disappeared!

    On 2012-08-14 02:54, josephkk wrote:
    > On Mon, 13 Aug 2012 16:48:06 GMT, "Carlos E. R." <> wrote:


    >> It holds several crucial databases, like for example, the rpm database. Without that YaST
    >> doesn't know what is installed. I also lost my mysql database, the hylafax database... I could
    >> have lost the entire nntp cache, except that it resides on a different partition and was safe.
    >>
    >> But also several directories in /etc and /root were lost. Files everywhere were lost, but only
    >> /var was evident.
    >>
    >> System would not boot.

    >
    > Big time ouch. Was fsck run at that point?
    >
    > ??-)


    Let me see...

    I was working on 12.1 on an auxiliary partition. With that one running I mounted the main
    partition(s), which is 11.4. I was attempting to do a backup, and when testing recovery I
    destroyed part of /root (I'm keeping the description simple). /usr and /home and /boot were
    safe because they were not mounted. Without knowing what I had done, I rebooted to the main
    system, and it failed. I'm not sure if an fsck did run on it or not, I don't remember now. I
    think not.

    The destroying part is because I attempted an "rsync ... -del backup_usr /main_root".
    Everything that was not in the backup of /usr was going to be deleted in the real root
    partition - what I was attempting was to check the command to create a copy of /usr. Be careful
    with "--del" in rsync.


    --
    Cheers / Saludos,

    Carlos E. R.
    (from 11.4 x86_64 "Celadon" at Telcontar)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •