Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: Backup ideas that can detect missing files?

  1. #1

    Default Backup ideas that can detect missing files?

    Ok, this is the delima I've thought about for years. I have about 10 years of home videos and pictures on my computer that I can't loose. I run Crashplan to back up my files. But, I have hundreds of folders holding pictures. If a folder all the sudden is empty or the pictures become corrupt, how will I know? I don't check pictures from 10 years ago often. If they become missing or corrupt, I may not know for months or years. By then my backups are long gone. It would be nice to find a backup that would backup all my files, and if it detected any missing files, it should alert me.. send me a report of what files were there yesterday but are missing today. Does such a program exist?

    (I am now more worried because today I noticed a lot of folders that contained pictures from about ten years ago I found empty today. They may have been empty for the last year.. I don't look at the pictures daily. All I know is I checked in my backups and, as of 2 months ago, they were not there according to my backups. I have one other hope I can get them back when I get home. But I need to guarantee my home videos and pictures will be available 20 to 40 years from now. )

  2. #2
    Join Date
    Nov 2009
    Location
    West Virginia Sector 13
    Posts
    15,697

    Default Re: Backup ideas that can detect missing files?

    How are you doing the backup? Done correctly older files should not be erased just because the source files are missing.

    Simply copying files to the backup media should preserve all files. If you do some sort of partition imaging over writing the backup then if the files go missing in the source then they go missing on the image

  3. #3

    Default Re: Backup ideas that can detect missing files?

    Quote Originally Posted by gogalthorp View Post
    How are you doing the backup? Done correctly older files should not be erased just because the source files are missing.

    Simply copying files to the backup media should preserve all files. If you do some sort of partition imaging over writing the backup then if the files go missing in the source then they go missing on the image
    I currently am using Crashplan to backup. Problem is a few weeks ago, I formatted and reinstalled my OS and restarted the backups. So, I can't restore anything before that restore of my OS. If I was somehow receiving reports of filesystem deletions on the folder structures I'm backing up, it would've prevented these folders disappearing.

  4. #4
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,109
    Blog Entries
    2

    Default Re: Backup ideas that can detect missing files?

    You should consider backup to optical media, specifically BR or DVD (not to CD),
    Then storing that data disk in a controlled environment, eg air tight and maybe even in nitrogen or vacuum with zero humidity and away from light and heat.
    Done properly, it should be good indefinitely and not subject to erosion.

    And as always develop a redundancy of three...
    ie
    At least 3 copies
    At least 3 different ways of making copies
    Stored in at least 3 geographical separate locations
    etc.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  5. #5

    Default Re: Backup ideas that can detect missing files?

    The other way I'm hoping I can get the pictures back is a year or two back, I copied all my home videos and pictures to DVD and I'm 99% sure they're sitting in a box in my closet. When I get home tonight, I'm going to look through those to see if I can find the pics for the empty folders. The pics that are missing are pics from about 10 years ago, so my 1-2 year old DVD backups should have them. I need to stay on top of it though and make more regular DVD backups.

  6. #6
    Join Date
    Jan 2018
    Location
    Canada
    Posts
    122

    Default Re: Backup ideas that can detect missing files?

    For long term storage I use Amazon Glacier and CrossFTP for management. It’s currently $0.004 per GB per month so I pay $1.08US per month for 271 GB of storage.

    The CrossFTP program can be used to synchronize in either direction, and shows what will be copied or deleted. I don’t remember the cost, but it wasn’t much and updates are free.

  7. #7
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    11,109
    Blog Entries
    2

    Default Re: Backup ideas that can detect missing files?

    Quote Originally Posted by doscott View Post
    For long term storage I use Amazon Glacier and CrossFTP for management. It’s currently $0.004 per GB per month so I pay $1.08US per month for 271 GB of storage.

    The CrossFTP program can be used to synchronize in either direction, and shows what will be copied or deleted. I don’t remember the cost, but it wasn’t much and updates are free.
    Just a FYI for anyone considering Glacier (It's under consideration for a project I'm working on),
    It's incredibly cheap to upload and store long term, but be prepared for a massive cost if you ever want to retrieve more than 5% of your files in a given month.
    Plus, files should be stored for over 90 days before you attempt to retrieve.

    More restrictions in the fine print which is extensive.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  8. #8
    Join Date
    Jan 2018
    Location
    Canada
    Posts
    122

    Default Re: Backup ideas that can detect missing files?

    You are correct but for long term storage the need to retrieve everything in a hurry would be unusual.

    But even bulk retrieval at a low rate is pretty cheap.

    https://aws.amazon.com/glacier/faqs/

  9. #9

    Default Re: Backup ideas that can detect missing files?

    So, I did find a lot of pictures last night from my backup DVDs I kept in my closet. Still, the part of the question I'm way more interested is not so much how to back up, but maybe a script that could detect a major change in my files and warn me that files might be missing so I don't go months or years before noticing a ton of files just vanished.

  10. #10
    Join Date
    Jan 2018
    Location
    Canada
    Posts
    122

    Default Re: Backup ideas that can detect missing files?

    Personally I backup everything to a local NAS using rsync via a cron script, with 14 days backup. The log files are mailed to me and I examine them daily to see if anything unexpected has happened. This is my first line of defence.

    I also do a daily backup to Amazon S3 using the s3cmd sync utility, via a cron script, for anything that I deem not to be long term storage, with a 2 month retention policy on changed files. Again, the log files are emailed to me so I can check for surprises.

    As mentioned above I use Amazon Glacier for long term storage, things like photos, receipts, etc, that I manually sync to avoid accidentally deleting something that has disappeared locally.

    Almost anything is doable, but exact help will depend on what cloud you are using and what software you are using.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •