Files disappearing

English Hardware
#1

Previously I posted a thread because I thought my hard drive was bad. For now I’m assuming the drive is ok. Today I deleted my mp3 folders, and restored them from DVD backups. I made the folders read-only hoping they wouldn’t mistakenly be deleted. Already I found that in one sub-folder, 3 “sub-sub-folders” (different CD rips) where in the “main” folder, not in the sub-folder for a particular artist. And one “sub-sub-folder” was missing. And one song in one of the misplaced “sub-sub-folders” was missing. I checked the backup DVD again, all the files were hwere they belonged. All I can do is scratch my head. I can assume there are more files in the wrong place or missing. The last time I noticed this problem, one mp3 file had become a folder. This time I haven’t used any mp3 editing programs, nothing. Just played some mp3s from playlists using Audacious. So I’m certain I didn’t cause the problem.

I tried this again:


smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.28-desktop] (SUSE RPM)
Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda Green (AF)
Device Model:     ST2000DL003-9VT166
Serial Number:    5YD6GFWT
LU WWN Device Id: 5 000c50 0462b7d6f
Firmware Version: CC3C
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5900 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Tue Mar 25 22:16:40 2014 EDT

==> WARNING: A firmware update for this drive may be available,
see the following Seagate web pages:
http://knowledge.seagate.com/articles/en_US/FAQ/207931en
http://knowledge.seagate.com/articles/en_US/FAQ/218171en

SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
See vendor-specific Attribute list for marginal Attributes.

General SMART Values:
Offline data collection status:  (0x82)    Offline data collection activity
                    was completed without error.
                    Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:         (  623) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:      (   1) minutes.
Extended self-test routine
recommended polling time:      ( 348) minutes.
Conveyance self-test routine
recommended polling time:      (   2) minutes.
SCT capabilities:            (0x30b7)    SCT Status supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   110   099   006    Pre-fail  Always       -       27818600
  3 Spin_Up_Time            0x0003   092   092   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   099   099   020    Old_age   Always       -       1035
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   039   030   030    Pre-fail  Always   In_the_past 23798461380823
  9 Power_On_Hours          0x0032   091   091   000    Old_age   Always       -       8199
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   099   099   020    Old_age   Always       -       1039
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   097   000    Old_age   Always       -       4295032959
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   069   061   045    Old_age   Always       -       31 (Min/Max 19/32)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       24
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       1038
194 Temperature_Celsius     0x0022   031   040   000    Old_age   Always       -       31 (0 16 0 0 0)
195 Hardware_ECC_Recovered  0x001a   028   013   000    Old_age   Always       -       27818600
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       92071213932606
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       2300893613
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       1989630910

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%       356         -
# 2  Short offline       Aborted by host               90%       356         -
# 3  Short offline       Completed without error       00%       356         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

In a previous thread it was suggested there was no problem with the drive, probably a filesystem problem (if I remember correctly). I have these files in a data partition I added during installation of openSuse 12.3. After install, this was mounted to “/data” just like I set it up for, and had root permissions. I used the superuser version of Dolphin to get the properties of /data, and set the permissions to “users”, and allowed read and write for all the contents. Should I have done that some other way? If I decide to try this without this partition, and want to delete the partition, and use the left over space for my “home” partiton, is there a way to do that? Maybe GParted Live CD can do that?

I’ve never seen this happen before I tried a separate data partiton, so these are all just guesses. I have no clue what could cause this. Unless the drive really is bad… any ideas?

#2

I just restored one of my mp3 backup DVDs to my “home” partition, and all the files are there. In my “data” partition, all but 4 or 5 of the subfolders are missing, as are all of the .m3u playlists! Weird! I’ll check it gain tomorrow, see if they’re still there. So now I’m suspecting my “data” partition. It should work, right? Maybe I set it up incorrectly? Or a bad drive? I’m stumped… using ext4, by the way, if that’s important.

#3

In all honesty, I am suspecting user error. But I don’t have enough information to really be able to tell.

#4

That’s what I am suspecting, too :slight_smile:

Is there any particular information I could post that might help?

If not, is it possible to delete that partition, then resize the home partition to use the left over drive space? Without wiping out “home”? I’d like to try that instead of a full re-install. The data partition is a little over 1GiB, I’d like to recover that to add to “home”. I don’t mind going back to the default partitoning that openSuse uses. As long as it works again. I never had user errors like this back then. I’ve outdone myself this time!

#5

I ran:

mount | column -t

And these are my partitions listed - sda4 is the one that I added during installation. Maybe this info is a clue to someone…


/dev/sda2   on  /                             type  ext4             (rw,relatime,data=ordered)
/dev/sda4   on  /data                         type  ext4             (rw,nosuid,nodev,noexec,relatime,data=ordered)
/dev/sda3   on  /home                         type  ext4             (rw,relatime,data=ordered)

The complete output is this, if anyone needs it:


devtmpfs    on  /dev                          type  devtmpfs         (rw,relatime,size=3916508k,nr_inodes=979127,mode=755)
tmpfs       on  /dev/shm                      type  tmpfs            (rw,relatime)
tmpfs       on  /run                          type  tmpfs            (rw,nosuid,nodev,relatime,mode=755)
devpts      on  /dev/pts                      type  devpts           (rw,relatime,gid=5,mode=620,ptmxmode=000)
/dev/sda2   on  /                             type  ext4             (rw,relatime,data=ordered)
proc        on  /proc                         type  proc             (rw,relatime)
sysfs       on  /sys                          type  sysfs            (rw,relatime)
securityfs  on  /sys/kernel/security          type  securityfs       (rw,nosuid,nodev,noexec,relatime)
tmpfs       on  /sys/fs/cgroup                type  tmpfs            (rw,nosuid,nodev,noexec,mode=755)
cgroup      on  /sys/fs/cgroup/systemd        type  cgroup           (rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup      on  /sys/fs/cgroup/cpuset         type  cgroup           (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup      on  /sys/fs/cgroup/cpu,cpuacct    type  cgroup           (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup      on  /sys/fs/cgroup/memory         type  cgroup           (rw,nosuid,nodev,noexec,relatime,memory)
cgroup      on  /sys/fs/cgroup/devices        type  cgroup           (rw,nosuid,nodev,noexec,relatime,devices)
cgroup      on  /sys/fs/cgroup/freezer        type  cgroup           (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup      on  /sys/fs/cgroup/net_cls        type  cgroup           (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup      on  /sys/fs/cgroup/blkio          type  cgroup           (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup      on  /sys/fs/cgroup/perf_event     type  cgroup           (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup      on  /sys/fs/cgroup/hugetlb        type  cgroup           (rw,nosuid,nodev,noexec,relatime,hugetlb)
systemd-1   on  /proc/sys/fs/binfmt_misc      type  autofs           (rw,relatime,fd=25,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
mqueue      on  /dev/mqueue                   type  mqueue           (rw,relatime)
hugetlbfs   on  /dev/hugepages                type  hugetlbfs        (rw,relatime)
tmpfs       on  /var/lock                     type  tmpfs            (rw,nosuid,nodev,relatime,mode=755)
debugfs     on  /sys/kernel/debug             type  debugfs          (rw,relatime)
tmpfs       on  /var/run                      type  tmpfs            (rw,nosuid,nodev,relatime,mode=755)
/dev/sda4   on  /data                         type  ext4             (rw,nosuid,nodev,noexec,relatime,data=ordered)
/dev/sda3   on  /home                         type  ext4             (rw,relatime,data=ordered)
fusectl     on  /sys/fs/fuse/connections      type  fusectl          (rw,relatime)
gvfsd-fuse  on  /run/user/1000/gvfs           type  fuse.gvfsd-fuse  (rw,nosuid,nodev,relatime,user_id=1000,group_id=100)
gvfsd-fuse  on  /var/run/user/1000/gvfs       type  fuse.gvfsd-fuse  (rw,nosuid,nodev,relatime,user_id=1000,group_id=100)
/dev/sr0    on  /run/media/user/mp3-set3      type  iso9660          (ro,nosuid,nodev,relatime,uid=1000,gid=100,iocharset=utf8,mode=0400,dmode=0500,uhelper=udisks2)
/dev/sr0    on  /var/run/media/user/mp3-set3  type  iso9660          (ro,nosuid,nodev,relatime,uid=1000,gid=100,iocharset=utf8,mode=0400,dmode=0500)
#6

I’m not sure what you are doing, and that makes it hard.

You might see if the missing files are somewhere else on your system. Perhaps some command is automatically putting them somewhere different from what you expected.

I’m not sure. You might be able to do that booted to a “gparted” live CD. But it is risky, and could lose data. It’s safer to backup, reformat, restore. You should not need to reinstall if you are only changing “/home” and “/data”.

If you can find the files elsewhere, maybe the careful use of symlinks will work to avoid having to change your partitioning.

#7

I used Dolphin to copy my mp3 backups (one directory and lots of subdirectories for each disc) from 4 DVDs to the directory “/data”, the partition I added at install time. I’ve never seen this happen with Dolphin before, not so sure it’s that. Right after that, files were missing or in the wrong place. They should have just copied to the same directories that are on the DVDs. I copied one backup DVD to my home partition, that worked fine. No lost files. That’s why I suspect that I did something wrong setting up that data partition.

#8

What would I need to back up in “/home” besides any of my files? Would that include all those hidden files? I have all the stuff in “/data” backed up already. This sounds like it’s worth a try…

#9

http://kde-look.org/CONTENT/content-pre1/164331-1.jpg

How I have permissions set for the data partition - could these choices cause the trouble I’m having?

#10

Please spare yourself the trouble of creating those pictures. When you want to know about the properties of /data, then do

ls -l /data

That is a language we all understand.

BTW, I did not read all of the above, but like nrickert I am afraid you do something strange that we can not see (allways difficult with this way of communication, but we can not visit you and see what you are doing). When you say you have backups and restore from there, without further detail, one can make many errors. There are many, very many ways to make “backups” (which is a generic term anyway). So more detail may be needed there about backup/restore tools, etc.

#11

Ok, so here’s ls -l /data

total 192
drwxrwxrwx  3 user users  4096 Jan 22  2012 AudioCaptures2-HeavyProgressive
drwxrwxrwx  2 user users  4096 Mar 13  2012 AudioCaptures-MixedProgressive
drwxrwxrwx  2 user users  4096 Mar 13  2012 AudioCaptures-mp3
drwxrwxrwx  8 user users  4096 Mar  9 03:29 BassGuitarTutorialStuff
drwxrwxrwx 19 user users 12288 Feb 22 02:46 Documents
drwxrwxrwx  9 user users  4096 Mar 25 19:29 DVD
drwxrwxrwx  7 user users  4096 Mar 22 19:27 dwhelper5
drwxrwxrwx  4 user users  4096 Mar  9 13:51 dwhelper5b-Full-BackedUp
drwxrwxrwx  3 user users  4096 Jan 26  2013 FREESOUND_ORG
drwxrwxrwx 55 user users  4096 Oct  1  2012 Gif
drwxrwxrwx  2 user users  4096 Mar 24 23:13 HomeVideo-HighDef
drwxrwxrwx  2 user users 16384 Jan  2 15:00 lost+found
drwxrwxrwx  7 user users  4096 Feb 18 21:49 MasBackups-Feb18-2014
drwxrwxrwx 12 user users  4096 Dec 31 02:38 MIDI-files
drwxrwxrwx  2 user users  4096 Jan 19 00:32 MKV
drwxrwxrwx  9 user users  4096 Mar 24 23:53 mp3-current
drwxrwxrwx 10 user users  4096 Mar 11 18:08 MP3-DownloadedFromAmazonDotCom-99CentsEach
drwxrwxrwx 16 user users  4096 Mar 22 19:37 Mp3-ForBassPractice
drwxrwxrwx 30 user users  4096 Jan  4 15:31 mp3-OldStuff-Web
drwxrwxrwx 44 user users  4096 Aug 23  2010 mp3-set1
drwxrwxrwx 29 user users  4096 Aug 23  2010 mp3-set2
drwxrwxrwx 29 user users  4096 Mar 25 21:58 mp3-set3
drwxrwxrwx  7 user users  4096 Jul 12  2013 mp3-set4
drwxrwxrwx  3 user users  4096 Oct 17 23:12 my-pcx
drwxrwxrwx  2 user users  4096 Mar 22 19:58 my-shell-scripts
-rw-rw-rw-  1 user users   184 Mar 22 20:02 NOTICE
drwxrwxrwx  7 user users 28672 Mar  5 15:14 PDF
-rw-rw-rw-  1 user users  3172 Jan 13 14:50 ProgressiveRock-MP3.m3u
drwxrwxrwx  2 user users  4096 Feb  7 13:48 SavedEmails
drwxrwxrwx  9 user users  4096 Feb  6 18:32 ScottsBassLessons-LessonModules
drwxrwxrwx 51 user users  4096 Mar 25 03:50 TuxGuitar
drwxrwxrwx  7 user users  4096 Jan 26 15:28 Videos
drwxrwxrwx  3 user users  4096 Mar 14 17:22 Wallpapers-1920x1080
drwxrwxrwx  2 user users  4096 Jan 18 23:14 Wallpapers-ForSlideshow-1
drwxrwxrwx  4 user users 12288 Dec  7  2012 WAVfiles

The “mp3-set1” through “mp3-set4” directories are where I noticed the missing files. I haven’t checked anywhere else on that partition yet. Not remembering how many files I put on there, I might not even know if any are missing. Playing mp3s from a playlist is noticeable, because Audacious will tell me there’s a problem, and no play time is shown for the missing songs.

As for the 'backups", I used K3B to burn directories of files onto DVD+R discs. To restore, I copied them to the hard drive. I didn’t use any backup software - sorry about the confusion. By the way, these “backups” were done before I installed openSuse 12.3, they’re not from the new installation. Lots of mp3 files, home videos, etc. Never had problems restoring (copying back to the hard drive) before, but I always just had one large home partition, the default partitioning you get when installing.

One of my concerns was that after changing the permissions of /data so it was “owned” (if that’s the right term) by “users”, later it was set for “root” again. I know I didn’t set it back that way. So I set it back to “users” again. Today it’s still “users”. I’ve never tried adding a partition before, was just trying to figure out if I did it wrong. It’s not a big deal, I don’t mind just wiping the drive and reinstalling openSuse, and not messing with the partitioning.

#12

Have you run chkfs?

Dolphin has known to sometime miss files when copying mass amounts are you sure that the files are there after the restore?

BTW I don’t think you said which file system you use.

#13

I don’t see chkfs in Yast, and it doesn’t seem to exist. Tried running as root, still not there. I’m using ext4…
As for Dolphin, this is news to me! If never had it screw up copying back all my mp3s. It worked flawlessly copying to my home partition just yesterday. I only get this problem in the added data partition. Files are definitely missing. Not all of them, but quite a few.

#14

Opps brain fart lol

it is fsck lol

You need to run it from out side live since it won’t work entirely if the partition being check is mounted

#15

I checked the data partition using

fsck /dev/sda4

It says “clean”

#16

I checked the boot log, this line shows it’s clean too

systemd-fsck[496]: /dev/sda4: clean, 20589/84557824 files, 59048206/338226432 blocks

The other partitions are shown as “clean” also.

#17

Ok, much embarrasment now!

I was playing around with different things, and decided to copy back my mp3s from DVD+r discs - again. On the last disc, Dolphin couldn’t copy some files. So I tried playing the associated playlist from the DVD+R, and Audacious shows some ridiculously long play time for the somgs, and all the same time at that. Of course, they don’t play. So, I tried the disc on my mother’s Win7 PC, and guess what - it works fine. Dammit! It’s obviously my DVD burner dying. How embarrasing! So, my backups are ok, but the burner has been corrupting or losing them while copying to the hard drive. So at least now I can rule out the hard drive. Sheesh! Ok, so now I’m ordered a new burner, at least those are dirt cheap. Sorry for wasting everyone’s time!

#18

On 2014-03-26 03:36, HighBloodSugar wrote:
>
> Previously I posted a thread because I thought my hard drive was bad.
…
> I tried this again:
…


>   smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.28-desktop] (SUSE RPM)

>   9 Power_On_Hours          0x0032   091   091   000    Old_age   Always       -       8199
....
>   SMART Self-test log structure revision number 1
>   Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
>   # 1  Short offline       Completed without error       00%       356         -

On the previous thread I suggested you run the short and long smart
tests - you have not done it. I can see above you have not tested the disk.

–
Cheers / Saludos,

Carlos E. R.
(from 13.1 x86_64 “Bottle” at Telcontar)

#19

If this is a double post, forgive me - I thought I replied to this a minute ago, but I don’t see it… getting very tired tonight… Anyway, I ran the “SeaTools” bootable CD, ran the short and long tests, it passed both. I have no floppy drive, so I couldn’t save the log to post here.

#20

You were not “wasting” our time. The fact that you discuss with us and we give suggestions keep you going in walking new pathes in debugging. Else you might have given up. No, when others check your way of thought and help you thinking, that may lead to results (as here) even if nobody here came up with the solution in a direct way. That is how our community should function.