ext3 errors only when mounted

Just performed an installation. Ran “tune2fs -l” on the file system that mounts at root (/), which is ext3 and it reported a “First orphan inode.” The inode number changed after a few minutes with no other user-space activity.

Booted from DVD and ran “e2fsck -f -C0” on the file system, and it reported no errors.

Booted from the hard drive, and a new “First orphan inode” was reported (which also changed after a few minutes). There were no files in lost+found. Ran “e2fsck -f -C0 -n” on the file system and it reported:

e2fsck 1.41.1 (01-Sep-2008)
Warning! /dev/mapper/pdc_dbddcighd_part7 is mounted.
Warning: skipping journal recovery because doing a read-only filesystem check.
Pass 1: Checking inodes, blocks, and sizes
Deleted inode 1847532 has zero dtime. Fix? no

Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences: -(7381179–7381232)
Fix? no

Free blocks count wrong for group #223 (23960, counted=23959).
Fix? no

Free blocks count wrong (28945097, counted=28945096).
Fix? no

Inode bitmap differences: -1847532
Fix? no

/dev/mapper/pdc_dbddcighd_part7: ********** WARNING: Filesystem still has errors **********

/dev/mapper/pdc_dbddcighd_part7: 139304/7617024 files (0.3% non-contiguous), 1463607/30408704 blocks

Clean restarts (not shutdowns) each time. When I installed the OS (from OpenSUSE-11.1-DVD-x86_64.iso), I edited the default partitioning recommendation by deleting the /home partition (it would not let me reduce the size), enlarging the / partition to grab most of the space freed by deleting the /home partition, and recreating the /home partition using all the remaining space, but otherwise exactly the same parameters it had recommended. It wanted to make the /home partition take up approximately 85% of the hard drive, and make the / partition only around 20 GiB. Since most files I plan to add won’t be in the /home directory (/usr/local/bin/ instead), I figured I needed to do this to have the free space.

Maybe I screwed it up when I partitioned it this way, but I only mention it because I have no idea, not because I have any reason to suspect this was the cause of the errors.

Questions:

  1. What’s going on with the file system checking out okay when unmounted but not when mounted?

  2. Is there any way to safely fix errors that appear only when the system is mounted?

  3. Would the “First orphan inode” changing when the system is idle be attributable to background processes not successfully deleting files (I was in runlevel 5)?

  4. Anything wrong with my logic or actions regarding the partitioning setup?

You cannot expect valid results from doing a fsck on a live, mounted filesystem. That’s why the program wouldn’t let you make any changes to the filesytem.

I reinstalled using the default packages and the default partitioning proposal without a /home partition, and the problem persists.

I reinstalled again using “safe settings” for kernel settings, default packages and the default partitioning proposal with a /home partition. Still have the orphan inodes reported.

Used KDE 4.1 for every installation so far.

Are ext3 root partitions reporting orphan inodes on most installations?

ken_yap:

I knew that it would be bad to “fix” a mounted partition, but is performing a read-only check on a mounted partition useless? Should one expect to see errors reported that are false positives?

You cannot get meaningful results even doing a read-only check on a mounted partition. Since the check takes a finite time to run, the filesystem will change while the check is going on and you get spurious inconsistencies.

You may have a disk with hardware errors, run a SMART test on it using smartmontools.

Checked the disk with “smartctl -a.” Everything looked okay. Reinstalled to a different disk connected to a different SATA port, using all default and recommended settings, with GNOME this time. Still have a “First orphan inode,” same errors reported with the read-only mounted check, and still get a clean report from an unmounted check.

Tune2fs does not report an orphan inode when running it on the unmounted system.

Can someone with a well maintained system check if their ext3 root partition reports a “First orphan inode” when running tune2fs while the partition is mounted? I’m hoping that this is typical.

Are you chasing a red herring perhaps? Do you have operation problems? If none and if the disk always checks clean when unmounted, why do you think there is a problem? Those ext2/3 tools are meant to be run on unmounted disks. Any results from running on a mounted disk are not valid.

No other problems. Not sure if I’m chasing a red herring or not. If I get a couple of replies stating that good file systems give the same result, I’ll probably conclude that I am.

I can see how e2fsck could give false errors because it is scanning and testing a file system while it is changing, but I am assuming from the tune2fs man document and from the speed at which “tune2fs -l” runs (less than a hundredth of a second) that it is not testing anything, but rather reporting what is stored in the superblock (kind of like how smartctl reports the SMART status). So before tune2fs is run, there is a “First orphan inode” stored in the superblock, and tune2fs is reading that information. I’m just hoping to get confirmation that for some reason it should be expected that the superblock should contain a “First orphan inode” when everything is fine. Just seems strange.

No matter how fast a program may seem to you, we are talking about computer processes working with subhuman time intervals. I’d say you’re looking for a problem where there’s none. Nobody expects sane results from running the tools on a mounted filesystem, except in the case of viewing filesystem parameters set at creation time.

At runlevel 2, there is consistently no orphan inode reported, even while copying a bunch of files (made a duplicate of the /lib directory). If I run “/etc/init.d/nscd start” the orphan inode shows up about 20 seconds later. If I stop nscd, the orphan inode disappears about 20 seconds later. Stopping nscd along with all of the init scripts specific to runlevel 3 while in runlevel 3 still reports an orphan inode.

So it seems that the orphan inode is not related to general background file system activity. My best guess is that some files are being deleted while still being used by processes, and that this is probably normal and intentional behavior of those processes, so that the files disappear when they are closed.

I’d sure appreciate if a few people with well-maintained systems could run “tune2fs -l” on a mounted ext3 root partition (must be the root partition) and tell me they’re getting the same thing (a “first orphan inode”) so that I can put my paranoia to rest.

tune2fs -l /dev/sda1


tune2fs 1.41.1 (01-Sep-2008)                  
Filesystem volume name:   <none>              
Last mounted on:          <not available>     
Filesystem UUID:          1527f91a-1414-4e08-a1df-bef8f924eaa8
Filesystem magic number:  0xEF53                              
Filesystem revision #:    1 (dynamic)                         
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file                                                                                                             
Filesystem flags:         signed_directory_hash                                                                  
Default mount options:    (none)                                                                                 
Filesystem state:         clean                                                                                  
Errors behavior:          Remount read-only                                                                      
Filesystem OS type:       Linux                                                                                  
Inode count:              655776                                                                                 
Block count:              2622603                                                                                
Reserved block count:     0                                                                                      
Free blocks:              1012527                                                                                
Free inodes:              419359                                                                                 
First block:              0                                                                                      
Block size:               4096                                                                                   
Fragment size:            4096                                                                                   
Reserved GDT blocks:      640                                                                                    
Blocks per group:         32768                                                                                  
Fragments per group:      32768                                                                                  
Inodes per group:         8096                                                                                   
Inode blocks per group:   506
Filesystem created:       Mon Apr 20 19:12:00 2009
Last mount time:          Thu Apr 30 23:45:24 2009
Last write time:          Thu Apr 30 23:45:24 2009
Mount count:              48
Maximum mount count:      -1
Last checked:             Mon Apr 20 19:12:00 2009
Check interval:           10368000 (4 months)
Next check after:         Tue Aug 18 19:12:00 2009
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
**First orphan inode:       332352**
Default directory hash:   tea
Directory Hash Seed:      d3978195-bbcd-4191-b9d3-e2e21488eee4
Journal backup:           inode blocks