Continuous write access to external hard drive

Hi all,

I just got a new external hard drive (Seagate Expansion Desktop 3 TB) which I set up with LUKS (using cryptsetup) and formatted as ext4. Although the drive is empty and not in use at all, I see the hard disk activity light flashing constantly (USB 3.0) or every 2-3 seconds (USB 2.0) whenever the drive is mounted. Unmounting the drive, but not closing the LUKS setup with (i.e., without):

cryptsetyp luksClose /dev/mapper/encrypteddrive

makes the writes stop. (I also tried formatting as ext3 but had the same problem.) Using

tune2fs -l

I confirmed that there are actual writes going on whenever the disk (i.e., the encrypted LUKS partition) is mounted.

I spent all night yesterday researching the issue, and although a few (but not a lot) similar problems have been reported (see e.g. [1]), there appears to be no solution - only workarounds that I cannot use in my particular situation (e.g. increasing the ‘commit=’ interval). (For those following the link: although it has been suggested that this is due to journaling, disabling the journal does not make a difference. Also it appears that journal updates are only a symptom, but not the cause for this problem. There is something else that causes the writes in the first place.)

I am completely lost here and all clues would be very much appreciated. I would like to have the drive hooked up permanently and have it spin down after a few minutes of inactivity, but this won’t work when these writes keep it busy.

Tested with:

  • OpenSUSE 13.1, 64-bit, kernel 3.11.6-4
  • Raspbian Wheezy, ARM, kernel 3.10.23+ (on a Raspberry Pi)

[1] https://bbs.archlinux.org/viewtopic.php?pid=691839

On 2013-12-15 12:06, john jtvds wrote:
>
> Hi all,

> makes the writes stop. (I also tried formatting as ext3 but had the
> same problem.)

You could try a totally different filesystem; I’d suggest XFS. That way
we’d know if it is related to extX code or not.


Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

Unfortunately OpenSUSE wouldn’t let me create an XFS file system, so I went ahead and tried ext2 (which has no journal) and vfat:

  • ext2 shows the same behavior: continuous writes
  • vfat, although returning an error regarding missing geometry upon creation (but it would still let me mount & write), did not cause random writes

What might also be relevant: I have a second drive of the exact same type (though a year older), which I mount as a TrueCrypt volume and which uses ext3 or ext4, where I do not experience this problem.

On 2013-12-15 16:16, john jtvds wrote:
>
> Unfortunately OpenSUSE wouldn’t let me create an XFS file system,

Humbug!

You need to install the xfs parts.

xfs, xfsprogs…


Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

Good point! (Why didn’t I think of that myself?! :beat-up:)

And speaking of good…good news everyone! With XFS it seems to be working properly: no more unaccounted for disk access that keeps the drive awake, which means spin-downs according to what is specified through ‘hdparm -S’!

On 2013-12-15 21:06, john jtvds wrote:
>
> Good point! (Why didn’t I think of that myself?! :beat-up:)

It happens :slight_smile:

Rather, it can be considered a bug that the YaST partitioner module,
when asked to create XFS partitions, does not install them itself, as
other modules do.

> And speaking of good…good news everyone! With XFS it seems to be
> working properly: no more unaccounted for disk access that keeps the
> drive awake, which means spin-downs according to what is specified
> through ‘hdparm -S’!

Aha!

So we know that the problem is in the ext2/3/4 code. You can create a
bugzilla with that info. And keep using XFS, it is a wonderful storage
for big partitions and files. I use XFS on home, and ext3/4 on root.

openSUSE:Submitting bug
reports

And mention the bug number here, so that people can track it too.


Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

Thanks for your help! I’ll file a bug report later and report its number here.

And also I will start using the XFS setup - it’s always nice to try new things :slight_smile:

I was going to file a bug report, when I stumbled onto this:

(…) the inode table will not be fully initialized by mke2fs. This speeds up filesystem initialization noticeably, but it requires the kernel to finish initializing the filesystem in the background when the filesystem is first mounted. (…)

(source: linux - is there such a thing as a ‘quick’ format for ext4? - Server Fault)

It sounds probable that this caused the writes I noticed, although if true, I would be surprised to learn that creating the file system causes 3 GB of data to be written (to a 3 TB disk).

File structures take up space 10% is not totally out of line.

On 2013-12-21 22:06, john jtvds wrote:
>
> I was going to file a bug report, when I stumbled onto this:
>
>> (…) the inode table will not be fully initialized by mke2fs. This
>> speeds up filesystem initialization noticeably, but it requires the
>> kernel to finish initializing the filesystem in the background when the
>> filesystem is first mounted. (…)
>
> (source: ‘linux - is there such a thing as a ‘quick’ format for ext4? -
> Server Fault’ (http://tinyurl.com/n7tafyv))

Wow.

One never stops learning…

> It sounds probable that this caused the writes I noticed, although if
> true, I would be surprised to learn that creating the file system causes
> 3 GB of data to be written (to a 3 TB disk).

Could be, yes…


Cheers / Saludos,

Carlos E. R.
(from 13.1 x86_64 “Bottle” (Elessar))