file read magic

I measure the block read time at file read at large files.
The result is at read of first block the reader process sometimes stall for 2-60sec. This mean to get the first block of the file need sometimes 60 sec. This cause sometimes various timeouts at reader process.
Looks like the cause of this problem is the kernel? or fs? after open try to cache mouch as possible file data. If the hardware has enough ram then cache 3GB of the file content. At cache fill no data passed to requester process. This is why stall many times for long time.

So how it possible to limit the disk cache for specific shell. The limitation of resident size is not too helpfull because most of our applications use more GB ram as resident memory.

I recognize this problem at 10.3, 11.3, no raid, raid 0, raid 1 configuration.

Warning! Reproduce this problem is tricky. With small file you never see this problem. Need to generate two large (I use 3GB) file in a same shell and need read to alterante (eg. from file A then B then A …).

Sorry that was my fault.
The cache are done at the implemeting languag library level not at kernel level.

On 2011-12-04 23:36, kzsolt2 wrote:
>
> Sorry that was my fault.
> The cache are done at the implemeting languag library level not at
> kernel level.

I was going to say that your problem is too specific, you have to give a
sample code producing the problem, before anybody can try to say anything.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

“I was going to say that your problem is too specific,”
Yes sorry again. If we discover this is not at kernel level beacuse at implemeting languag library level then we found the solution within 1 min.