I measure the block read time at file read at large files.
The result is at read of first block the reader process sometimes stall for 2-60sec. This mean to get the first block of the file need sometimes 60 sec. This cause sometimes various timeouts at reader process.
Looks like the cause of this problem is the kernel? or fs? after open try to cache mouch as possible file data. If the hardware has enough ram then cache 3GB of the file content. At cache fill no data passed to requester process. This is why stall many times for long time.
So how it possible to limit the disk cache for specific shell. The limitation of resident size is not too helpfull because most of our applications use more GB ram as resident memory.
I recognize this problem at 10.3, 11.3, no raid, raid 0, raid 1 configuration.
Warning! Reproduce this problem is tricky. With small file you never see this problem. Need to generate two large (I use 3GB) file in a same shell and need read to alterante (eg. from file A then B then A …).