in reply to Re^3: Muy Large File
in thread Muy Large File

If you do manage to split this into threads you may actualy reduce performance as each time a different thread gets a shot at it it forces the HD to drag it's heads over to a completely different part of disk. A single thread reading the file sequentialy will not be making the heads seak so much, assuming the file is not desperately fragmented on the media.
That may be true if the file is stored on a single disk. But somehow I doubt an 8-way box dealing with 45-50Gb files uses filesystems that are layed out over single disks. It's far more likely some kind of volume manager (either in software using Solstice Disksuite or Veritas Volumemanager, to name two common products used with Solaris, or in hardware, either by using a RAID card, or by having the RAID done by the backend storage, which in turn, could be done by a specific diskarray, or by the network itself (NAS)). Without knowing more about the volume layout and implementation, it's hard to say how much performance is helped by using separate threads or processes. It becomes unlikely that performance will actually decrease, although bad volume setups happen all the time. Often unknowingly, but also because people want to know the disk a certain file is stored on.

Replies are listed 'Best First'.
Re^5: Muy Large File
by Random_Walk (Prior) on Mar 16, 2005 at 15:21 UTC

    That is why I said may actually reduce performance.

    With a RAID array the parts of a large file are still hopefully stored in reasonable proximity across however many disks they are spread across. If the 50Gb file is spread across 5 disks then when it was written ideally the heads on each of the five disks would have been able to put their (10Gb+parity data) down in one long stripe. Admittedly if the head has to seek from one end to the other now it has a little more than 1/5th of the previous distance but there is still a good chance of a bonus for sequential read.

    Even NAS still has spinning platters and inertialy challenged heads at the coal face.

    Cheers,
    R.

    Pereant, qui ante nos nostra dixerunt!