in reply to Re^3: Threads Doubt
in thread Threads Doubt

No offense intended, BrowserUK. My (rather roundabout) point was that even putting files on different disks might not make a difference.

You are right to raise warning flags concerning the urge to multi-thread, for many reasons. I humbly submit that, given large reads and non-trivial data processing, it is possible to design a 2-thread solution that is likely to improve performance (1 thread reading a 10MB chunk while the other is processing its 10MB chunk from memory).

Replies are listed 'Best First'.
Re^5: Threads Doubt
by BrowserUk (Patriarch) on Oct 17, 2008 at 18:27 UTC
    No offense intended,

    None taken :)

    given large reads and non-trivial data processing, it is possible to design a 2-thread solution that is likely to improve performance

    I've had several attempt at this using Perl and found that unless the processing is really quite involved, the cost of iThreads scalar sharing negates any advantage. I also found it actually works best using smaller buffers: 64kb to 1MB.

    I've had far greater success using C and threads where memory is shared directly without the threads::shared overheads, but once you drop into C, using memory mapped files or overlapped IO is far more productive.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom