For a simple search like this where the processing involved for each line is a single, simple 10-char string comparison (which takes ~2.5e-7 seconds on my 2GHz processor using perl), whereas reading that same 10-char line from a file takes ~2.5e-6, the process is going to be waiting on IO for ~90% of it's time, so the runtime is controlled entirely by the time taken by the kernel to do IO.

Whether 1 thread reading from 1 file, or 4 threads reading from 4 files, all the disk access is going to be serialised through the device driver anyway.

The only way I've succeeded in deriving any performance benefit from multi-threading a perl process doing IO, is when the processing of the lines can be overlapped with the IO waits. And that was only possible where the processing time was greater than the IO time--complex parsing for example.

In that case, it is possible to have a single thread reading to shared (*but non-cloned*), buffers, and then have 2 or 3 threads reading from the buffers and doing the parsing. Even then, ensuring that the buffers do not get overwritten before the've been processed without ham-stringing the reading thread with locking is a fine tuned balancing act.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

In reply to Re^3: threads and multiple filehandles by BrowserUk
in thread threads and multiple filehandles by zentara

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.