The easiest way would be to process more than one file, by launching your program several times, for example through runN.

Other than that, most likely, the processing your program does within the loop is likely dwarfed by the time needed to read the next line. Adding more threads there won't speed it up.

Maybe you can reduce the time needed for IO by compressing your input files and reading them through a gzip (or bzip2) pipe:

open my $fh, "gzip -cd '$file' |"
    or die "Couldn't read '$file': $!";

while (<$fh>) {
    ...
};

[download]

This shifts the time to read the data from "processing" to when the data arrives. Most likely, you won't gain much unless you need to read the data more than once. Then, maybe also creating an index file or storing the data in a database will speed things up.

Update: Fixed link to runN

In reply to Re: threading a perl script by Corion
in thread threading a perl script by Boetsie

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.