See, it made you think :). Your threaded solution assigns a file to a thread to process, so what if one of the files is much larger than the sum of the rest of the files. The whole execution time will be determined by that one big file.

Now, where did you wander off thinking of bin packing. It is nice that you posses such knowledge but it seems to be getting in the way a bit. How about treating the whole set of files as a single file and piping line by line to each of the work nodes in turn? Would that not divide the load equally?

Your suggestion to order the files by sizes when using threads is certainly an improvement that has the advantage of keeping the size and complexity of the code low but it is still sub-optimal.

The main reason I stick with my approach is because it sort of forces you to think about the design of the solution and I mean this in more general terms. Still, your solution is elegant as it can get for this particular problem.

In reply to Re^7: Converting a parallel-serial shell script by kubrat
in thread Converting a parallel-serial shell script by Corion

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.