The best practice is to not recompress. As someone else mentioned, your quickest / easiest / cheapest fix is to not alter your machine or your Perl environment, but your algorithm. Re-compressing your file is an unnecessary waste. At worst, uncompress a COPY of the compressed file, process it, then throw it away. Do it in /tmp, or not in /tmp, whatever works best for you. Just don't recompress everything.

After that, you must know your system bottleneck before you can decide what else to change. Threading may or may not be a good idea.

BTW, the "system" call always waits for the process to terminate. So your idea won't work. But, if you are not recompressing as the last step, it doesn't really matter. :)

In reply to Re: Best Practices for Uncompressing/Recompressing Files? by husker
in thread Best Practices for Uncompressing/Recompressing Files? by biosysadmin

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.