Right now I'm revising a script that has to do the following:
- iterate over a list of ~600 files
- uncompress each of the files using gzip
- reformat each uncompressed file using a proprietary program over which I have no control
- recompress each file so that our disk doesn't run out of space
Right now, I'm doing this all sequentially, one step after another and man is it SLOW. There are lots of ways to optimize this, the easiest and most obvious of which is to 'use Thread'. However, my Perl wasn't compiled with Thread support, and I'm reluctant to replace it on my system when so much of what we do depends on Perl. :(
A few questions that I have:
- Should I even try to optimize this without using Thread? Any ideas on how to do this?
- Is recompiling Perl with threading support that big of a deal?
- How usable is Perl threading for a task such as this? It's not enabled in a default compile of Perl, is it a stable feature or a semi-usable hack?
Being lazy I've left my script it's current state until I've had time to step back and really look at it. I've thought about using some system("gunzip $myfile &") style calls, but I can't figure out how to make synchronize the way I want without some unnecessary assumptions about file size and how it relates to compression/uncompression speed.
Also, I'm a long time reader of PM, and first time poster, I just made a small donation to the cause. Much thanks to the Perl Monks community for making this such a valuable resource. :)
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
| |
For: |
|
Use: |
| & | | & |
| < | | < |
| > | | > |
| [ | | [ |
| ] | | ] |
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.