happi has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I want to put together a perl script (or one-liner) and be able to parse a whole directory and do work on the files that match the wanted expression. The key is that I want the script to perform the work on 3 files at a time, even if there are 75 file to get done. When the 3 files are done, (or when on file gets done, the script grabs another file to perform the work on...) This is for unix Perl... IDEAS ?? -Happi

Replies are listed 'Best First'.
•Re: to limit active processes
by merlyn (Sage) on Jan 04, 2005 at 00:48 UTC
Re: to limit active processes
by Zaxo (Archbishop) on Jan 04, 2005 at 00:54 UTC
Re: to limit active processes
by BrowserUk (Patriarch) on Jan 04, 2005 at 01:58 UTC

    If your not stuck on using processes, it's pretty simple with threads.

    #! perl -slw use strict; use threads; use Thread::Queue; our $W ||= 3; my( $dir, $re ) = @ARGV; my $Q = new Thread::Queue; my $running : shared = 0; sub worker { $running++; sleep 1; while( $Q->pending ) { my $file = $Q->dequeue; print "Processed $file"; } $running--; } $Q->enqueue( grep{ m[$re] } glob "$dir/*" ); threads->new( \&worker )->detach for 1 .. $W; sleep 1 while $running < $W; sleep 1 while $running; __END__ [ 1:55:22.37] P:\test>419115 . htm$ Processed ./370949.htm Processed ./376142.htm Processed ./377082db.htm [ 1:55:31.92] P:\test>419115 data dat$ Processed data/1mb.dat Processed data/1millionlines.dat Processed data/5mb.dat Processed data/bin.dat

    Examine what is said, not who speaks.
    Silence betokens consent.
    Love the truth but pardon error.
Re: to limit active processes
by NetWallah (Canon) on Jan 04, 2005 at 01:52 UTC
    How about "use threads;" and Thread::Queue ?

    Sample code is in this node.

        ..."I don't know what the facts are but somebody's certainly going to sit down with him and find out what he knows that they may not know, and make sure he knows what they know that he may not know, and that's a good thing. I think it's a very constructive exchange," --Donald Rumsfeld