If your target OS is one with a native fork then that or one of the CPAN wrappers is possibly the way to go, but if you wanted a threads solution, your attempt was making a mountain out of a molehill.

There is certainly no logic to using 3 queues to pass the path, the filename, and the path+filename. Pass it once and split it up there if need be.

And there is no need to count the threads or detect when one has finished. Just start the number your want to use (say 4), and have them loop over the queue. As soon as they finish one piece of work they'll pick up the next automatically. It is also far more efficient than starting a new thread for each file.

When you've finished pushing all the filenames, push 4 undefs and the threads will self terminate when there is nothing left to do. Simplicity itself.

#! perl -slw use strict; use threads; use Thread::Queue; use File::Find; use File::Copy; use File::Path; my $THREADS = 4; my $Q = new Thread::Queue; sub worker { while( my $path = $Q->dequeue ) { print "Creating directory" . $workdirectory . "\n"; mkpath($workdirectory); copyfiles( $workdirectory, $FilenamewithpathQueue->dequeue, $filename[0] ); print "Processing Calculations...\n"; open(RUNSCRIPT,">runscript.bash"); print RUNSCRIPT <<EOD cd $workdirectory perl scriptthatincludesfortran.pl >& $workbasedirectory/$filen +ame[0].log EOD close(RUNSCRIPT); if (system("bash runscript.bash") != 0) { die "failed!\n";} else { print "done!\n"; } print "bash $workdirectory/runscript.bash\n"; print "\n"; } } my @threads = map{ threads->create( \&worker ) } 1 .. $THREADS; find( sub{ $Q->enqueue( $File::Find::name ) }, $basedirectory ); $Q->enqueue( (undef) x $THREADS ); $_->join for @threads;

The code is untested because I couldn't make much sense of your OP code, as half the variables in it are never declared and never initialised.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

In reply to Re: Parallelization of heterogenous (simple with threads) by BrowserUk
in thread Parallelization of heterogenous (runs itself Fortran executables) code by Jochen

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.