mantyBBQ has asked for the wisdom of the Perl Monks concerning the following question:

Gracious and wise Monks,

I am attempting to thread a simple parsing subroutine using Thread::Pool::Simple. I can add the thread to the thread pool successfully, the threaded sub's args are passed successfully.

From within the thread, I can, of course, print to STDOUT and I am able to call subs that exists in the main script as well as subs that are imported (for example, use DateTime::TimeZone).

I am not able to call anything external from the thread that has not been imported. Before attempting threading, I was able to call the external sub dynamically (while the sub name does not change, each external .pm is specific to a file format).

This used to work (left out some code that us irrelevant):

use strict; use threads; use threads::shared; use Thread::Pool::Simple; use Getopt::Long; # read parameters use Pod::Usage; use Log2t::Common; use DateTime::TimeZone; use Digest::MD5; use Time::HiRes;

then creation of thread pool:

# prepare thread pool my $pool = Thread::Pool::Simple->new( min => 2, # at least 2 workers max => 10, # at most 10 workers load => 3, do => [\&parse_artifact] ) || print "couldn't create thread pool: $!\n";;

then, the input modules (each is for a file format...iehistory.pm is one of these modules) are loaded...BUT...my threads seem to puke when referencing a sub within them

# load all format files eval { # include the format foreach( keys %formats ) { require $format_dir . '/' . $formats{$_}->{'file'}; } };

then recursive directory search and wait for thread pool...within parse_dir, if a file is of an appropriate type, it should be parsed in the thread pool using parse_artifact

# now we need to recursively go through every file and directory in th +at we got provided as an input and verify if we can parse it (and the +n parse the file) parse_dir( $dir );
# test if we need to parse the artifact and then parse it if ($verify->{'success'}) { $pool->add($formats{$f_key}, $_ ); } </code>

after directory has been recursed...pool waits for completion

# wait for thread pool to empty $pool->join();

here is where things start smelling badly...the parse_artifact() sub gets the args just fine, can print to standard out...and can even call a sub in a loaded module fine...but, when the iehistory.pm is called (mind you, this is just a parser...no OLE or anything odd), the thread just gets stupid...before threading...this ran like the wind

# we have the artifact here, it's verified so we just need to parse it sub parse_artifact() { my ($format, $file) =@_; my $t_line; my @args = undef; print Log2t::Common::get_version(); my $description = Log2t::input::iehistory::get_description(); print "##this was the IE get_description() output: $descriptio +n \n"; # load the log file (prepare, action provided here depends on +the format in question) print "parse_artifact fired; format passed is: $format->{'name +'}; file passed is $file\n";

Replies are listed 'Best First'.
Re: threaded sub cannot access external module
by Corion (Patriarch) on Apr 16, 2010 at 16:22 UTC

    You don't show any relevant code, but guessing from iehistory, you might be trying to use (Windows) objects created outside the thread within another thread. That will likely give you nothing but grief.

      I pasted much more code of the code...if it helps the relevance of my question, great. If not, I may have to trace Thread::Pool::Simple.

        You have posted some parts, and I still cannot conveniently reproduce your problem. This makes it hard for me to help you. But at least one of my guesses is that you are loading the modules later than you create your threads. Maybe the modules are not available in the other namespaces threads. Try loading the modules before you create your threads.