Dear Fellow monks,

I am trying to build a Pool of threads, using the Thread::Pool module.
I don't like to waste peoples' time, so there is already a similar node here (Building a thread pool). I've looked at it. It helps in some way.

Now using the example in the module's doc. I tried that code:
#!/usr/bin/perl
use strict;
use warnings;
use Thread::Pool;
my %resolved : shared;
my $pool = Thread::Pool->new(
    {
        workers => 3,
        do      => \&do,
        monitor => \&monitor,
    }
);
$pool->job($_) for 1 .. 100;
$pool->shutdown;
print $_, "\t", $resolved{$_}, $/ for sort { $b <=> $a } keys %resolve
+d;

sub do
{
    $resolved{$_[0]} = $_[0] * $_[0];
    $_[0];
}
sub monitor {return 1}

[download]
And it's great it works...
The only problem is whenever, I go away from trivial examples, it does not work anymore.
So here is another piece of code that DOESN'T work. I'd be grateful to anybody who can tell me why.
#!/usr/bin/perl
use strict;
use warnings;
use Thread::Pool;
my %resolved : shared;
my $pool = Thread::Pool->new(
    {
        workers => 3,
        do      => \&do,
        monitor => \&monitor,
    }
);
$pool->job($_) for 1 .. 100;
$pool->shutdown;
print join("\t",($_,$resolved{$_}->{2},$resolved{$_}->{3},$resolved{$_
+}->{4})), $/ for sort { $b <=> $a } keys %resolved;

sub do
{
    my %sh;
    $sh{2}      = $_[0] * $_[0];
    $sh{3}      = $_[0] * $_[0] * $_[0];
    $sh{4}      = $_[0] * $_[0] * $_[0] * $_[0];
    $resolved{$_[0]} = \%sh;
    $_[0];
}
sub monitor {return 1}

[download]
I am running FC5, Perl 5.8.8 on a dual Xeon Machine, the error message I get is:
Cannot find result for streaming job 1 at /usr/lib/perl5/site_perl/5.8
+.8/Thread/Pool.pm (loaded on demand from offset 20056 for 2428 bytes)
+ line 769.

[download]
Thanks in advance,

Update :
I think I haven't been clear enough about what I would like.
I'd like to build a work crew model of threads, where I could submit 1000s of jobs a the entry, have, e.g., 4 threads running at the same time and an output queue that returns the results whenever they are ready.
Hope that is helpful for those who might have some insights.
Thanks.

Update 2: (mainly for browserUK)
Thanks for your genuine help
I work at a University, so nothing is proprietary
Basically, I want to read in a file that contains some data and treat this data as follow:
Since the initial file contains, e.g., 500 different piece of data to be treated and that every treatment takes ~5min, i'd like to be able to use the 4 processors on the opteron machines to speed the whole process by, e.g., 4.
Now I am not super familiar with threads and all that, but I can make threading more or less work. What I have a problem with is how to stack 1000 jobs in a queue, and have 3 or 4 threads running at a time, whenever one of them is finished, it should return the result and another thread should start the next job waiting on the queue, etc., until all the jobs are completed...
Thanks again
--
jey

In reply to Building a Pool of threads by jey

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.