Hello, I wrote some time ago a script to calculate the standard deviation between the corresponding values found in different input files and this week I decided to convert it to use multiple processors. I am using perl 5.8.9 compiled with macports with support for ithreads.

Conceptually, I read with the master thread the data of the different files, one at time, and I interleave them in an array. Later I start multiple threads, each one working on a different part of the initial array. I use Statistics::Basic to calculate the standard deviation of the values of a small slice of the initial array and I put the results in a temporary array, then copied to the array of the results, that is the only shared array I use.

This is the code of the main thread managing the work:

my %mythreads;
my $work_package = int($elements / $number_threads + 0.5);
my @ReturnData :shared;
my $start = 0;
my $stop = $start + $work_package * $files - 1;
$stop = $elements * $files - 1 if ($stop >= $elements * $files);

for (my $i = 0; $i < $number_threads; $i++) {
    $mythreads{$i} = threads->create(\&do_work);
    $start = $stop + 1;
    $stop += $work_package * $files;
    $stop = $elements * $files - 1 if ($stop >= $elements * $files);
}
# collect data
foreach (sort(keys(%mythreads))) {
    $mythreads{$_}->join();
}

[download]
Each worker thread executes the following sub: 
sub do_work {
    my @working_block = @input_data[$start .. $stop];
    my @partial_output_data;
    $, = undef;
    $\ = undef;

    my $output_line = 0;
    for (my $i = 0; $i < @working_block; $i += $files) {
        # calculates the stddev of each slice and outputs the formatte
+d resuls
        if (max(@working_block[$i .. $i + $files - 1]) == 0) {
            $partial_output_data[$output_line] = '0.00000E+00';
        } else {
            $partial_output_data[$output_line] = 
                sprintf('%.5E', stddev(@working_block[$i .. $i + $file
+s - 1]) / 
                mean(@working_block[$i .. $i + $files - 1]));
        }
        $output_line++;
    }
    @ReturnData[$start / $files .. $stop / $files] = @partial_output_d
+ata;
    return;
}

[download]
I timed the execution and the results are found here: http://img62.imageshack.us/i/perlthreading.png/ 6 seconds are taken by the other parts of the script, the remaining is for the code displayed in this post.

Is this behaviour normal? what could I do to improve the performances? this was more a test than a real need, but I would like to know for the future. I thought threads were easy enough not to increase too much the code size/effort, while processes would have required IPC or something else.

Thank you very much.

In reply to Poor performances with threads by olafmar

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.