I have observed that the terms "short" and "long" sleep-times, as you put it, depend on the number of threads (=time to fill the @pool=when to join). For example in my system, a sleep of 0.7s will display delays but only when the number of threads is greater than 300. While a sleep of 1s does show these problems but for 500+ threads. This code my @pool = map { threads->create(\&mysub) } 1 .. 50; creates a thread, runs it and then adds it to the @pool and repeats. The important thing to notice is that the thread starts execution immediately while others are still being created. And when thread execution involves system (=fork+exec+waitpid) which has to exit very shortly, somewhere there is a race, given that threads and fork involve replicate program state. Now this is a rough sketch and probably silly, I am very far from being an expert on Perl gastroenterology.

I have a possible workaround, which is lame in that it will probably take longer to finish ... Just make the created threads to wait for the pool to fill up and then do the actual work (system command). That can be achieved with a shared flag like below:

use threads;
use threads::shared;
use strict;

use Time::HiRes qw/clock_gettime CLOCK_REALTIME usleep/;

my $flag:shared = 1;

print "starting at ".clock_gettime(CLOCK_REALTIME)."\n";

my @pool;
for(1..50){
    push @pool, threads->create(\&mysub);
}
print "giving the flag at ".clock_gettime(CLOCK_REALTIME)."\n";
$flag = 0; # all threads go!
$_->join for (@pool);
 
sub mysub {
    my $tid = threads->tid;
    while( $flag ){
    print "thread $tid created and waiting for the flag at ".clock_get
+time(CLOCK_REALTIME)."\n";
    usleep(1000);
    }

    #my $x = 0.0001 + rand(0.0015);
    my $x = 0.001;
    #my $x = rand() >= 0.5 ? 0.001 : 1;
    print "thread $tid is working with sleep=$x, at ".clock_gettime(CL
+OCK_REALTIME)."\n";
    my $id;
    for my $i (1..100){
    $id = "i=$i, tid=$tid";
#    print `
    `
        echo -n "start($id/\$\$) on cpu "\`cat /proc/\$\$/stat | cut -
+d' ' -f39\`": "; date +%s.%N
        sleep $x
        echo -n " stop($id/\$\$) on cpu "\`cat /proc/\$\$/stat | cut -
+d' ' -f39\`": "; date +%s.%N
    `;
    }
    print "ended " . threads->tid . " at ".clock_gettime(CLOCK_REALTIM
+E)."\n";
}
__END__

[download]

An explanation of what's going on should probably involve the fact that system involves fork, exec and waitpid (see Learning Perl, ch 14). And that both threads and fork replicate the program state. And choroba's citing the doc here:

Thinking of mixing fork() and threads? Please lie down and wait until the feeling passes. Be aware that the semantics of fork() vary between platforms. For example, some Unix systems copy all the current threads into the child process, while others only copy the thread that called fork(). You have been warned!

bw, bliako

In reply to Re: Perl threads loss of performance with system call by bliako
in thread Perl threads loss of performance with system call by daniel85

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.