This takes 5.25 seconds on my machine (4GB, 4 cores, Linux).

By "This", I assume you mean this:

use 5.010; use Time::HiRes qw(time); my $data = 'x' x 500 * 1024**2; my $t = time; for my $n ( 1 .. 100 ) { unless (fork) { substr( $data, 4096 * $_ + $n, 1 ) |= 1 for 0 .. 124; exit; } } waitall; say time - $t;

Sounds good, but it isn't doing what you think its doing, nor what I intended, because of uncorrected typos in the (untested) example that you've "ignored". So, lets correct the deficiencies in the forked code:

I bet if you put back the strict and warnings, and fix the errors so that it is actually doing something:

<code> use strict; use warnings; use 5.010; use Time::HiRes qw(time); my $data = 'x' x 500 * 1024**2; my $t = time; for my $n ( 1 .. 100 ) { unless (fork) { substr( $data, 4096 * $_ + $n, 1 ) |= chr(1) for 0 .. 124; exit; } } waitall; say time - $t;

it will take a little longer. (I'll have to sort out and fire up my Unbuntu VM to be sure.)

The respective threads versions take: 164.842848062515 172.88907790184

Well yeah! If you use threads like fork, it will take a while.

For a start, you're running each of the 100 threads serially.

Second, you're building a new copy of the 500MB string in each thread, modifying 125 bits within it, and then discarding it.

The point of the exercise was to modify the same string, not 100 * 500MB copies. That's 100 times what the forked version is sharing! You're surprised that it is slow?

So, lets address those deficiencies of your threaded version:

#! perl -slw use strict; use 5.010; use threads ( stack_size => 4096 ); use Thread::Queue; use Time::HiRes qw(time); use constant DATA_SIZE => (500 * 1024**2); sub thread { my( $Q, $n ) = @_; $Q->enqueue( 4096 * $_ + $n ) for 0 .. 124; $Q->enqueue( undef ); } my $t = time; my $Q = new Thread::Queue; my @t = map threads->create( \&thread, $Q, $_ ), 1 .. 100; my $data = 'x' x DATA_SIZE ; for(1..100) { substr( $data, $_, 1 ) |= chr(1) while $_ = $Q->dequeue; } $_->join for @t; say time - $t; printf "%d bits changed\n", DATA_SIZE - ( $data =~ tr[x][] ); __END__ c:\test>junk44 2.03600001335144 12500 bits changed

So, 100 threads calculating modifications to 1/2GB of data, and 2.03 seconds.

Oh. And when you work out how to print last line of output, we can try for the real range of 0 .. 124_999 :)


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy

In reply to Re^24: Strange memory leak using just threads (forks.pm) by BrowserUk
in thread Strange memory leak using just threads by MnkyBrain

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.