in reply to Re^6: while reading a file, transfer the same data to two different processes.
in thread while reading a file, transfer the same data to two different processes.

Interesting, thank you Almut.

If you have time & inclination, I've one final line of enquiry: 

use strict;
use warnings;
use threads;

my $str = 'x' x 32_000_000;

sub fork_wait {
    my $pid = fork;
    if ($pid) {
        wait;
    }
    else {
        substr( $str, $_, 1 ) &= ~ ' ' for 0 .. length( $str ) -1;
        exit 0;
    }
}

sub create_thread_join {
    threads->create( sub {
        substr( $str, $_, 1 ) &= ~ ' ' for 0 .. length( $str ) -1;
    } )->join;
}

use Benchmark qw( cmpthese );

cmpthese - 3,
  {
    Forks   => \&fork_wait,
    Threads => \&create_thread_join,
  };

[download]
  • Comment on Re^7: while reading a file, transfer the same data to two different processes.
  • Download Code

Replies are listed 'Best First'.
Re^8: while reading a file, transfer the same data to two different processes.
by almut (Canon) on May 21, 2010 at 02:02 UTC

    As given, the benchmark didn't complete within more than half an hour (not sure why this is so excruciatingly slow). So I stopped it, and reduced the string length 

    my $str = 'x' x 1_000_000;
...
cmpthese 100,
...

    [download]

    Results (similar on both above machines/versions of Perl): 

              Rate Threads   Forks
Threads 2.02/s      --     -4%
Forks   2.11/s      5%      --

    [download]

    (which is not all that surprising, I guess, as modifying the string results in copy-on-write, which destroys fork's advantage)

    Also, the time taken to fiddle with the string here far outweighs the time needed to create a new thread or process...

    Update: with a significantly shorter string, threads actually become faster, with an optimum advantage at a length of around 100 to 10000 (for this particular code sample): 

              Rate   Forks Threads
Forks   67.1/s      --    -64%
Threads  189/s    181%      --

(v5.10.1, Ubuntu 8.04 (kernel 2.6.24), string length 100)


          Rate   Forks Threads
Forks   84.7/s      --    -44%
Threads  152/s     79%      --

(v5.12.0, SUSE 11.1 (kernel 2.6.27), string length 10000)

    [download]
[reply]
[d/l]
[select]

      Those are some really surprising results.

      With fork: as you say COW means that the big string will get copied on demand in pages--probably 4k or 64k chunks--as the string is processed, so lots of page faults.

      With threads, the copy is done once up front.

      Funny how they come out almost the same for the 1MB string.

      As for why so slow, 32e6 calls to substr (which isn't quickest built-in in Perl) explains a lot of it. This would probably run hugely faster. (Which is a sneaky way of asking for one more 'final' test, but just for curiosities sake :): 

      use strict;
use warnings;
use threads;

my $str = 'x' x 32_000_000;

sub fork_wait {
    my $pid = fork;
    if ($pid) {
        wait;
    }
    else {
        substr( $str, $_, 1 ) &= ~ (' 'x1000) for 0 .. (length( $str )
+/1000) -1;
        exit 0;
    }
}

sub create_thread_join {
    threads->create( sub {
        substr( $str, $_, 1 ) &= ~ (' 'x1000) for 0 .. (length( $str )
+/1000) -1;
    } )->join;
}

use Benchmark qw( cmpthese );

cmpthese - 3,
  {
    Forks   => \&fork_wait,
    Threads => \&create_thread_join,
  };

      [download]
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

        With cmpthese -3 (or any other reasonable negative value), it still seems to run "forever"... (not sure why), but when I say cmpthese 100, I get the following: 

                  Rate Threads   Forks
Threads 10.3/s      --    -23%
Forks   13.4/s     30%      --

        [download]
[reply]
[d/l]
[select]

        These tests did run for a while, but less than 5 minutes. My system is a rather old AMD X2 3800+ with 2 Gb RAM running Debian Squeeze (kernel 2.6.32-3-amd64).

        My v5.12.0 

                  Rate Threads   Forks
Threads 9.00/s      --    -54%
Forks   19.5/s    117%      --

        [download]

        Debian's v5.10.1 

                  Rate Threads   Forks
Threads 9.09/s      --    -28%
Forks   12.6/s     39%      --

        [download]
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom