in reply to Re^2: Module for transparently forking a sub?
in thread Module for transparently forking a sub?

So, you have time to make the claim, but not the time to substantiate it. There's a name for that:FUD!

Okay, here my counter claim.

I can start a thread, run a subroutine that returns a complex data structure, and retrieve that data structure to the calling code faster than you can do the same using fork. My timing is: 0.0261 seconds.

c:\test>junk8 -N=100 Time taken: 0.0261 seconds { A => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], ARGS => [1, "2.3", "four"], B => { a => "b", c => "d", e => "f", g => "h", i => "j", k => "l", "m" => "n", o => "p", "q" => "r", "s" => "t", u => "v", w => "x", "y" => "z", }, C => "Just a big scalarJust a big scala big scalarJust a big scalarJust a big sc }

And my benchmark code:

#! perl -slw use strict; use threads; use Time::HiRes qw[ time ]; use Data::Dump qw[ pp ]; our $N ||= 10; sub stuff { my %hash = ( ARGS => \@_, A => [ 1 .. 10 ], B => { 'a' .. 'z' }, C => 'Just a big scalar' x 100, ); return \%hash; } my $complexData; my $start = time; for ( 1 .. $N ) { ## "fork" the subroutine my( $thread ) = async \&stuff, 1, 2.3, 'four' ; ## Do other stuff sleep 1; ## Get the complex results $complexData = $thread->join; } printf "Time taken: %.4f seconds\n", ( time() - $start ) / $N - 1; ## Display them pp $complexData;

Care to substantiate your claim and disprove mine?


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Replies are listed 'Best First'.
Re^4: Module for transparently forking a sub?
by samtregar (Abbot) on Feb 14, 2009 at 05:33 UTC
    Ok, I took the bait and my faith in the general slowness of threading remains intact. Here's my results:

    $ perl thread.pl -N=100 Time taken w/ threads: 0.0077 seconds Time taken w/ forks: 0.0021 seconds

    And here's my code:

    #! perl -slw use strict; use threads; use Time::HiRes qw[ time ]; use Data::Dump qw[ dump ]; use Storable qw[ store_fd fd_retrieve ]; our $N ||= 10; sub stuff { my %hash = (ARGS => \@_, A => [1 .. 10], B => {'a' .. 'z'}, C => 'Just a big scalar' x 100,); return \%hash; } my $complexData; { my $start = time; for (1 .. $N) { ## "fork" the subroutine my ($thread) = async \&stuff, 1, 2.3, 'four'; ## Do other stuff sleep 1; ## Get the complex results $complexData = $thread->join; } printf "Time taken w/ threads: %.4f seconds\n", (time() - $start) +/ $N - 1; } my $complexData2; { my $start = time; pipe(READ, WRITE); for (1 .. $N) { my $pid = fork; if (!$pid) { # in the kid - do stuff() and send it back to parent store_fd(stuff(1, 2.3, 'four'), \*WRITE); exit; } ## Do other stuff sleep 1; ## Get the complex results $complexData2 = fd_retrieve(\*READ); waitpid($pid,0); } printf "Time taken w/ forks: %.4f seconds\n", (time() - $start) / +$N - 1; } # data should match if (dump($complexData) ne dump($complexData2)) { warn "Data did not match!"; }

    As I was coding it I realized it's kind of a bizarre benchmark since it's not really testing any concurency. It's only testing how fast a single thread/process can be spawned and send back data. And really there's just no way Perl's threads are going to beat fork() at that test!

    One neat thing I learned - I didn't realize you could use use Storable's store_fd() and fd_retrieve() to pass messages like this. I'd previously used nstore() and thaw() with a prefixed length() of the message so the other end would know how much to read. This is so much easier!

    -sam

    PS: I just noticed you're on Windows (or DOS, I guess)! You don't have a real fork() there, so I guess you're not going to be able to replicate my results. Oh well.

        Nothing too fancy - it's a Core 2 Duo T7200 @ 2GHz running Fedora 8. It was top-o-the-line when I bought it 1.75 years ago, but time marches on...

        Frankly, I couldn't figure out what you were doing with dividing the time by $N-1, but I decided not to care since I could just do the same for threads and forks and they should be affected equally. Also, sleep(1) in a benchmark could create some interesting problems - the OS generally doesn't guarantee how long that will take, only that it will take at least 1 second.

        -sam

Re^4: Module for transparently forking a sub?
by samtregar (Abbot) on Feb 13, 2009 at 22:40 UTC
    Oh fine. But you'll have to wait. It does sound fun, but real work beckons...

    -sam

Re^4: Module for transparently forking a sub?
by Anonymous Monk on Feb 14, 2009 at 18:05 UTC
    --Care to substantiate your claim and disprove mine? Sure, you cannot fork on windows.