in reply to Wanting some clarification / opinions on MCE vs Threads
For the curious, here is a version of TGrep.pm by BrowserUk modified to use MCE. Simply remove or comment out "use threads" if Perl lacks support for threads. Threads is not necessary for MCE::Queue.
package TGrep; use strict; use threads; use MCE; use MCE::Queue; our $WORKERS = 4; sub tgrep(&@) { my $workers = $WORKERS; my $code = shift; my @results; my $Qin = MCE::Queue->new( fast => 1 ); my $Qout = MCE::Queue->new( queue => \@results ); my $mce = MCE->new( max_workers => $workers, user_func => sub { $Qout->enqueue( $code->() ? $_ : () ) while local $_ = $Qin- +>dequeue; } )->spawn; $Qin->enqueue( map{ ref $_[0] ? @{ $_ } : $_ } @_ ); $Qin->enqueue( (undef) x $workers ); $mce->run; return wantarray ? @results : \@results; } sub import { no strict 'refs'; my $pkg = caller; *{ $pkg . '::' . $_ } = *{ $_ } for qw[ tgrep ]; } 1;
Thus, new results emerges.
# $N = 1e6 (4 workers) TGrep....: Took 30.264018774 seconds TGrep MCE: Took 15.292614937 seconds (fast => 0) TGrep MCE: Took 7.824651003 seconds (fast => 1)
I was curious about how MCE::Queue compares to Thread::Queue for the demonstration. The fast option is beneficial for an already populated queue.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: Wanting some clarification / opinions on MCE vs Threads
by BrowserUk (Patriarch) on Feb 12, 2015 at 08:52 UTC |