Update: Added chunk_size option to the example.
Update: Added note on one worker reading at any given time.
The example by BrowserUk is awesome. Perhaps you're having to work with Perl not compiled with threads; e.g. running Perl on Solaris. A pattern counter can be done with MCE using the following code. The usage is the same between the two scripts.
This is safe from random I/O thrashing. The read pattern is sequential, not random between workers due to one worker reading at any given time.
#! perl -slw
use strict;
use warnings;
use MCE::Loop;
our $T //= 'auto';
my ( $file_name, $target ) = @ARGV;
MCE::Loop::init {
use_slurpio => 1, max_workers => $T,
chunk_size => 1024 * 1024 * 16,
};
my @result = mce_loop_f {
my ( $mce, $slurped_chunk, $chunk_id ) = @_;
my $count = 0;
$count++ while ( $$slurped_chunk =~ /$target/g );
MCE->gather($count);
} $file_name;
my $total = 0;
$total += shift @result while @result;
print "Found $total '$target' lines";
Usage:
mce_script.pl -T=n theFile.txt "the string"
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.
|