Yet another demonstration to be sure MCE::Hobo and MCE::Shared are not impacted when modifying the record separtor. Not localizing the record separator works too. Workers store the result into a shared array.

#!/usr/bin/perl

use strict;
use warnings;

use MCE::Hobo;
use MCE::Shared;

my $mbnum = $ARGV[0] or die "usage: $0 mbnum\n";

my @ldif_files = qw(
    /path/to/file1.ldif.bz2
    /path/to/file2.ldif.bz2
    /path/to/file3.ldif.bz2
);

my $ret = MCE::Shared->array();

for my $idx (0 .. $#ldif_files) {
    my $file = $ldif_files[$idx];

    mce_async {
        local $/ = ""; $ret->set($idx, "");

        open my $fh, "-|", "/usr/bin/bzcat $file"
            or warn "open error ($file): $!";

        if (defined fileno($fh)) {
            while (<$fh>) {
                if (/uid=$mbnum/m) {
                    $ret->append($idx, $_);
                    last;
                }
            }
            close $fh;
        }
    };
}

$_->join for MCE::Hobo->list;   # or MCE::Hobo->waitall;

print join('', $ret->values) if $ret->len;

[download]

Regards, Mario

In reply to Re^6: Searching large files a block at a time by marioroy
in thread Searching large files a block at a time by JediWombat

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.