Below, an example using MCE::Flow. It chunks and outputs swiftly. The relay option when defined loads MCE::Relay and with that enables relay capabilities. Relay is beneficial in places where workers must run orderly and serially. Only a single worker can run inside the relay block. The important thing is that workers enter it orderly by "chunk id" value.

If threads is desired on a Unix platform, simply load threads prior to loading MCE. By default, MCE spawns threads if present. Unlike MCE.pm where chunk_size defaults to 1, chunk_size is configured automatically for MCE models.

use strict;
use warnings;
use MCE::Flow;

## Make gzip file

{
    open my $fh, '|-', 'gzip > test.txt.gz';

    foreach (1..100000) {
        print {$fh} sprintf('%04d',$_).('abc123' x 10)."\n";
    }

    close $fh;
}

## Read gzip file

open my $fh, '-|', 'gzip -cd test.txt.gz' or die "open error: $!\n";

STDOUT->autoflush(1);  # important

MCE::Flow->init(
    max_workers => 3,
    init_relay  => 0
);

sub test {
    my ($mce, $chunkref, $chunkid) = @_;
    my ($buf, $wid) = ('', MCE->wid());

    for my $i (0 .. $#{ $chunkref }) {
        $buf .= $chunkref->[$i];
    }

    MCE::relay {
        print "## worker: $wid, chunkid: $chunkid\n".$buf;
    };
}

MCE::Flow->run_file(\&test, $fh);
MCE::Flow->finish();

close($fh);
[download]

Cheers, Mario.