Hi again,

Here is the same thing using MCE::Hobo. Similar code, but processes instead.

use warnings;
use feature 'say';
use MCE::Hobo;
use MCE::Shared;

# use IO::Handle;  # <-- loaded automatically by MCE and MCE::Shared::
+Server

my $q = MCE::Shared-> queue;

my @gang = map mce_async( sub {
    while ( defined( my $f = $q-> dequeue )) {
        require Compress::Zlib;
        say MCE::Hobo-> tid;
    }
}), 1 .. 100;

$q-> enqueue( $_ ) for 1 .. 100;
$q-> end;

$_-> join for @gang;
[download]

For modules not multi-process safe, another thing one can do on Unix platforms is having MCE::Hobo default to posix_exit to avoid all END and destructor processing.

use warnings;
use feature 'say';
use MCE::Hobo;
use MCE::Shared;

# use IO::Handle;  # <-- loaded automatically by MCE before spawning

MCE::Hobo->init( posix_exit => 1 );

my $q = MCE::Shared-> queue;

my @gang = map mce_async( sub {
    while ( defined( my $f = $q-> dequeue )) {
        require Compress::Zlib;
        say MCE::Hobo-> tid;
    }
}), 1 .. 8;

$q-> enqueue( $_ ) for 1 .. 8;
$q-> end;

$_-> join for @gang;
[download]

Note that the posix_exit option is not recommended if constructing an object inside the worker involving a temp file. In that case one may want the worker to exit normally. Anyway, the posix_exit option is there if needed as a last resort.

Taken from the MCE::Hobo manual: Set posix_exit to avoid all END and destructor processing. Constructing MCE::Hobo inside a thread implies 1 or if present CGI, FCGI, Coro, Curses, Gearman::Util, Gearman::XS, LWP::UserAgent, Mojo::IOLoop, Prima, STFL, Tk, Wx, or Win32::GUI.

A lot of modules are not multi-process safe and the reason for setting to 1 automatically. Btw, Prima is now multi-process safe recently.

Kind regards, Mario

So, the problem is not about Compress::Zlib (or poor Text::PDF::Filter... did we torture it beyond convalescing? oops, sorry). The minimal script is then:

use strict;
use warnings;
use feature 'say';

use threads;
use Thread::Queue;

my $q = Thread::Queue-> new;

my @gang = map async( sub {
    while ( defined( my $f = $q-> dequeue )) {
        require IO::Handle;
        say threads-> tid;
    }
}), 1 .. 8;

$q-> enqueue( $_ ) for 1 .. @gang;
$q-> end;

$_-> join for @gang;
[download]

which sometimes (often, both Windows and Linux) fails randomly. Further, replacing the require line with

use XSLoader; XSLoader::load 'IO';
[download]

sometimes there are strange warnings, like:

Constant subroutine SEEK_END redefined at threads.pl line 4294967295
[download]

Maybe this line number ("broken source"?) is important, maybe not. Until (if ever) the issue is fixed, maybe it would be good idea to advise (documentation, i.e.) to use IO:Handle in any threaded code? Won't hurt...

As to issue of "multi-process safe modules", I can't get same problem if above script is simply re-written using fork (Linux), perhaps it's unrelated issue?

And, certainly, thanks for showing that even a throw-away, few dozen lines script is as easily written using MCE, as "vanilla Perl", except it "just works", because author has put his knowledge and effort to take care of possible bugs. Thanks, Mario :)