Crash with ForkManager on Windows

amitsq has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Crash with ForkManager on Windows by Discipulus (Canon) on Sep 20, 2017 at 21:07 UTC
Hello amitsq, I'm not at all a parallel monk and here there are, very skilled. > my $pm= new Parallel::ForkManager(8); #didn’t matter if it’s 1 or 5 or whatever else your original program crash here too and in the same way, but with 1 does not. I have borrowed the below modifying it a little from example from P::FM docs and the resulting code runs with no crashes. At least something to start with: use Parallel::ForkManager; use LWP::Simple; my $pm = Parallel::ForkManager->new(10); LINKS: for my $link ('http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif', 'http://hooboy.no-such-host.int/', 'http://www.yahoo.com', 'http://www.ora.com/ask_tim/graphics/asktim_header_main.gif', 'http://www.guardian.co.uk/', 'http://www.pixunlimited.co.uk/siteheaders/Guardian.gif') { $pm->start and next LINKS; my ($content_type, $document_length, $modified_time, $expires, $ +server) = head($link); print "$link fetched: declared lenght $document_length, \n"; $pm->finish; }; # output http://hooboy.no-such-host.int/ fetched: declared lenght , http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif fetched: declared leng +ht 5611, http://www.guardian.co.uk/ fetched: declared lenght 628397, http://www.ora.com/ask_tim/graphics/asktim_header_main.gif fetched: de +clared lenght , http://www.pixunlimited.co.uk/siteheaders/Guardian.gif fetched: declar +ed lenght , http://www.yahoo.com fetched: declared lenght 12, [download] L* There are no rules, there are no thumbs.. Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.	[reply] [d/l]
Re^2: Crash with ForkManager on Windows by amitsq (Beadle) on Sep 21, 2017 at 09:53 UTC
i tried your reduced code, 100% copy - no change and it crashes. I might run it 20 times with no crashes, but then the crashes happened like before. Even once i got a very strange error message on the terminal: http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif 1354074654, Number found where operator expected at D:/Perl64/lib/File/Glob.pm lin +e 13, near "over 4" (Do you need to predeclare over?) Semicolon seems to be missing at D:/Perl64/lib/File/Glob.pm line 16. Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 17, near "<GLOB_NOSORT> is" (Missing operator before is?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 18, near ") rather" (Missing operator before rather?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 24, near ") returns" (Missing operator before returns?) Can't locate object method "_init_implementor" via package "URI::http" + at D:/Perl64/lib/URI.pm line 137. Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 25, near "&File::Glob::GLOB_ERROR will" (Missing operator before will?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 26, near "&File::Glob::GLOB_ERROR is" (Missing operator before is?) Number found where operator expected at D:/Perl64/lib/File/Glob.pm lin +e 29, near "over 4" (Do you need to predeclare over?) Semicolon seems to be missing at D:/Perl64/lib/File/Glob.pm line 32. Semicolon seems to be missing at D:/Perl64/lib/File/Glob.pm line 36. Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 41, near ") has" (Missing operator before has?) Operator or semicolon missing before &File::Glob::ERROR at D:/Perl64/l +ib/File/Glob.pm line 43. Ambiguous use of & resolved as operator & at D:/Perl64/lib/File/Glob.p +m line 43. Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 45, near ") deviates" (Missing operator before deviates?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 45, near ") behaviour" (Missing operator before behaviour?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 46, near ") will" (Missing operator before will?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 47, near "continue processing" (Do you need to predeclare continue?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 47, near "<GLOB_ERR> flag" (Missing operator before flag?) Number found where operator expected at D:/Perl64/lib/File/Glob.pm lin +e 54, near "over 4" (Do you need to predeclare over?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 60, near ") isn't" (Missing operator before isn't?) Semicolon seems to be missing at D:/Perl64/lib/File/Glob.pm line 66. Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 67, near "use of backslash" (Do you need to predeclare use?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 68, near ") solution" (Missing operator before solution?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 68, near "use forward slashes" (Do you need to predeclare use?) String found where operator expected at D:/Perl64/lib/File/Glob.pm lin +e 70, near "match "normal practice"" (Do you need to predeclare match?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 70, near ""normal practice" on" (Missing operator before on?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 71, near ") only" (Missing operator before only?) String found where operator expected at D:/Perl64/lib/File/Glob.pm lin +e 72, near "metacharacters '['" (Do you need to predeclare metacharacters?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 77, near "use the real" (Do you need to predeclare use?) Array found where operator expected at D:/Perl64/lib/File/Glob.pm line + 89, at end of line Semicolon seems to be missing at D:/Perl64/lib/File/Glob.pm line 90. Array found where operator expected at D:/Perl64/lib/File/Glob.pm line + 91, at end of line Array found where operator expected at D:/Perl64/lib/File/Glob.pm line + 92, at end of line Array found where operator expected at D:/Perl64/lib/File/Glob.pm line + 93, at end of line Semicolon seems to be missing at D:/Perl64/lib/File/Glob.pm line 93. Number found where operator expected at D:/Perl64/lib/File/Glob.pm lin +e 96, near ") 1989" (Missing operator before 1989?) Bareword found where operator expected at D:/Perl64/lib/File/Glob.pm l +ine 96, near "1993 The" [download] No idea where that pop up from. It only happened once in all the times trying.	[reply] [d/l]
Re^3: Crash with ForkManager on Windows by marioroy (Prior) on Sep 22, 2017 at 19:10 UTC
When seeing errors (Bareword found where operator expected), try pre-loading the module before running parallel. See, this post. `use LWP::Simple; # On the Windows platform, pre-loading essential modules # (required at runtime) may be necessary for extra stability. if ( $INC{'LWP/UserAgent.pm'} && !$INC{'Net/HTTP.pm'} ) { require IO::Handle; require Net::HTTP; require Net::HTTPS; } # run parallel ...` [download]	[reply] [d/l]
Re: Crash with ForkManager on Windows by kcott (Archbishop) on Sep 21, 2017 at 08:45 UTC
G'day amitsq, I suspect your problems are related to the additional `$pm->finish` and `next` statements. Your instantiation of the Parallel::ForkManager object may also be an issue. What you have is different from every example shown in the documentation; also see "perlobj: Invoking Class Methods" (including its "Indirect Object Syntax" subsection). Using exactly the same message to identify your different cases is pointless: use meaningful messages. I rewrote your code like this (`pm_1199760_parallel_forkmgr_link_checker.pl`): #!/usr/bin/env perl -l use strict; use warnings; use LWP::Simple; use Parallel::ForkManager; my @urls = qw{ http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif http://hooboy.no-such-host.int/ http://www.yahoo.com http://www.ora.com/ask_tim/graphics/asktim_header_main.gif http://www.guardian.co.uk/ http://www.pixunlimited.co.uk/siteheaders/Guardian.gif }; my $max_processes = $ARGV[0] \|\| 8; my $pm = Parallel::ForkManager::->new($max_processes); print 'Start link checking.'; for my $url (@urls) { $pm->start and next; my ($type, undef, $mod) = head($url); if (defined $type) { print "PASS: $url"; print ' MOD: ', $mod \|\| 0; } else { print "FAIL: $url"; } $pm->finish; } $pm->wait_all_children; print 'Link checking completed.'; [download] Here's the output with no argument: `$ pm_1199760_parallel_forkmgr_link_checker.pl Start link checking. FAIL: http://hooboy.no-such-host.int/ PASS: http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif MOD: 1354074654 PASS: http://www.guardian.co.uk/ MOD: 0 FAIL: http://www.ora.com/ask_tim/graphics/asktim_header_main.gif PASS: http://www.yahoo.com MOD: 0 FAIL: http://www.pixunlimited.co.uk/siteheaders/Guardian.gif Link checking completed.` [download] I then ran it with arguments of `8`, `5`, `3`, `2` and `1`. Lke the first run (with no argument), all ran to completion: no crashes! With the exception of `1`, they produced the same results (although the order of output varied). Running with an argument of `1` rather defeats the purpose of running processes in parallel; however, as you said you'd used it, I tried it also. Here's what I got: `$ pm_1199760_parallel_forkmgr_link_checker.pl 1 Start link checking. PASS: http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif MOD: 1354074654 FAIL: http://hooboy.no-such-host.int/ PASS: http://www.yahoo.com MOD: 0 FAIL: http://www.ora.com/ask_tim/graphics/asktim_header_main.gif FAIL: http://www.guardian.co.uk/ FAIL: http://www.pixunlimited.co.uk/siteheaders/Guardian.gif Link checking completed.` [download] I ran that a second time: the result was the same. I'll leave you to investigate that further. — Ken	[reply] [d/l] [select]
Re^2: Crash with Parallel::ForkManager on Windows by Discipulus (Canon) on Sep 21, 2017 at 09:27 UTC
Hello kcott, where are you running your code? Unfortunately on Perl (v5.14.2) built for MSWin32-x86-multi-thread on win7 and `Parallel::ForkManager 1.19` I get more than 50% of crashes, using your code. Testing more times mine, as posted above, I get ~25% of crashes.. (not saying mine is better: better is no crash at all ;=) Unfortunately windows is a shaggy beast and I'm not able to debug this behaviour further: what I can add is that during installation I got some warning about disappeared PIDs even if the overall testing process ended ok: t/00-report-prereqs.t ..... ok t/01-utf8-all.t ........... skipped: This is a bug in perl itself on W +indows t/02-callback.t ........... # This test can take 10-20 seconds, please + wait. Started at Thu Sep 21 11:23:08 2017 t/02-callback.t ........... ok t/03-callback-data.t ...... # This test can take 2-6 seconds, please w +ait. Started at Thu Sep 21 11:23:25 2017 t/03-callback-data.t ...... ok t/basic-methods.t ......... ok t/changing-pids.t ......... ok t/start_child.t ........... ok t/waitpid-conflict.t ...... # code outside of P::FM stole -1152 child process '-1152' disappeared. A call to `waitpid` outside of Para +llel::ForkManager might have reaped it. t/waitpid-conflict.t ...... ok t/waitpid-waitonechild.t .. # code outside of P::FM stole -5456 # found warning: child process '-5456' disappeared. A call to `waitpid +` outside of Parallel::ForkManager might have reap ed it. t/waitpid-waitonechild.t .. 1/? # expected to find warning: (?^:child +process '\d+' disappeared. A call to `waitpid` out side of Parallel::ForkManager might have reaped it\.) t/waitpid-waitonechild.t .. ok t/waitpid_blocking.t ...... ok All tests successful. [download] L* PS modified the title a bit for better later search There are no rules, there are no thumbs.. Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.	[reply] [d/l] [select]
Re^3: Crash with Parallel::ForkManager on Windows by kcott (Archbishop) on Sep 21, 2017 at 10:03 UTC
G'day Discipulus, Thanks for the follow-up. While it's clear the OP was using Windows, I wasn't aware that this was a Windows-specific problem: I don't see anything like "This worked on XYZ, but not on Windows". I'm using: `Perl 5.26.0 (built for darwin-thread-multi-2level) Parallel::ForkManager 1.19 LWP::Simple 6.26` [download] I had to install Parallel::ForkManager and the `cpan` output is still on one of my screens. Here's my equivalent lines to those you posted: t/00-report-prereqs.t ..... ok t/01-utf8-all.t ........... skipped: Need utf8::all for this test cras +hing on Windows # This test can take 10-20 seconds, please wait. Started at Thu Sep 21 + 17:07:29 2017 t/02-callback.t ........... ok # This test can take 2-6 seconds, please wait. Started at Thu Sep 21 1 +7:07:45 2017 t/03-callback-data.t ...... ok t/basic-methods.t ......... ok t/changing-pids.t ......... ok t/start_child.t ........... ok # code outside of P::FM stole 2369 child process '2369' disappeared. A call to `waitpid` outside of Paral +lel::ForkManager might have reaped it. t/waitpid-conflict.t ...... ok # code outside of P::FM stole 2371 t/waitpid-waitonechild.t .. ok t/waitpid_blocking.t ...... ok All tests successful. [download] You appear to have had problems with `t/waitpid-waitonechild.t` that I didn't have; everything else looks fairly similar. To be honest, I started the installation, went and made a coffee, came back and saw: `YANICK/Parallel-ForkManager-1.19.tar.gz /usr/bin/make install -- OK cpan[2]>` [download] I rarely review the (often many screenfuls of) output unless the installation failed. — Ken	[reply] [d/l] [select]
Re^4: Crash with Parallel::ForkManager on Windows by holli (Abbot) on Sep 21, 2017 at 12:46 UTC
Re^5: Crash with Parallel::ForkManager on Windows by kcott (Archbishop) on Sep 22, 2017 at 06:15 UTC
Re^2: Crash with ForkManager on Windows by amitsq (Beadle) on Sep 21, 2017 at 10:18 UTC
Taking your code, i get always the same output regardless which param i take. But it crashes too.	[reply]
Re^3: Crash with ForkManager on Windows by kcott (Archbishop) on Sep 22, 2017 at 06:07 UTC
Feedback, so far, is showing crashes on Windows Perl versions `5.14` and `5.16`; however, `5.24` on Windows did not crash. "CPAN Testers Matrix: Parallel-ForkManager 1.19" is showing failures (for "mswin32") up to `5.22.3`, but no failures for `5.24.0` or higher. There's virtually no failures for any other platforms: two very old "linux" ones for `5.8.x`; and one other for "cygwin" `5.22.2` (which I believe only runs on Windows anyway). You should probably dig a bit deeper into this than I have, but problems do not appear to be solely related to ActivePerl; while looking around I came across this GitHub bug report: "Parallel::Forkmanager 1.19 cpan install fails on Windows Server 2016 Strawberry Perl 5.16.3". (Don't take that one report as being in any way conclusive.) Obviously, installation failures are not the same as crashes, but a trend is suggested. Again, something for you to look into further. I'd suggest installing a newer Perl (at least `5.24.0`; but `5.26.0` — the highest stable version — would be better) and trying the code again with that. Also ensure you've got the latest versions of the modules you're using. — Ken	[reply] [d/l] [select]
Re^4: Crash with ForkManager on Windows -- newer perl does not crash by Discipulus (Canon) on Sep 22, 2017 at 07:25 UTC
Re: Crash with ForkManager on Windows by holli (Abbot) on Sep 20, 2017 at 19:57 UTC
Try wrapping the relevant parts of your code in try/catch blocks and see if it is dying somewhere unexpectedly. holli You can lead your users to water, but alas, you cannot drown them.	[reply] [d/l]
Re^2: Crash with ForkManager on Windows by amitsq (Beadle) on Sep 21, 2017 at 10:01 UTC
ok i did that. Can't see why value shouldn't be defined though :/ `Can't call method "start" on an undefined value at line 18. Can't call method "finish" on an undefined value at .\multithreadsTe\3 +.pl line 25. </code which basically are following lines in my code: <code> line 18: $pm->start and next LINKS; line 25: $pm->finish;` [download] my modified code again: #!d:\perl\bin\perl.exe use Parallel::ForkManager; use LWP::Simple; use Try::Tiny; try{ my $pm = Parallel::ForkManager->new(6);} catch {print "error 1 \n"; exit;}; LINKS: for my $link ('http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif', 'http://hooboy.no-such-host.int/', 'http://www.yahoo.com', 'http://www.ora.com/ask_tim/graphics/asktim_header_main.gif', 'http://www.guardian.co.uk/', 'http://www.pixunlimited.co.uk/siteheaders/Guardian.gif') { try { $pm->start and next LINKS; } catch { warn $_; print "### ", $link, " \n error 2 \n"; next;}; try{ my ($content_type, $document_length, $modified_time, $expires, $ +server) = head($link); print "$link fetched: declared lenght $document_length, \n"; } catch{print "error 3 \n";}; try{$pm->finish; } catch{ warn $_; print "error 4 \n";}; }; [download]	[reply] [d/l] [select]
Re^3: Crash with ForkManager on Windows by holli (Abbot) on Sep 21, 2017 at 12:52 UTC
Can't see why value shouldn't be defined though Because you defined $pm lexically in the try - block and you don't have `strict` on. Edit: I just ran your code with the issue fixed but experience no crashes (Win 8.1, ActivePerl 5.24). I think there is something broken with your installation of the module and or Perl itself. I suggest a clean reinstall of both. holli You can lead your users to water, but alas, you cannot drown them.	[reply] [d/l] [select]
Re^4: Crash with ForkManager on Windows by amitsq (Beadle) on Sep 21, 2017 at 13:35 UTC
Re^3: Crash with ForkManager on Windows by soonix (Canon) on Sep 21, 2017 at 13:00 UTC
By wrapping the `my $pm = Parallel::ForkManager->new(6);` [download] into a `try` block, `$pm` now is local to that new block … I think `strict` would have told you about it.	[reply] [d/l] [select]
Re: Crash with ForkManager on Windows by karlgoethebier (Abbot) on Sep 22, 2017 at 05:06 UTC
You might also take a look at Yet another example to get URLs in parallel. Perhaps something like this works for you. You can simplify it for your needs (no role etc.). I vaguely remember that i tried something like this with LWP::Simple and it didn't work. I forgot why ;-) Hope this helps. Regards, Karl «The Crux of the Biscuit is the Apostrophe» `perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'`Help	[reply] [d/l]
Re: Crash with ForkManager on Windows -- further readings by Discipulus (Canon) on Sep 21, 2017 at 22:19 UTC
..Ok I was trying to solve this with different tools/approach but.. not enough time and no results to show. But LWP::Parallel on one side and Mojo::UserAgent + Mojo::IOLoop on another side seem paths worth to be investigated. If you want read more see WWW::Curl and making it thread safe and Re: Proper way to thread this in PERL. (mech lwp asynchronous) which point to a long read: Your main event may be another's side-show. Or you can give a try to BrowserUk's `divide et imperat` tecnique: Re^5: Threads and HTTPS Segfault share your findings! L* There are no rules, there are no thumbs.. Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.	[reply] [d/l] [select]
Re: Crash with ForkManager on Windows by marioroy (Prior) on Sep 22, 2017 at 08:56 UTC
Update: Modified script. Completes in 1 second with Cygwin Perl v5.22.4 and less than 1 second on Unix. Update: Modified list of modules to pre-load. On the Windows platform, run with Perl v5.26 or later for best results. Running parallel is problematic and not fun on the Windows platform. To increase the chance for failure, I quadrupled the input size. # Strawberry Perl on Windows 7 VM # LWP::Simple included with Perl # Parallel::ForkManager v1.19 # Testing involves running multiple times. # Failing indicates the script crashed one or more times. perl-5.10.1.2 - LWP::Simple v5.827 : pass, slow (> 14 seconds) perl-5.12.3.0 - LWP::Simple v5.835 : pass, slow (> 14 seconds) perl-5.14.4.1 - LWP::Simple v6.00 : fail perl-5.16.3.1 - LWP::Simple v6.00 : fail perl-5.18.4.1 - LWP::Simple v6.00 : fail perl-5.20.3.3 - LWP::Simple v6.15 : fail perl-5.22.3.1 - LWP::Simple v6.15 : fail perl-5.24.2.1 - LWP::Simple v6.26 : fail perl-5.26.0.2 - LWP::Simple v6.26 : pass, fast (~ 3 seconds) [download] A solution is pre-loading essential modules (required at runtime) by LWP::Simple before running parallel. use strict; use warnings; use LWP::Simple; # Pre-load essential modules for extra stability. if ( $INC{'LWP/UserAgent.pm'} && !$INC{'Net/HTTP.pm'} ) { require IO::Handle; require Net::HTTP; require Net::HTTPS; } my @urls = ( 'http://hooboy.no-such-host.int/', 'http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif', 'http://www.guardian.co.uk/', 'http://www.ora.com/ask_tim/graphics/asktim_header_main.gif', 'http://www.pixunlimited.co.uk/siteheaders/Guardian.gif', 'http://www.yahoo.com', ) x 4; use Parallel::ForkManager; my $pm = new Parallel::ForkManager(8); if ( $^O ne 'MSWin32' ) { $pm->set_waitpid_blocking_sleep(0); } foreach my $url ( @urls ) { $pm->start and next; my ($type, $length, $mod) = head($url); # if (!defined $type) { # ... # } # elsif ($mod) { # ... # } # else { # ... # } print "$url is done\n"; $pm->finish; } $pm->wait_all_children; [download] Results. # Strawberry Perl on Windows 7 VM # LWP::Simple included with Perl # Parallel::ForkManager v1.19 # Testing involves running multiple times. perl-5.10.1.2 - LWP::Simple v5.827 : pass, > 14 seconds perl-5.12.3.0 - LWP::Simple v5.835 : pass, > 14 seconds perl-5.14.4.1 - LWP::Simple v6.00 : pass, > 7 seconds perl-5.16.3.1 - LWP::Simple v6.00 : pass, > 7 seconds perl-5.18.4.1 - LWP::Simple v6.00 : pass, > 7 seconds perl-5.20.3.3 - LWP::Simple v6.15 : pass, > 6 seconds perl-5.22.3.1 - LWP::Simple v6.15 : pass, > 6 seconds perl-5.24.2.1 - LWP::Simple v6.26 : pass, > 6 seconds perl-5.26.0.2 - LWP::Simple v6.26 : pass, ~ 3 seconds perl v5.22.4 on Cygwin - LWP::Simple v6.27 : pass, 1 second ;-) [download] Perl 5.26 provides the best performance, completing in 3 seconds. Regards, Mario	[reply] [d/l] [select]
Re^2: Crash with ForkManager on Windows by marioroy (Prior) on Sep 22, 2017 at 09:17 UTC
Update: Modified script. Completes in 1 second with Cygwin Perl v5.22.4 and less than 1 second on Unix. Update: Modified list of modules to pre-load. On the Windows platform, run with Perl v5.26 or later for best results. The following does the same thing using MCE::Hobo. Pre-loading essential modules is necessary, even for Perl v5.26. The posix_exit option applies to UNIX-like OSes and is helpful in the event an underlying module or dependency isn't multi-process safe. use strict; use warnings; use LWP::Simple; # Pre-load essential modules for extra stability. if ( $INC{'LWP/UserAgent.pm'} && !$INC{'Net/HTTP.pm'} ) { require IO::Handle; require Net::HTTP; require Net::HTTPS; } my @urls = ( 'http://hooboy.no-such-host.int/', 'http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif', 'http://www.guardian.co.uk/', 'http://www.ora.com/ask_tim/graphics/asktim_header_main.gif', 'http://www.pixunlimited.co.uk/siteheaders/Guardian.gif', 'http://www.yahoo.com', ) x 4; use MCE::Hobo 1.831; MCE::Hobo->init( max_workers => 8, posix_exit => 1, ); foreach my $url ( @urls ) { mce_async { my ($type, $length, $mod) = head($url); print "$url is done\n"; }; } MCE::Hobo->waitall; [download] Regards, Mario	[reply] [d/l]
Re^3: Crash with ForkManager on Windows by marioroy (Prior) on Sep 22, 2017 at 10:06 UTC
Update: Modified script. Completes in 1 second with Cygwin Perl v5.22.4 and less than 1 second on Unix. Update: Modified list of modules to pre-load. On the Windows platform, run with Perl v5.26 or later for best results. The following is a demonstration using MCE::Loop. The behaviour is similar to running a pool of workers. Workers request the manager process the next URL to process. The job_delay option gives each worker time to load any missing modules at runtime before another worker starts processing. The delay occurs one time by each worker. use strict; use warnings; use LWP::Simple; # Pre-load essential modules for extra stability. if ( $INC{'LWP/UserAgent.pm'} && !$INC{'Net/HTTP.pm'} ) { require IO::Handle; require Net::HTTP; require Net::HTTPS; } my @urls = ( 'http://hooboy.no-such-host.int/', 'http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif', 'http://www.guardian.co.uk/', 'http://www.ora.com/ask_tim/graphics/asktim_header_main.gif', 'http://www.pixunlimited.co.uk/siteheaders/Guardian.gif', 'http://www.yahoo.com', ) x 4; use MCE::Loop; MCE::Loop->init( max_workers => 8, chunk_size => 1, posix_exit => 1, ); mce_loop { my ($type, $length, $mod) = head( my $url = $_ ); print "$url is done\n"; } \@urls; MCE::Loop->finish; [download] Regards, Mario	[reply] [d/l]


Don't ask to ask, just ask
	PerlMonks