Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: Crash with ForkManager on Windows

by kcott (Archbishop)
on Sep 21, 2017 at 08:45 UTC ( [id://1199811]=note: print w/replies, xml ) Need Help??


in reply to Crash with ForkManager on Windows

G'day amitsq,

I suspect your problems are related to the additional $pm->finish and next statements.

Your instantiation of the Parallel::ForkManager object may also be an issue. What you have is different from every example shown in the documentation; also see "perlobj: Invoking Class Methods" (including its "Indirect Object Syntax" subsection).

Using exactly the same message to identify your different cases is pointless: use meaningful messages.

I rewrote your code like this (pm_1199760_parallel_forkmgr_link_checker.pl):

#!/usr/bin/env perl -l use strict; use warnings; use LWP::Simple; use Parallel::ForkManager; my @urls = qw{ http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif http://hooboy.no-such-host.int/ http://www.yahoo.com http://www.ora.com/ask_tim/graphics/asktim_header_main.gif http://www.guardian.co.uk/ http://www.pixunlimited.co.uk/siteheaders/Guardian.gif }; my $max_processes = $ARGV[0] || 8; my $pm = Parallel::ForkManager::->new($max_processes); print 'Start link checking.'; for my $url (@urls) { $pm->start and next; my ($type, undef, $mod) = head($url); if (defined $type) { print "PASS: $url"; print ' MOD: ', $mod || 0; } else { print "FAIL: $url"; } $pm->finish; } $pm->wait_all_children; print 'Link checking completed.';

Here's the output with no argument:

$ pm_1199760_parallel_forkmgr_link_checker.pl Start link checking. FAIL: http://hooboy.no-such-host.int/ PASS: http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif MOD: 1354074654 PASS: http://www.guardian.co.uk/ MOD: 0 FAIL: http://www.ora.com/ask_tim/graphics/asktim_header_main.gif PASS: http://www.yahoo.com MOD: 0 FAIL: http://www.pixunlimited.co.uk/siteheaders/Guardian.gif Link checking completed.

I then ran it with arguments of 8, 5, 3, 2 and 1. Lke the first run (with no argument), all ran to completion: no crashes! With the exception of 1, they produced the same results (although the order of output varied).

Running with an argument of 1 rather defeats the purpose of running processes in parallel; however, as you said you'd used it, I tried it also. Here's what I got:

$ pm_1199760_parallel_forkmgr_link_checker.pl 1 Start link checking. PASS: http://us.a1.yimg.com/us.yimg.com/i/ww/m5v9.gif MOD: 1354074654 FAIL: http://hooboy.no-such-host.int/ PASS: http://www.yahoo.com MOD: 0 FAIL: http://www.ora.com/ask_tim/graphics/asktim_header_main.gif FAIL: http://www.guardian.co.uk/ FAIL: http://www.pixunlimited.co.uk/siteheaders/Guardian.gif Link checking completed.

I ran that a second time: the result was the same. I'll leave you to investigate that further.

— Ken

Replies are listed 'Best First'.
Re^2: Crash with Parallel::ForkManager on Windows
by Discipulus (Canon) on Sep 21, 2017 at 09:27 UTC
    Hello kcott,

    where are you running your code?

    Unfortunately on Perl (v5.14.2) built for MSWin32-x86-multi-thread on win7 and Parallel::ForkManager 1.19 I get more than 50% of crashes, using your code.

    Testing more times mine, as posted above, I get ~25% of crashes.. (not saying mine is better: better is no crash at all ;=)

    Unfortunately windows is a shaggy beast and I'm not able to debug this behaviour further: what I can add is that during installation I got some warning about disappeared PIDs even if the overall testing process ended ok:

    t/00-report-prereqs.t ..... ok t/01-utf8-all.t ........... skipped: This is a bug in perl itself on W +indows t/02-callback.t ........... # This test can take 10-20 seconds, please + wait. Started at Thu Sep 21 11:23:08 2017 t/02-callback.t ........... ok t/03-callback-data.t ...... # This test can take 2-6 seconds, please w +ait. Started at Thu Sep 21 11:23:25 2017 t/03-callback-data.t ...... ok t/basic-methods.t ......... ok t/changing-pids.t ......... ok t/start_child.t ........... ok t/waitpid-conflict.t ...... # code outside of P::FM stole -1152 child process '-1152' disappeared. A call to `waitpid` outside of Para +llel::ForkManager might have reaped it. t/waitpid-conflict.t ...... ok t/waitpid-waitonechild.t .. # code outside of P::FM stole -5456 # found warning: child process '-5456' disappeared. A call to `waitpid +` outside of Parallel::ForkManager might have reap ed it. t/waitpid-waitonechild.t .. 1/? # expected to find warning: (?^:child +process '\d+' disappeared. A call to `waitpid` out side of Parallel::ForkManager might have reaped it\.) t/waitpid-waitonechild.t .. ok t/waitpid_blocking.t ...... ok All tests successful.

    L*

    PS modified the title a bit for better later search

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

      G'day Discipulus,

      Thanks for the follow-up. While it's clear the OP was using Windows, I wasn't aware that this was a Windows-specific problem: I don't see anything like "This worked on XYZ, but not on Windows".

      I'm using:

      Perl 5.26.0 (built for darwin-thread-multi-2level) Parallel::ForkManager 1.19 LWP::Simple 6.26

      I had to install Parallel::ForkManager and the cpan output is still on one of my screens. Here's my equivalent lines to those you posted:

      t/00-report-prereqs.t ..... ok t/01-utf8-all.t ........... skipped: Need utf8::all for this test cras +hing on Windows # This test can take 10-20 seconds, please wait. Started at Thu Sep 21 + 17:07:29 2017 t/02-callback.t ........... ok # This test can take 2-6 seconds, please wait. Started at Thu Sep 21 1 +7:07:45 2017 t/03-callback-data.t ...... ok t/basic-methods.t ......... ok t/changing-pids.t ......... ok t/start_child.t ........... ok # code outside of P::FM stole 2369 child process '2369' disappeared. A call to `waitpid` outside of Paral +lel::ForkManager might have reaped it. t/waitpid-conflict.t ...... ok # code outside of P::FM stole 2371 t/waitpid-waitonechild.t .. ok t/waitpid_blocking.t ...... ok All tests successful.

      You appear to have had problems with t/waitpid-waitonechild.t that I didn't have; everything else looks fairly similar.

      To be honest, I started the installation, went and made a coffee, came back and saw:

      YANICK/Parallel-ForkManager-1.19.tar.gz /usr/bin/make install -- OK cpan[2]>

      I rarely review the (often many screenfuls of) output unless the installation failed.

      — Ken

        I am getting the bogus tests too, yet I can run this and get no crashes at all.
        Win 8.1 64-Bit, ActivePerl 5.24 built for MSWin32-x64-multi-thread


        holli

        You can lead your users to water, but alas, you cannot drown them.
Re^2: Crash with ForkManager on Windows
by amitsq (Beadle) on Sep 21, 2017 at 10:18 UTC
    Taking your code, i get always the same output regardless which param i take. But it crashes too.

      Feedback, so far, is showing crashes on Windows Perl versions 5.14 and 5.16; however, 5.24 on Windows did not crash.

      "CPAN Testers Matrix: Parallel-ForkManager 1.19" is showing failures (for "mswin32") up to 5.22.3, but no failures for 5.24.0 or higher. There's virtually no failures for any other platforms: two very old "linux" ones for 5.8.x; and one other for "cygwin" 5.22.2 (which I believe only runs on Windows anyway).

      You should probably dig a bit deeper into this than I have, but problems do not appear to be solely related to ActivePerl; while looking around I came across this GitHub bug report: "Parallel::Forkmanager 1.19 cpan install fails on Windows Server 2016 Strawberry Perl 5.16.3". (Don't take that one report as being in any way conclusive.)

      Obviously, installation failures are not the same as crashes, but a trend is suggested. Again, something for you to look into further.

      I'd suggest installing a newer Perl (at least 5.24.0; but 5.26.0 — the highest stable version — would be better) and trying the code again with that. Also ensure you've got the latest versions of the modules you're using.

      — Ken

        Hello kcott and all,

        I can confirm that Strawberry Perl portable 5.24.1.1 32bit on win7 64bit and Parallel::ForkManager 1.19 run the original code and mine without any crash. Good to know.

        L*

        There are no rules, there are no thumbs..
        Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1199811]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (9)
As of 2024-03-28 14:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found