Yes sir I have! There's a _check() routine in the script which runs by default after every 50th child to check to see if all the kids it thinks are alive, are in fact still alive, in case one manages to evade the reaper! That process as well as the overhead of checking the environment for safety seem to delay the process just enough to not fork() a billion processes at once. Of course, I could see that this might pose a problem down the road on some systems. I'm definately going to address this in a more permanent way by removing the current call to '/usr/bin/uptime' with something more reliable and more understanding. Granted, there will be a certain over head associated with determining the current load/mem/cpu usage before every fork() call, but the safeguards it'll provide should more than pay off. Additionally, I'll probably provide a mechanism to forego the safety net, because sometimes, I want enough rope to hang myself.

Its difficult to really provide a decent temporary solution for this problem as my children might only use 1% mem and 5% CPU, where as someone elses might use 54% mem and 70% CPU during processing. I suppose, it might help to profile the children as well. If we're processing on a big list, we might only fork() 5-10 processes for the first 60 seconds, and gather information on their peak performance, and be able to make an educated guess as to the resources these children will use on the system, and be able to dynamically adjust the number of concurrent processes based on that data.

I have a ton of ideas to make this the easiest to use, and most flexible process controller for Perl. Granted, poor coding in the children will almost always blow up in your face, I'm just aiming to make it harder to do so.

-brad..

In reply to Re: Re: My First Submission to CPAN (Parallel::ForkControl) by reyjrar
in thread My First Submission to CPAN (Parallel::ForkControl) by reyjrar

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.