BrowserUK's solution was not to fork each globbed directory, but to create one large glob of all the directories. This is what I am claiming is not feasible.

I admit that I thought the enormous sz from ps in each child processes I forked came from its own instance of the Perl intrepreter, but I never claimed that HPUX didn't use copy-on-write.

My problem is I have no way of profiling it - how can I tell how much memory is really being used and how much is shared.

I have thought of a few more ways to optimize speed and memory allocation of the original code, but that won't get rid of the overhead I mentioned by my example of just a simple script. 

#!/usr/bin/perl -w
use strict;
while (1) {
  print "I am only printing and sleeping\n";
  sleep 1;
}

[download]

That tiny program shows up in ps with a sz comparable to my full blown script.

If I can't tell how much of that is shared by fork'ing another process - I have no idea if the project is viable or if it should be scrapped.

Now your proposal is a tad bit different from the others as your forked children die returning all memory to the system, they are just spawned each iteration, which means the memory is MORE available (during sleep) to the system and since all the variables will be pretty stagnant once the child is forked - it won't start getting dirty before its dead. This is food for thought.

Thanks and cheers - L~R

In reply to Re: Re: Re: Re: 3 weeks wasted? - will threads help? by Limbic~Region
in thread 3 weeks wasted? - will threads help? by Limbic~Region

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.