zeni has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

What is the solution for "Out Of Memory" error? I have a main perl package which calls functions of 7 other packages.

One function of a particular package creates thousands of combinations of values using 'glob'. Sometimes combinations can be few lakhs also.

Pls help.

Replies are listed 'Best First'.
Re: "Out Of Memory" error
by BrowserUk (Patriarch) on Sep 02, 2009 at 06:25 UTC

    Almost all uses of glob can be replaced by other fairly simple substitutes that consume less memory, but you'll have to show what you (the module) is using it for. Examples of inputs and corresponding outputs to that function would probably get you a solution to the problem.

    BTW: Does "lakh" mean 100,000?


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      Hi, i have replied to the above monk regarding the glob usage. Pls refer to it.

Re: "Out Of Memory" error
by ikegami (Patriarch) on Sep 02, 2009 at 05:50 UTC

    I wonder if you might be running out of C stack space. I believe glob builds all the results up front, even if you use the iterator form. The solution would be to write an alternative to glob that generates the results one at a time or an alternative that generates the results in Perl memory rather than on the C stack.

Re: "Out Of Memory" error
by stevieb (Canon) on Sep 02, 2009 at 06:01 UTC

    Providing information regarding the problem may help

    "Out of memory" errors generally refer to two possible problems: the Random Access Memory (RAM) in the computer that was responsible for the execution of the Perl program became exhausted, or the memory of the user who executed the program became exhausted. ;)

    Which Perl program led to the error/warning? What type of system are you running the program on? What is the name of the function that creates "thousands of combinations of values using 'glob'", and which package does that function originate from?

    Steve

      Hi Steve, There is this function called genCombinations which generates several combinations for a set of values using glob. Ex:

      f1= (1,2) f2 = (3,4,5)

      No. of unique combinations = 6

      1,3 1,4 1,5

      2,3 2,4 2,5

      Fields can be several tens and they can also have a range of values like:

      f1 = range(0 to 100)

      f2 = values(2,5)

      In such case no. of unique combinations will be 200. So i used a glob to create these combinations.

      @arr = glob{ (0 to 100),(2,5) };

      How else can i create these combinations so that the Out Of Memory error vanishes!