This is a follow on from a previous post Out of memory., which resulted in not finding a way to handle a problem within memory. The program aborts with either a core dump or an "Out of Memory Error!"

The PERL was ran on Linux (Redhad), Solaris, Windows XP, with 4-16 gigs of memory using various builds of PERL 5.8 and 5.10 with the same result.

Strategically, if a problem is too big to model in RAM is using DBI:DBD the fastest and simplest way to store data outside of RAM? I am assuming from a high level there are only a few methods are available. Also, is there in Windows a way to detect how much heap a perl program has? (I don't think this is a global value.)

The goal is to convert "if-then-else" statements in a 4th GL scripting language into equations. For example "if(a) {}elsif (b) {} else {}" would becomes (a+b+c).

Once accomplished all paths through the 4th GL script can be found by "multiplying" the equations through. For example, two simple if/else statements could be represented as (a+b)(c+d.) The independent paths throught the code is: ac+ad+bc+bd.

The code provided here does this equation transformation.

The difficulty is that the equation is 64K in characters and will result in a couple million independent elements as a result. I'd like to be able to run the program on windows as well as UNIX, but need a way in windows to detect how much heap is available for the program. (I think a general how much memory is on the system will not work since the "Out of Memory!" problem will still occur. I could be wrong though.) My thought is that a database is the best way to externalize and keep the speed up.

The data is available here: Out of memory.

From the last post (the latest in-memory attempt was: 
$|=1;

sub multiply {
    my( $ref1, $ref2 ) = @_;
    my @temp;
    for my $r1 ( @{ $ref1 } ) {
        for my $r2 ( @{ $ref2 } ) {
            push @temp, $r2 . $r1;
        }
    }
    return \@temp;
}

$/ = ',';
my @stack;

open POSTFIX, '<', 'postfix.txt' or die $!;

while( my $op = <POSTFIX> ) {
    chomp $op;
    chop $op if $op =~ tr[\n][\n]; ## remove any trailing newline

print "MULTIPLY\n";
    if( $op eq 'x' ) {
        push @stack, multiply( pop( @stack ), pop( @stack ) );
    }
print "ADD\n";
    elsif( $op eq '+' ) {
        push @stack, [ @{ pop @stack }, @{ pop @stack } ];
    }
    elsif( $op =~ m[^\d+$] ) {
        push @stack, [ pack 'v', $op ];
    }
    else {
        die "Bad '$op' at position;" . tell( POSTFIX );
    }
}

print "stacksize: " . @stack;

for my $item ( @stack ) {
    printf "%s", join ',', unpack 'v*', @{ $item }[ 0 ];
    for my $group ( @{ $item }[ 1 .. $#$item ] ) {
        printf "+%s", join ',', unpack 'v*', $group;
    }
    print "\n";
}

[download]
However, it ran out of memory as well. Any insights in regards to approach is appreciated.

In reply to Out of Memory 2. by dneedles

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.