UPDATE: I haven't confirmed this yet, but xdg mentioned that versions 5.8.1 up to (but not including) 5.8.3 have a memory leak in RegEx. It seems likely that this is my problem.

I'll update after I've upgraded and tested my code.

--------------------------------------------------------

Hello folks. I'm exhausted, so please forgive me if this isn't well presented:

I have two test files. A data file of about 150,000 lines and a temp file with about 800 lines, each containing 3 values to use in my search expression (I only use two of them).

This code is intended to perform the substitutions and insertions, then save out the file to a new location.

Unfortunately, it consumes all of my physical memory in a few seconds, then chugs along until windows cuts it off at about 2GB and then kills it.

Although the code isn't as pretty or concise as it could be, is there anything here that can be changed to prevent this gross overuse of memory? It's as if the program is making a complete copy of the data for every iteration.

#! perl

open (XAPFILE, 'C:\Documents and Settings\Nick\Desktop\040408 Work Fil
+es\stranger.xap') or die;
my $file = join ("",<XAPFILE>);
my ($Cue, $Sound, $Pri);
my $r_file = \$file;

open (PRIFILE, '<C:\temp.txt') or die;
open (OUTFILE, '>C:\stranger.xap') or die;

while (<PRIFILE>){
    {
    ($Cue, $Sound, $Pri) = split ("\t");
    chomp ($Pri);
    if ($$r_file =~ s/(Sound\s*\{\s*
             Name\ =\ $Sound;\s*
            Priority\ =\ )([\d\D]*?);/$1$Pri;/xm){
        print "Update $Sound > $Pri \n";

        #print "'$1$Pri;'\n";
    }elsif ($$r_file =~ s/(Sound\s*\{(\s*)
             Name\ =\ $Sound;\s*$)/$1$2Priority\ =\ $Pri;\n/xm){      
+   
        #print "$1$2Priority = $Pri;\n";
        #print "1'$1'\n2'$2'\n3'$3'\n4'$4'\n";
        print "Add $Sound > $Pri \n";
    }else{
        print "ERROR $Cue > $Sound > $Pri \n";
    }
    }
}

print OUTFILE $file;

[download]
Any thoughts would be greatly appreciated. I've been trying to teach myself references, OO, modules, etc. to make a tool. That's been my life for the past 4 days. This is just a hack to get the task done on my deadline, and I'm at my wits end trying to make it finish.

FYI - The data that is found is never more than four lines long, and each line is only a few dozen characters at most. I don't care if it takes 10 minutes to process, I just want it to finish.

Thanks

Nick

In reply to RegEx on 4MB file consumes of 2GB of ram before windows shuts it down (Memory Leak in 5.8.2) by Ardemus

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.