The villain here is probably the %main_data hash, which is filling up with all your fetched data. So it's not when you get it from the DB, it's when you squirrel it away for use in your program.

The minimal mutilation solution is to do what OS's do to extend their memory: use the hard drive. You can use the MLDBM module which allows you to store complex Perl data structures on disk and manipulate them transparently (when you tie the data structure).

Not completely related to that suggestion: something that might help you make this stretch of code more maintainable: use a hash slice and an array slice instead of those multiple assignments: 

@fields = qw(Resd Edlv); # just a sample
@main_data{curpidm}{@fields} = @data[1,2];

[download]

Update gotta check that, not sure it will work as advertised ...

Second update it won't ... the handy @hashname shorthand is short-circuited by the fact that we've got a reference here. So, unless somebody else figures out the syntax, disregard this suggestion.

HTH

Philosophy can be made out of anything. Or less -- Jerry A. Fodor

In reply to Re: Re: Re: Out of memory! message by arturo
in thread Out of memory! message by Dude

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.