Are there any tips on efficient memory usage?

Perl is a compromise. Whenever there has been a tradeoff between speed and memory the usual response has been to use more memory.

Depending on your application the obvious solution is to leave as much data on disk as possible. Tie::Hash is one possible solution as is using a database for the data and using the inbuilt database functionality to do as much pre-processing as possible.

Although this is pretty basic stuff I will just mention it is much more memory efficient to do stuff like this: 

while(<FILE>) { do stuff }
# rather than
@data = <FILE>;
for (@data) { do stuff }

# pass data to and from subs as a reference (effectively a pointer)
# this saves making duplicate copies of data structures
my $data_ref = process_data(\@data);
sub process_data {
    my $ref = shift;
    for (@$ref) { do stuff }
  return $ref;
}

[download]

Perl also has the -i inplace edit fuction which may be useful. There may also be value in actively undefing a data structure so it can be garbage collected if your code has finished with it. The Devel:: range of modules provide lots of insight into speed/memory size in a program. If you are using grep (especially in scalar context) you should be aware that it will build a complete temp array and iterate through the entire data set... 

# this is short but memory intensive
do_stuff() if grep { /something/ } @array
# compared to this which is faster and uses less memory
# but takes 4 lines to write instead of one....
for(@array) {
    next unless /something/;
    do_stuff();
  last;
}

[download]

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

In reply to Re: How do I dumping Perls memory usage? by tachyon
in thread How do I dumping Perls memory usage? by jaa

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.