Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Memory leak in map function?

by ezekiel (Pilgrim)
on Aug 20, 2003 at 05:04 UTC ( [id://285100]=perlquestion: print w/replies, xml ) Need Help??

ezekiel has asked for the wisdom of the Perl Monks concerning the following question:

Hello all

I have the following two pieces of code that I thought were equivalent. However, the first runs smoothly, while the second (using map) progressively chews up memory until it crashes. Can someone explain the difference and what I need to do to get the second to work?

I am also under the impression that the second method should be faster. Is that true?

First method:

foreach (@objects) { $module->target($_); my $results = $module->process(); undef $results; }

Second method:

map { $module->target($_); my $results = $module->process(); undef($re +sults) } @objects;

Thanks for your help.

Replies are listed 'Best First'.
Re: Memory leak in map function?
by blokhead (Monsignor) on Aug 20, 2003 at 05:19 UTC
    In your second example, you're using map in void context, which is generally considered a no-no. In general, foreach is for general-purpose looping, and map is more of a special-purpose type of foreach just for generating lists. You don't seem interested in generating another list as a result of this loop, so your first example is at least semantically clearer.

    My hunch is that, even though map is in void context, it is still generating the return-list that is scalar(@objects) elements large. In your case, every element of this anonymous list is undef due to the last line of the block. To test this hunch, change the code to the following:

    map { $module->target($_); my $results = $module->process(); (); } @objects;
    .. so that the return-list never gets added onto. See if that brings performance in line with the foreach (not that I'm encouraging you to stay with the void-context map here).. <runs off to benchmark...>

    I am also under the impression that the second method should be faster. Is that true?

    I highly doubt that, but don't know enough about the internals to say one way or another. Clearly (from your experience) in void context, map seems to have the extra overhead of maintaining the return-list.

    PS: undef $results is not needed, as $results falls out of scope at the end of the block each time. But you probably just had that in to double-check your suspicion of a memory leak..

    Update: Yep, the benchmark showed that there is a significant improvement by having (); as the last statement of the void-context map BLOCK:

    use Benchmark 'cmpthese'; + my @objects = (1 .. 100_000); + cmpthese( -1, { map1 => sub { map { my $xyz = $_ + 1000; undef $xyz; } @objects; 1; +}, map2 => sub { map { my $xyz = $_ + 1000; (); } @objects; 1; +}, for => sub { for (@objects) { my $xyz = $_ + 1000; undef $xyz } 1; +} }); __END__ Benchmark: running for, map1, map2 for at least 1 CPU seconds... for: 1 wallclock secs ( 1.10 usr + 0.01 sys = 1.11 CPU) @ 11 +.71/s (n=13) map1: 1 wallclock secs ( 1.08 usr + 0.00 sys = 1.08 CPU) @ 7 +.41/s (n=8) map2: 2 wallclock secs ( 1.06 usr + 0.00 sys = 1.06 CPU) @ 10 +.38/s (n=11) Rate map1 map2 for map1 7.41/s -- -29% -37% map2 10.4/s 40% -- -11% for 11.7/s 58% 13% --
    .. My simple for loop was faster than both, but please avoid map in void context for semantic reasons alone.

    blokhead

Re: Memory leak in map function?
by Zaxo (Archbishop) on Aug 20, 2003 at 05:24 UTC

    Without knowing your $module instance's methods or the size and nature of @objects, it's hard to say why your memory runs away. Where does $module come from?

    map builds a list of its results, which is why it is inadvisable to use it in void context as you have. Undefining $results ought to be unnecessary in the first case, and may be damaging in the second. map's block should end with the expression you want to have in the list it produces.

    The for loop is probably what you want to do here. It is usually slower to call map because you need to build a list of all the results.

    What happens if you say this?,

    my @results = map { $module->target($_); $module->process(); } @objects;

    After Compline,
    Zaxo

      map builds a list of its results, which is why it is inadvisable to use it in void context as you have.

      The fact that map does so in void context, and hence isn't paying attention to context, is a bug in the implementation of map. This has been known for eons, but noone ever cared enough to provide a patch.

      Abigail

Re: Memory leak in map function?
by TomDLux (Vicar) on Aug 20, 2003 at 05:19 UTC

    map generally takes simple expressions in the braces.

    Why do you assign the return value of process() to a variable, if you don't do anything with it? Just invoke it as a routine:

    $module->target($_); $module->process();

    As for the memory leak, I suspect my is generating new variables for each array element, which aren't being released until map finishes. Try encapsulating the map code into a subroutine.... that will make fix the leak, I bet.

    --
    TTTATCGGTCGTTATATAGATGTTTGCA

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://285100]
Approved by blokhead
Front-paged by broquaint
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (3)
As of 2024-04-20 01:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found