I just got done tracking down a nasty memory leak due to my use of natatime(). It's not a function I've ever used before but it seemed convenient enough.

Run this code and watch the memory use grow: 

use List::MoreUtils qw(natatime);

my $count = 0;
while (1) {
    my @rows = ("...") x 100;
    my $it = natatime(10,@rows);
    while(my @chunk = $it->()) {
        $count += @chunk;
        if ($count % 1000 == 0) {
            warn "COUNT : $count\n";
        }
    }
}

[download]

I haven't dug any deeper than this, so I'm not sure if it's something in List::MoreUtils XS code or a side-effect of using a closure for the iterator, or what. I'm also going to send this to the maintainer, so no need to remind me!

UPDATE: It would seem to be an XS bug - setting the environment var LIST_MOREUTILS_PP=1 makes it go away.

UPDATE 2: Corion asked, so I'll tell you - this is List::MoreUtils v0.22 on Perl v5.8.8. Oh, and the maintainer's email address doesn't work. Anybody want to take over and fix this?

-sam

Replies are listed 'Best First'.
Re: Beware List::MoreUtils natatime()!
by moritz (Cardinal) on Feb 09, 2009 at 09:21 UTC
    Others who are interested in List::MoreUtils' bugs will probably look at the bug tracker, so maybe you should open a ticket there.

    And FYI, there are already two other reports about memory leaks in that queue.

[reply]
Re: Beware List::MoreUtils natatime()!
by JavaFan (Canon) on Feb 09, 2009 at 10:17 UTC
    It would seem to be an XS bug - setting the environment var LIST_MOREUTILS_PP=1 makes it go away.
    Not for me: 
    $ LIST_MOREUTILS_PP=1 perl foo
String found where operator expected at (eval 1) line 206, near "Carp:
+:croak "each_arrayref: argument is not an array reference\n""
        (Do you need to predeclare Carp::croak?)
String found where operator expected at (eval 1) line 226, near "Carp:
+:croak "each_array: unknown argument '$method' passed to iterator.""
        (Do you need to predeclare Carp::croak?)
Use of inherited AUTOLOAD for non-method List::MoreUtils::natatime() i
+s deprecated at foo line 14.
Can't locate auto/List/MoreUtils/natatime.al in @INC (@INC contains: /
+opt/perl/lib/5.10.0/i686-linux-64int-ld /opt/perl/lib/5.10.0 /opt/per
+l/lib/site_perl/5.10.0/i686-linux-64int-ld /opt/perl/lib/site_perl/5.
+10.0 /opt/perl/lib/site_perl/5.8.8 /opt/perl/lib/site_perl .) at foo 
+line 14

    [download]
    It runs (with the memory leak) if I don't set LIST_MOREUTILS_PP.
[reply]
[d/l]

      Your bug sounds like your version is depending on an accident of not-quite recent perls. It used to often "just work" to say Carp::croak '...';. That was accidental and whenever you see an exception like this, just make sure to load Carp in advance of loading the thing you're using.

      In your example, do:

      $ LIST_MOREUTILS_PP=1 perl -MCarp foo
[reply]
[d/l]
[select]
        Looking into the source of List::MoreUtils, I find: 
        require Carp;
Carp::croak "some message";

        [download]
        That only works if Carp was already loaded. In older versions of Perl, there was a common pragma (vars?) that loaded it, but that no longer is loaded in 5.10.

        Hence the error. And another reason to not use List::MoreUtils.

[reply]
[d/l]

Back to Meditations