othane has asked for the wisdom of the Perl Monks concerning the following question:

Hey guys, being a relative beginner to perl this might be an easy one for most of you, but here we go: I'm using Algorithm::SVMLight to classify some very large input (billions of lines). The problem is that memory usage is steadily growing, although it should not be.

Here's a snippet:
my $svm = new Algorithm::SVMLight; $svm->read_model($myconfig->{'model_file'}); while (<>) { $svm_res = $svm->predict_i(\@indices, \@values); }

Basically memory usage keeps on going up with each call of the predict_i subroutine. Obviously I have other things in my loop, but If I comment it this subroutine, there's no increase in memory usage, so it has to be the source of the problem. Can I prevent this problem without actually changing the subroutine itself? Any help or comment is much appreciated.

Replies are listed 'Best First'.
Re: memory usage keeps on growing
by roboticus (Chancellor) on Dec 04, 2013 at 12:41 UTC

    othane:

    Possibly, but to do so, you'd need to know where that memory is--i.e., what variables are holding it. Then you could clear the data. Perhaps simply deleting and rebuilding $svm may be enough to clear the memory for you. But without knowing what the subroutine is doing, I couldn't offer you more help than that.

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

      Thanks, as you suggested, rebuilding $svm is the only simple solution at the moment, as ugly it is.