in reply to Re: Allocating more memory to perl process...
in thread Allocating more memory to perl process...

Thanks Elian

As I said above, memory usage is definitely the issue here. My wimpy 250 MB of memory is not cutting it here and the process crashes out on me. So my options are to get more memory or to reimplement the algorithm in an iterative fashion. Actually, there are more options than those, but I'd rather not go there. ;)

Bamafan
  • Comment on Re: Re: Allocating more memory to perl process...

Replies are listed 'Best First'.
Re: Re: Re: Allocating more memory to perl process...
by Elian (Parson) on Nov 13, 2002 at 02:06 UTC
    Well, there's always more swap, but that's an option of limited utility. Perl data structures are reasonably large (I'll put in the obligatory plug for Devel::Size here) and it's pretty easy to chew up a lot of memory quickly.

    An iterative algorithm is probably more in order if you're blowing memory or, if this is a program that works on the same set of data over multiple runs, you might want to consider something more persistent and less memory hungry, such as a database, that you can connect to and only process the data you need to.

    (Or, if this is math heavy, consider something like PDL which can represent a lot of numeric data densely)

Re: Re: Re: Allocating more memory to perl process...
by seattlejohn (Deacon) on Nov 13, 2002 at 02:02 UTC
    Are you sure you've implemented the recursive algorithm right in the first place? The fact that you say you're not moving that much data around -- yet you have recursion more than 100 levels deep -- makes me suspicious that you might inadvertantly have created the recursive equivalent of an infinite loop. 250MB is actually quite a lot of memory for most text-ish material.

    One way to get a sense of what's going on in your recursion is to print the values of arguments each time the sub is entered. You may discover that your base case isn't returning properly, or that you're not decomposing the problem the way you thought you were, and thus not making progress with each recursive call.

    Apologies if this is obvious stuff that you've tried already, but I'd be wary of jumping to the conclusion that this is a perl or system issue before you're confident that the algorithm itself is correct.

            $perlmonks{seattlejohn} = 'John Clyman';