A question of memory, mod_perl, and references and subroutines

Yohimbe has asked for the wisdom of the Perl Monks concerning the following question:

Given the following code:

#!/usr/bin/perl
use strict;

sub handler {
    my %foo=(a=>1,b=>2,c=>3);
    a(\%foo);
    print "A -> $foo{a} B -> $foo{b} C -> $foo{c}\n" ;
}

sub a 
{
    my $ref=shift;
    $ref->{a} = 100;
    my %copy=%$ref;
    $copy{a} = 10;
    my $ptr =\$ref->{b};
    $$ptr = 20;
}
[download]

What is the correct jargon for the 3 separate constructs? I think of them as references, copies and another syntax to do a reference that I think of as pointers.
Another question I have is in regards to performance and memory use:
Given that %foo starts to be 10 or 20 items, and this would be running in mod_perl, I sometimes use the my %foo = %$foo construct to make the syntax easier to type, but it does make a copy. Perl is famous for not giving memory back, so i assume that making a copy in a sub will start to gobble ram if the hash is of appreciable size. At what point does perl free ram used for my() constructs in subroutines? Does it ever? Does it keep it on the perl stack and reuse it?

Similarly, I have a mod_perl based system thats just crazy for ram use. With PerlFreshRestart On, the parents and children grow by 30 megs a day. But, strangely enough, they grow only in the shared section. private section ( difference between ram and shared size) is only about 2 megs per process. It seems that somehow PerlFreshRestart does not re-use the ram allocated for earlier copies of the libraries. I've turned off FreshRestart but it still seems to leak. I SIGUSR1 the process every night for log rotation, and this seems to be when the process memory growth occurs. Anything I can do?
--
Jay "Yohimbe" Thorne, alpha geek for UserFriendly

Comment on A question of memory, mod_perl, and references and subroutines Download Code

Replies are listed 'Best First'.
Re: A question of memory, mod_perl, and references and subroutines by Masem (Monsignor) on Apr 15, 2001 at 18:06 UTC
On the first quetion, you have the reference right, and the copy of the original data element right, but I would avoid applying the concept of pointers in perl, since they do not exist; $ptr in your code is still a reference just as much as $ref is. IMO, that last element is simply another reference. Memory is supposted to be freed once the variable is out of scope and there are no more references to that data element. So in your example above, while the memory that $ref points to will continue to live outside of sub a, %copy and $ptr will not. Now if you returned, say "return \%copy;", and assigned a variable that value in the main part of the program, %copy's data will remain around even though %copy itself is no longer defined in it's block, until that new variable is to go out of scope. With my experience with mod_perl is that you improve performance by avoiding having those large arrays and hashes which are mutable in memory, and use some sort of 'disk paging' system, whether that be through flat files, a DB, or some other means. The only data that should be persistent in a mod_perl script in order to avoid untamed memory growth are static, unchanging variables. As you've got it, it seems that you're trying to copy a hash often enough, and that might be causing a memory growth. This definitely sounds like you've got handing references to your various copies of that hash, and thus increasing the memory space. Is copying the hash necessary? This is probably the first question to ask, as PSI::ESP tells me that there's probably a better way to do it. Also, unrelated, but I've found that the module Clone is better to do any necessary duplication of non-scalar values, as opposed to trying to coerce references to work that way, particularly when they get into nested structures. Dr. Michael K. Neylon - mneylon-pm@masemware.com \|\| "You've left the lens cap of your mind on again, Pinky" - The Brain	[reply]
Memory, Perl and the OS by knobunc (Pilgrim) on Apr 16, 2001 at 18:51 UTC
Perl is famous for not giving memory back Well... it depends what you mean by that. If you have removed all references to the memory then it will be released to the Perl intepreter for reuse.1 However, I don't think that Perl interpreter gives it back to the OS for use by other processes. Remember that there are two levels of memory pools: the OS and Perl. So when your program needs some memory Perl will check its internal pools and see if it has any it can allocate. If there is no memory (of the right size, etc.) kicking around then it will ask the OS for more. When there are no more references to a piece of memory left then Perl will notice the reference count has hit 0 and add it to Perl's internal free list. It will not give it back to the OS for use by other programs. Note that this behavior is common to most programs since malloc and free manipulate the internal pools of a program and ask the OS for more but do not release it back until the program ends. On UNIX the brk and sbrk call are the system calls that actually manipulate the memory that a program can see. However, these only allow you to shrink or grow the heap at the end. So if you have 1 large array then allocate another small integer after it on the heap even if the large array is freed your program could not shrink the heap until the integer goes away (or if the program compacts itself). I don't think Perl compacts the heap since it probably assumes that if you allocated memory once you are likely to need it again. Most of the time this is a reasonable assumption and pays off for performance. -ben 1 - Although things are sometimes not freed when you think they should be. Some loop variables are kept around if Perl thinks the loop is likely to happen again.	[reply]