Re^2: demonstrate that perl can give back memory to the OS

2. actually return memory to the OS. This is, in general, not possible. In *nix, once a process is allocated pages, there is no mechanism for giving them back. The only exception to this is memory added via mmap. (...) mmap, however, is typically only used for very specific things, like shared-lib access

But this seems to be just what is happening (i.e. mmap/munmap):

  PID    SZ    VSZ
 5645   604  11064
  PID    SZ    VSZ
 5645  4516  14976
  PID    SZ    VSZ
 5645  2560  13020
[download]

(This is the output I get when running Joost's snippet — even on an old Linux box with a rather rusty 2.6.16 kernel.)

The last memory printout is definately about 2M smaller (the PV/string part of that scalar variable) than the previous one...

And strace shows — when I replace the system("/bin/ps",...) with a simple print "---\n" (to leave a marker in the output):

$ strace ./746953.pl
(...)
brk(0x66a000)                           = 0x66a000
read(4, "#!/usr/bin/perl\n\nsub ps {\n    #s"..., 8192) = 1003
fcntl(4, F_SETFD, FD_CLOEXEC)           = 0
write(1, "---\n", 4---
)                    = 4
mmap(NULL, 2002944, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -
+1, 0) = 0x2ba8bbdd5000
mmap(NULL, 2002944, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -
+1, 0) = 0x2ba8bbfbe000
write(1, "---\n", 4---
)                    = 4
munmap(0x2ba8bbfbe000, 2002944)         = 0
write(1, "---\n", 4---
)                    = 4
lseek(4, 196, SEEK_SET)                 = 196
lseek(4, 0, SEEK_CUR)                   = 196
close(4)                                = 0
exit_group(0)
[download]

Comment on Re^2: demonstrate that perl can give back memory to the OS Select or Download Code

Replies are listed 'Best First'.
Re^3: demonstrate that perl can give back memory to the OS by Illuminatus (Curate) on Feb 27, 2009 at 19:10 UTC
For shared-mem/mmap operations, you can't rely on ps, as every process that mmap's a file will have it included in it's total memory usage. There is really only a single copy in RAM for the OS -- it is simply mapped into different locations of different processes. When a given process munmaps a file, its 'ps' memory usage will say it is smaller, but the memory is still in use in the other processes. Only if it is the last process to have the file mapped will the memory actually be 'returned' to the OS. And, again, mmap is generally only used for shared libraries.	[reply]
Re^4: demonstrate that perl can give back memory to the OS by almut (Canon) on Feb 27, 2009 at 19:53 UTC
For shared-mem/mmap operations... What you're saying about shared mmap is correct, but I'm not sure why you think this has to do with what's going on here. The strace clearly shows MAP_PRIVATE, and also, why should that string (`$x`) be shared with other processes (like code of shared libs)? And, again, mmap is generally only used for shared libraries. I don't think so (though it depends on how you define "generally"). The three `write` lines in the log mark: (1) before allocation and (2) after allocation of `my $x = " " x 2_000_000;`, and (3) after `undef $x;`. Why would those mmap/munmaps that happen in between those markers be related to loading of shared libs, or some such? Doesn't make much sense to me. The only thing that remains to be explained (IMO) is why perl mmaps two blocks of 2002944 bytes, but only munmaps one... (When you do the same with an array, e.g. `my @x = (); $x[1_000_000] = 0; # mmaps 8003584 bytes undef @x; # munmaps 8003584 bytes` [download] the required size (to grow the array) is mmapped/munmapped exactly once each.)	[reply] [d/l] [select]
Re^4: demonstrate that perl can give back memory to the OS by zwon (Abbot) on Feb 27, 2009 at 19:30 UTC
mmap is generally only used for shared libraries AFAIK, modern malloc(2) implementations actively use mmap to allocate memory, so it's not correct	[reply]