Find total "actual" memory usage of forked processes

suaveant has asked for the wisdom of the Perl Monks concerning the following question:

Is there a programmatic way to actually figure out the total memory footprint of a group of forked processes. If you add up all the memory sizes of 40 instances of a script I have you get 6.5GB, but examining total memory usage via top I only see usage increasing somewhere along the lines of 100MB.

I realize that some memory is shared between forked processes but is there any good way to measure that?

- Ant
- Some of my best work - (1 2 3)

Comment on Find total "actual" memory usage of forked processes

Replies are listed 'Best First'.
Re: Find total "actual" memory usage of forked processes by kyz (Acolyte) on Aug 14, 2009 at 14:34 UTC
It depends on your operating system more than Perl. Linux 2.6.14 introduced the `/proc/pid/smaps` magic file, which counts up "private" and "shared" memory usage, as well as "clean" and "dirty". In this case, you want to know the amount of "private, dirty" pages you have. You can use the existing CPAN module `Linux::Smaps` to read these `smaps` files.	[reply]
Re: Find total "actual" memory usage of forked processes by cdarke (Prior) on Aug 14, 2009 at 15:05 UTC
Unix::Getrusage which includes a getrusage_children. It includes items such as: maximum resident set size, integral shared memory size, integral unshared data size, integral unshared stack size, page reclaims, page faults, swaps.	[reply]
Re: Find total "actual" memory usage of forked processes by pingo (Hermit) on Aug 14, 2009 at 14:36 UTC
Not much of an answer, but be aware that measuring memory usage is not exactly trivial: http://www.kdedevelopers.org/node/1445	[reply]
Re: Find total "actual" memory usage of forked processes by JavaFan (Canon) on Aug 14, 2009 at 14:52 UTC
The problem is, what do you mean by "actual" memory usage? Does that include shared memory? Does that include memory used by the program itself? If you fork(), on a modern operating system, memory pages are only copied if either process changes them. How do you count them? Do you include memory swapped out? How about claimed (by the OS), but not used memory? What about file buffers for files opened by the processes involved? Regardless of the answers of the questions above, you're probably best of using tools geared towards the OS - AFAIK, none of them are written in Perl. Some commercial Unices have good tools available.	[reply]
Re: Find total "actual" memory usage of forked processes by bpoag (Monk) on Aug 14, 2009 at 16:07 UTC
Most OS'es (at least on the Unix side of the fence) have commands that list out active/inactive shared memory segments and message queues, and which PIDs have a hand in them. That would probably be where i'd start. You'll need to cross-reference that against whatever processes happen to be running at the time. Google is thy friend, brother.	[reply]