Re^5: Memory management

First, if you want to look more at Windows internals using a Windows app, I would recommend, TaskInfo: http://www.iarsn.com/taskinfo.html. No I don't get any money for this recommendation.

Anyway with TaskInfo you can see how many files and exactly which ones a process has open and details like that which aren't available in the Windows Task Manager (what's paged in/out, etc). Doing advanced things like dynamically lowering the priority of a thread within a process is possible! Whoa!

So anyway on to your question with some code!
I wrote this "quickie" thing...

Run with std Windows TaskManager open and this Perl program in the command window so that you watch both at the same time.

The Windows Task Manager is a bit weird. But this will show a clear spike in memory usage. Then you see the point that the @list has nothing in it anymore, but that won't change windows process usage. Then finally Perl ends and level drops back to what it was before. As Windows background things come and go (like virus software, there are variations, but this @list thing is big enough that the results should be clear.

#!/usr/bin/perl -w
use strict;
use Data::Dumper;

my @list;

populate_list(\@list,10000000);
countdown(10);

@list = (); #list has nothing in it now
countdown(10);

sub populate_list
{
   my ($lref,$num) = @_;
   foreach my $x (0..$num)
   {
      push (@$lref, $x);
   }
}

sub countdown
{
   my $num = shift;
   while ($num--){sleep(1);print "countdown $num\n";}
}
[download]

Comment on Re^5: Memory management Download Code

Replies are listed 'Best First'.
Re^6: Memory management by Anonymous Monk on Aug 14, 2009 at 12:31 UTC
I find http://sysinternals.com ProcessExplorer (and handles/listdlls...) more useful.	[reply]
Re^7: Memory management by Marshall (Canon) on Aug 14, 2009 at 12:53 UTC
Thanks for the post! MS does provide a number of utilities free of charge. Some of them work great and some not. If you know of other MS stuff, please post URL's. One subject that comes up fairly often are automation tools for Win GUI apps. If you have some suggestions along that line, they will get used often. This forum is about Perl. But a very common use of Perl is in sys admin type functions. Some Monks consider this "off topic" but I don't. Because sys admin is an excellent and common use of Perl. To talk about things that Perl could potentially use is, in my opinion "on topic".	[reply]
Re^6: Memory management by desemondo (Hermit) on Aug 16, 2009 at 11:17 UTC
Cheers Marshall - I too use perl to do a lot of sys admin type stuff. Lately though i've been doing more graphing. I'll give this a try tomorrow, with taskinfo or process explorer.	[reply]
Re^7: Memory management by Marshall (Canon) on Aug 18, 2009 at 16:37 UTC
Have fun! The memory effect showed up pretty clearly when I made the array big enough. BTW, what graphics drawing packages have you been having success with? I've got some requirements for things along those lines. One of the reasons to "hold onto memory" once you get it is that the OS memory allocator is "expensive" in terms of CPU. Often when making a memory region allocated to you either bigger or smaller, the OS will get another contiguous hunk of the right "new size" and then copy all the old stuff into the new area! All this copying around can get really expensive! Perl has to do extra work to keep track of memory management, but at the end of the day this works out to be very much more performance efficient. I've run some tests in C with gcc on my WinXP platform to just to experiment with this although I don't think it appropriate to post C code here. But Perl is written in C and will be making all the same malloc(),calloc(),realloc() type memory calls.	[reply]
Re^8: Memory management by desemondo (Hermit) on Aug 20, 2009 at 10:04 UTC
cheers for that - I gave it a try and definitely could see the memory footprint growing, but once it had completed one cycle of the loop it didn't grow any further. Is that what I should expect to see with that script? From what I gather - this makes sense. Like you say, if the OS kept having to allocate and reallocate memory it would slow things down a lot... So far I've been playing around with GD::graph, in particular gd::graph::line. Sometimes its a bit limiting, but overall its done very well. I use it a lot for analysing some of my systems at work, primarily things like throughput and latency, etc. Are you familar with .Net? Some of the guys at work have been getting into the .net charts - I'd love it if someone created an interface for perl for something like these. dotnet_graphs	[reply]