Possumfoot has asked for the wisdom of the Perl Monks concerning the following question:

I have a very simple perl script that I am using to run a backup job on a Windows Server 2003. The script uses system() to call a stand-alone .exe which backs up a GroupWise post office. It calls this .exe 4 times for 4 different POs. During the process, there seems to be some threshold where the machine locks up and this error is repeated in the event log:

Event ID 2019: The server was unable to allocate from the system nonpaged pool because the pool was empty.

From the research that I have done this seems to be a memory leak. I have adjusted the memory settings on the server. This pushed the threshold further down the road but did not get rid of it. I suspect that this is caused by using the system() command. I was using ActivePerl 5.8.4 build 810. I have just recently upgraded this to the latest version and am running a test now.

In the event that this does not fix my problem, does anyone have any suggestions how I might find/fix the leak or at least flush the memory after each system() call so that it does not lock up the server?

Replies are listed 'Best First'.
Re: Activeperl & Memory Leaks
by Util (Priest) on Jan 18, 2005 at 17:55 UTC

    I suspect the memory leak is in the stand-alone .exe, not in Perl. (BTW, which .exe are you using?) To prove it, change your program to take the commands that system() had been running, and write the commands into a batch file. Either schedule the Perl job separately from the batchfile job, or run the batchfile by hand from a separate command-prompt window. If the batch file fails, then the leak must be in the commands that it ran. You can then add code to your Perl program to split the backup load into smaller chunks, so that you run a higher number of smaller backups (or get a bug-fix for your .exe).

    It may also be helpful to watch the actual stats for memory usage during the run of the backup executable. These steps work on WinNT and Win2000; Win2003 is probably similar.

    1. Start the Windows Task Manager (Ctrl-Alt-Del), click the "Processes" tab.
    2. From the menu, do View->Select Columns, then enable columns PID, CPU Usage, CPU Time, Memory Usage, Virtual Memory Size, Paged Pool, and Non-paged Pool. Click OK.
    3. Click on the "NP Pool" column header, to sort by that column.
    See also: Memory Leaks on WinNT

    If all else fails, then trim down the program to the smallest version that still exhibits the problem, and post it as a follow-up to this thread.

      Good thought but I tried that. This backup process started life as a batch file. I rewrote it using Perl in Dec so that I could add better logging and an email notification at the end.

      The .exe I am using is gwback32.exe. It was a download from Novell.

      Watching the stats during the process should prove "exciting". I have been running a full backup for the past 3.5 hours and it is still working on step 1 of 3. Over that time the PF Usage has risen steadily from 250 to 362.

      Here's a thought: Suppose the leak is in gwback.exe. As I understand it a system() command from perl does a fork to run the command. Once that spawned process ends, are the resources that were leaked during that child process released? What about in a batch file rather than a perl script? That may account for why the .bat works and the perl does not.

      Here is the trimmed code. As mentioned it is pretty simple:

      #!/usr/local/bin/perl use strict; use warnings; use Date::Calc qw( Today_and_Now ); use MIME::Lite; #print "** start **\n"; my ( $cmd ) = ( '' ); my $pofront = 'C:\gwbackup\gwback32.exe '; append_log ( "\n--- Begin Nightly Backup ---", 0); $cmd = $pofront . '/po-\\\\Usacscpost52\POVOL\cscpos52 /backup-e:\cscp +os52 /silent'; do_cmd_and_log( $cmd, "Copy cscpos52" ); $cmd = $pofront . '/po-\\\\Usashacmail\SYS\SHACPOST /backup-e:\shacpos +t /silent'; do_cmd_and_log( $cmd, "Copy shacpost" ); $cmd = $pofront . '/po-\\\\Usamcpost\sys\umcpost /backup-e:\umcpost /s +ilent'; do_cmd_and_log( $cmd, "Copy umcpost" ); append_log( "End Backup", 1 ); #send_complete_msg(); #print "** done **\n\n"; exit; sub append_log { my ( $txt, $dotime ) = @_; if ( $dotime ) { my @now = Today_and_Now(); $now[3] = "0$now[3]" if ($now[3] =~ /^\d{1}$/); $now[4] = "0$now[4]" if ($now[4] =~ /^\d{1}$/); $txt = " $now[1]/$now[2]/$now[0] $now[3]:$now[4] $txt"; } open( OUTFILE, '>>C:\gwbackup\gwback.log'); my $old_handle = select(OUTFILE); print "$txt\n"; close OUTFILE; select($old_handle); } # append_log sub do_cmd_and_log { my ( $cmd, $txt ) = @_; append_log( "Begin $txt", 1 ); system($cmd); } # do_cmd_and_log