jwn has asked for the wisdom of the Perl Monks concerning the following question:

I am importing data from a COM Object into perl for processing. The data is passed in a SafeArray by reference. I have memory leak and run out of memory rapidly (a large amount of data is passed to perl). The relevant variable is: $pvarMassList. I am hoping someone can tell me how to deallocate the memory associated with $pvarmassList after I copy the data into perl. As this script runs, I can watch the memory increase in the Windows Task Manager. Any help is greatly appreciated. Thanks. 
use strict;
use Win32::OLE;
use Win32::OLE::Variant qw(:DEFAULT nothing );
Win32::OLE->Option( Warn => 3 );

# Create MSFileReader object, must have MSFileReader dll installed
my $thermo_raw_file = 
     eval{Win32::OLE->new('MSFileReader.XRawfile.1', sub{$_[0]->Close;
+})};
if($@){
  die "Could not create XRawfile object. This function requires Thermo
+ MSFileReader\n";
}

# Open raw file and set controller to first(1) mass spec device (0)
$thermo_raw_file->Open("RSK2017C.RAW");
$thermo_raw_file->SetCurrentController(0,1);

my @TempScan = ();
my $NumberOfSpectra = Win32::OLE::Variant->new(VT_I4 | VT_BYREF, 0);
$thermo_raw_file->GetNumSpectra($NumberOfSpectra);

my $scan_num;
for ($scan_num = 1; $scan_num <= $NumberOfSpectra; $scan_num++) {
  my $CurrentScanFilter = Variant(VT_BSTR | VT_BYREF);
  $thermo_raw_file->GetFilterForScanNum($scan_num, $CurrentScanFilter)
+;
  if ($CurrentScanFilter =~ m/ms /) {    
    my $pnScanNumber = Win32::OLE::Variant->new(VT_I4 | VT_BYREF, $sca
+n_num);
    my $szFilter = Win32::OLE::Variant->new(VT_BSTR | VT_BYREF);
    my $nIntensityCutoffType = Win32::OLE::Variant->new(VT_I4 | VT_BYR
+EF, 0); 
    my $nIntensityCutoffValue = Win32::OLE::Variant->new(VT_I4 | VT_BY
+REF, 0);
    my $nMaxNumberOfPeaks = Win32::OLE::Variant->new(VT_I4 | VT_BYREF,
+ 0); 
    my $bCentroidResult = Win32::OLE::Variant->new(VT_BOOL, 0); 
    my $pdCentroidPeakWidth = Win32::OLE::Variant->new(VT_R8 | VT_BYRE
+F, 0);
    my $pvarMassList = Win32::OLE::Variant->new(VT_VARIANT | VT_BYREF)
+;
    my $pvarPeakFlags = Win32::OLE::Variant->new(VT_VARIANT | VT_BYREF
+);
    my $pnArraySize = Win32::OLE::Variant->new(VT_I4 | VT_BYREF, 0);
    $thermo_raw_file->GetMassListFromScanNum($pnScanNumber, $szFilter,
+ 
       $nIntensityCutoffType, $nIntensityCutoffValue, $nMaxNumberOfPea
+ks, 
       $bCentroidResult, $pdCentroidPeakWidth, $pvarMassList, 
       $pvarPeakFlags, $pnArraySize);

    @TempScan = @{$pvarMassList->Value()};
    # undef @TempScan; does not clear memory
    # Need to deallocate $pvarMassList from memory
    # Help here!!!!
    
    print "Scan number: $scan_num Size: $#{$TempScan[0]}\n";
 }

}
Win32::OLE->Uninitialize;

print "Press any key to exit.\n";
<STDIN>;

exit;

[download]
  • Comment on Memory Leak using Win32::OLE::Variant for a SafeArray passed by Reference
  • Download Code

Replies are listed 'Best First'.
Re: Memory Leak using Win32::OLE::Variant for a SafeArray passed by Reference
by Anonymous Monk on Jun 20, 2014 at 07:27 UTC

    Where is GetMassListFromScanNum documented?

    How do you know @TempScan is the variable eating memory?

    Try  $pvarMassList->DESTROY; undef $pvarMassList; or equivalent; try it for all the variables local to the loop

    Also see Foreach Loops because  for( my $ix = 0; $ix < $max; $ix++ ){ ... }

[reply]
[d/l]
[select]

      The function GetMassListFromScanNum is part of a DLL (MSFileReader) to read scientific data from a file that uses a proprietary format.

      The memory leak occurs even if I never declare the array @TempScan. Calling the GetMassListFromScanNum is sufficient to allocate the memory. The documentation for pvarMassList is given here:

      The mass list contents are returned in a SafeArray attached to the pvarMassList VARIANT variable. When passed in, the pvarMassList variable must exist and be initialized to VARIANT type VT_EMPTY. If the function returns successfully, pvarMassList is set to type VT_ARRAY | VT_R8. The format of the mass list returned is an array of double precision values in mass intensity pairs in ascending mass order (for example, mass 1, intensity 1, mass 2, intensity 2, mass 3, intensity 3, and so on).

      Unfortunately, ->DESTROY is not a valid method for the MSFileReader software.

      undef $pvarMassList does not clear the memory leak. It will prevent the script from accessing the SafeArray with the data. The memory leak is on the order of 400 kb where the compressed data file is about 300 kb. Only the SafeArray referred to by $pvarMassList is large enough to explain the memory leak. If appears that simply calling the GetMassListScanNum brings the data into memory.

      Even the $thermo_raw_file->Close(); command does not deallocate the memory associated with perl in the Task manager.

      I think I need a way to deallocate the SafeArray using perl (prefered) or a way to deallocate all memory associated with the MSFileReader.

      I am not sure how a foreach loop helps since the memroy leak when I call the GetMassListFromScanNum function outside the loop, the loop just makes the memory leak blow up.

      Thanks for the help, any further thoughts are appreciated.

[reply]
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom