rajiyer has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I am writing a small C++ application that invokes the Perl Interpreter and calls a perl function 2000 times. An array of strings is passed to the perl function, which pushes another value into the array. Here's the code: 

#pragma warning(disable:4786)
#pragma warning(disable:4788)
#pragma warning(disable:4503)

#include <vector>
#include <string>
#include <iostream>
using namespace std;

#include <EXTERN.h>
#include <perl.h>
#include <XSUB.h>

typedef vector<string> V_S;

void addValue(V_S& argVec)
{
    dSP;
    ENTER;
    SAVETMPS;
    PUSHMARK(SP);

    AV* array = newAV();
    for (int i = 0; i < argVec.size(); i++)
    {
        SV * value = newSVpvn( (char *)argVec[i].c_str(),
argVec[i].length() );
        av_push(array, value);
    }

    XPUSHs(newRV((SV*)array));
    PUTBACK;
    perl_call_pv("Test::AddValue", G_EVAL|G_SCALAR);
    SPAGAIN;

    argVec.clear();
    {
        I32 len = 0;
        while( av_len(array) >= len )
        {
            SV ** v = av_fetch(array, len, FALSE );
            SV * value = NULL;
            if( v )
           {
                value = *v;
                int len = SvLEN(value);
                char * dataPtr = new char[len];
                memcpy( dataPtr, SvPV_nolen(value), len );
                argVec.push_back(string(dataPtr, len));
                delete[] dataPtr;
           }
           sv_free(av_delete(array, len, FALSE));
           len++;
      }
 }
 av_undef(array);

 PUTBACK;
 FREETMPS;
 LEAVE;
}

void main()
{
     V_S myVals;
     myVals.push_back("value11");
     myVals.push_back("value12");

     static PerlInterpreter* my_perl;
     my_perl = perl_alloc();
     perl_construct(my_perl);
     char* argv[] = {"", "F:\\Personal\\Misc
Code\\PerlFromC\\Debug\\Test.pm"};
     int status = perl_parse(my_perl, NULL, 1, argv, NULL);
     if (status != 0)
     {
          cout << "Error in perl_parse";
          return;
     }
     for (int i = 0; i < 2000; i++)
     {
          V_S copyVec = myVals;
          addValue(copyVec);
          cout << "============== After calling Perl Module
================\n";
          V_S::iterator vItr;
          for (vItr = copyVec.begin(); vItr != copyVec.end(); vItr++)
          cout << "Value = " << vItr->c_str() << "\n";
     }
     perl_destruct(my_perl);
     perl_free(my_perl);
}

[download]
Here's the Perl module (Test.pm), that is being invoked through the above program. 
package Test;
require Exporter;
@ISA = (Exporter);
sub AddValue
{
     $_[0]->[2] = "myvalue";
}

[download]
This program works fine as such. But if you monitor it through the Windows task manager, the memory consumption keeps increasing. It is stable initially upto some point, beyond which it keeps rising. I tried using Rational Purify (leak detection tool) with Perl source code, but still did not get any pointers.

Any help with this would be highly appreciated.

Regards,
Rajalakshmi Iyer.

Replies are listed 'Best First'.
Re: Embedded Perl - Memory leak
by Molt (Chaplain) on Oct 16, 2003 at 10:39 UTC

    You may want to look into making 'value' mortal. It's been a while since I touched internals but what I think is happening is that the newSVpvn() call is producing as SV with a refcount of one, when it's pushed onto the array this is going up to two, and hence going back down to one when the array is destroyed.. and not being destroyed.

    I seem to recall this being discussed in "Extending and Embedding Perl", but I don't have my copy at work so can't check.

[reply]
      Actually, I have tried making 'value' a mortal. If I do that, I will have to change: 
      sv_free(av_delete(array, len, FALSE));

      [download]
      to 
      av_delete(array, len, FALSE);

      [download]
      otherwise, I get the error 'Attempt to free unreferenced scalar' on FREETMPS call. What this implies is that av_push does not increment the reference count of 'value'. Correct me if I am wrong.
[reply]
[d/l]
[select]
        It was long time when I wrote Perl modules in C last time but IIRC av_push does increment the reference count. On the other hand I'm not sure why you need sv_free - av_delete should decrease reference counter of deleted SV itself. Try to add sv_dump() calls to check it yourself.

        --
        Ilya Martynov, ilya@iponweb.net
        CTO IPonWEB (UK) Ltd
        Quality Perl Programming and Unix Support UK managed @ offshore prices - http://www.iponweb.net
        Personal website - http://martynov.org

[reply]
      You may want to look into making 'value' mortal.
      No. Never make things you put into an aggregate (or things you've fetched out of an aggregate) mortal. It messes up the refcounts, will result in things being prematurely destroyed, and in some cases may end up with unusual garbage inside the aggregates.

      Putting things into aggregates and taking them back out doesn't affect their refcounts.

[reply]
Re: Embedded Perl - Memory leak
by ptkdb (Monk) on Oct 16, 2003 at 13:09 UTC
    Leak detection used to be easier.

    It's become complicated by the fact that OS's and programs will put off disposing of free'd blocks until "they feel like" it. Java is particularly troublesome about this. I've had to toss 100s of objects out of scope before I see their finalize methods finally start getting called.

    You may want to try more interations. I've found that programs have to run awhile before their memory consumption levels off. There are artifacts in support libraries that keep their own private stashes of memory and don't start giving that up(if ever) until they've been used for a while.

    I haven't used Purify for a while, are you certain that it managed to instrument the perl library as it was going along?

[reply]
      Yes, Purify does perform a precise instrumentation of the Perl library. One would assume that with FREETMPS being called at the end of every iteration, the memory used by temporary variables does get freed up. Any comments?
[reply]

Back to Seekers of Perl Wisdom