Bare-bones embedding and memory leakage

polettix has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to find a Perl embedding solution without getting all memory grabbed by the process itself; in this process, I've isolated the very few code lines that really make me go nuts.

Consider the following bare-bones program, which actually doesn't try to do nothing really interesting but setting the same Perl variable one million times.

#include <EXTERN.h>
#include <perl.h>
#include <unistd.h>
#include <stdio.h>

void do_something () {
    dSP;
    ENTER;
    SAVETMPS;
    PUSHMARK(SP);
    eval_pv("$a = 1", 1);
    SPAGAIN;
    FREETMPS;
    LEAVE;
}

int main (int argc, char* argv[]) {
    PerlInterpreter *my_perl = perl_alloc();
    char *embedding[] = {"", "-e", "0"};
    I32 i;
    char c;

    perl_construct(my_perl);
    perl_parse(my_perl, NULL, 3, embedding, NULL);
    PL_exit_flags |= PERL_EXIT_DESTRUCT_END;

    printf("Press a key to start\n");
    read(0, &c, 1);
    for (i = 0; i < 1000000; ++i)
        do_something();
    
    printf("Press a key to continue\n");
    read(0, &c, 1);
    PL_perl_destruct_level = 1;
    perl_destruct(my_perl);
    perl_free(my_perl);
    
    printf("Press a key to exit\n");
    read(0, &c, 1);

    return 0;
}
[download]

What I really don't understand is why the process keeps growing demanding more memory as the index variable i increases.

After a suggestion from Joost (see below his response), I ended up finding that the following version of do_something shows less leakage (but not zero):

void do_something () {
    dSP;
    ENTER;
    SAVETMPS;
    eval_pv("$a = 1", 1);
    FREETMPS;
    LEAVE;
}
[download]

The only thing that has popped to my mind is that maybe the return value from eval_pv, which is actually an SV*, could be the source of my problems, but I tried to either mark it as mortal or decrease its reference count with the only result of getting an error message about disposing of a variable without references.

What am I missing (apart from memory)?

Thank you in advance,

-- Flavio

Comment on Bare-bones embedding and memory leakage Select or Download Code

Replies are listed 'Best First'.
Re: Bare-bones embedding and memory leakage by Joost (Canon) on Mar 16, 2005 at 15:23 UTC
You're missing the fact that eval_pv returns data which you're not freeing. See my reply at eval_pv leaking. "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re^2: Bare-bones embedding and memory leakage by polettix (Vicar) on Mar 16, 2005 at 15:59 UTC
Hi Joost, thank you for your kind response. Actually, I don't see the point of defining the variable in the for cycle. You're assigning a value to a C variable, which actually has no reference count or implicit disposal; moreover, the Perl interpreter has no clue about what you're going to do with the return value (unless you tell it). What I'm saying is that your code is equivalent to the following cycle: `for (;;) { ENTER; SAVETMPS; eval_pv("1;", TRUE); FREETMPS; LEAVE; }` [download] which has no explicit handling of the return value of eval_pv. But I still thank you because your suggestion actually made me think: why do I need PUSHMARK(SP) and SPAGAIN? I removed them and... I had less leakage. (BTW, I'm going to modify my original post to include this enhancement). Yes, less but not zero leakage. When my program starts, now, it has the following memory footprint (i.e. what you get before the first "read"): `Name: embed01 Pid: 13386 FDSize: 256 VmSize: 2696 kB VmLck: 0 kB VmRSS: 940 kB VmData: 256 kB VmStk: 8 kB VmExe: 836 kB VmLib: 1516 kB` [download] When I let the program go, I notice an increment in VmSize of around 4 kB every million iteration i let go; so, after 4 million iterations, I get: `Name: embed01 Pid: 13386 FDSize: 256 VmSize: 2712 kB VmLck: 0 kB VmRSS: 1056 kB VmData: 272 kB VmStk: 8 kB VmExe: 836 kB VmLib: 1516 kB` [download] that is a quite subtle leak IMHO but still a leak. Note that the memory increase is quite slow and time-linear, i.e. if I stop at 2 million iterations I get 2704 kB or something similar. This is quite an improvement with respect the previous version, but my question remains more or less the same: where is it leaking? Moreover: if I'll ever actually need PUSHMARK(SP) and SPAGAIN... how will I avoid leakage? TIA, --Flavio	[reply] [d/l] [select]
Re^3: Bare-bones embedding and memory leakage by dave_the_m (Monsignor) on Mar 16, 2005 at 17:36 UTC
why do I need PUSHMARK(SP) You don't need `PUSHMARK`, and in fact using it there is wrong. You only need it when the the thing you're calling expects a list of args on the stack (eg when calling `call_pv()`). Here, you're using `eval_pv()`, which doesn't expect a list of args, so using it is wrong. VmSize of around 4 kB every million iteration IIRC, there was a bug that caused every 65536th allocated op not to be freed; this is fixed in recent Perls. You don't say which version you're using, but try it with something like 5.8.6 and see if the residual leak goes away. Dave.	[reply] [d/l] [select]
Re^4: Bare-bones embedding and memory leakage by polettix (Vicar) on Mar 17, 2005 at 13:44 UTC
Re^4: Bare-bones embedding and memory leakage by polettix (Vicar) on Mar 18, 2005 at 17:32 UTC