in reply to Re^2: Out of memory using chart::clicker
in thread Out of memory using chart::clicker

Just a small example of memory leak 
#!/usr/bin/perl
use strict;
use Chart::Clicker;
use Chart::Clicker::Context;

use Chart::Clicker::Data::Range;
use Chart::Clicker::Renderer::Bubble;
use Chart::Clicker::Data::DataSet;
use Chart::Clicker::Data::Series::Size;
print "run 'top' in linux and look in perl process!\n";
my $j = 100000000;
for (my $i =1;$i < $j;$i+=1){
    print "$i iteration\n";
    mysub($i);
}
sub mysub{
    my ($var_i) = @_;
    my @k = (1,2,3);

    my $cc = Chart::Clicker->new(width => 800, height => 400 , format 
+=> 'png', padding => 10);
    my $series = Chart::Clicker::Data::Series::Size->new(
        keys => \@k,
        values => \@k,
        sizes => \@k,
        name => "$var_i",
    );
    my $ds = Chart::Clicker::Data::DataSet->new(
        series => [ $series ]
    );
    $cc->add_to_datasets($ds);

    my     $defctx = $cc->get_context('default');
        $defctx->renderer(Chart::Clicker::Renderer::Bubble->new);
    $cc->write_output($var_i);
    #undef $cc;
    print "\tmemory leak after 'cc->write_output'\n";
}

[download]
I can't find any solution and need quick help. I don't want to use any other perl chart lib. But I think I have no choice. :( With pleasure BorisPlus

Replies are listed 'Best First'.
Re^4: Out of memory using chart::clicker
by Peterpion (Acolyte) on Feb 23, 2014 at 11:17 UTC
    Hi monks and thanks for all the replies. I continued to work on this over the last couple of days and found that even if everything is declared as my variables inside the sub with nothing global and nothing passed in, the leak still happens, so although I don't feel confident enough to say on my own that the leak is in the module, I think from the replies above that the leak must be in chart::clicker. Heres what I tried and this still leaks:
    
#!/usr/bin/perl -s -w

use strict;
use Chart::Clicker;
use Chart::Clicker::Data::Series;
use Chart::Clicker::Data::DataSet;

$| = 1;

sub mysub {
    my @x_axis;
    my @y_axis;
    
    for (my $t=0;$t<200;$t++){
    $x_axis[$t] = $t*10;
        $y_axis[$t] = int(rand(100));
    }

    my $chart = Chart::Clicker->new;
    
    my $series = Chart::Clicker::Data::Series->new(
    keys    =>      \@x_axis,
    values  =>      \@y_axis,
    );

    my $dataset = Chart::Clicker::Data::DataSet->new(
    series  =>      [ $series ],
    );

    $chart->add_to_datasets($dataset);

    $chart->write_output('test.png');
}
    
for (my $x=0;$x<10000;$x++) {    
    &mysub;
    print ".";
}

    [download]
    So I will both report it to the module writer and try to find it next week myself (but its the first time I have delved into such a complicated module so I think it might be quite difficult and I'm not sure how well I will get on). Looking at the dependencies it looks like there are components written in other languages than perl which although I understand some of, they can tend to be very large (ie C) and I don't have any debugging tools set up for that kind of thing (or much experience with memory leaks etc).

    Anyway I thought about the problem a bit more and rewrote the program to write out files which contain one graphs worth of data with a header to describe the information required as state (like absolute time etc). This works well but obviously is less elegant, however its probably more in the spirit of perl as it Just Gets The Job Done nicely in the real world. Although I haven't added the code to fork / exec yet, I will perhaps fork a few child processes to do the graph writing (dependent on the number of processors would be nice) to take advantage of dual or quad cores without the complexity and room for errors that threaded code has. These forked processes could then reexec after a particular number of iterations to deal with the memory leaks.

    So all in all I think thats a workable solution but it would be nice to find the leaks in the module still. I wonder if the fact that the process is growing so large would slow it down at all? IE does constantly asking for more memory from the OS slow it down much? (and should that affect my decision as to how many charts to let each child process produce before they commit hari kari?)

    Lastly a new question has arisen from this but I think its so unrelated that I better post it as a new question but for those who see it, the question relates to this same program (related to how to input a very large amount of data from an external program called from my program). Thanks very much to all who commented.
[reply]
[d/l]
Re^4: Out of memory using chart::clicker
by Peterpion (Acolyte) on Feb 24, 2014 at 10:56 UTC
    Hi Boris, Just in case you didn't see the above, consensus seems to be that the module or its dependancies have a leak which might be tricky to find for someone in a hurry like it sounds you are, so a workaround is probably the best route. Either fork children to process the graphs (but be careful you don't start too many children at once if you do this), or the way round I have used currently is to write temp files of data and then subsequently in the same program I start to process the graphs and write them out. Once I have processed n graphs (the value of n depending on how large you are happy for your perl process to grow) you can exec your perl process (perhaps with a flag to indicate that it can progress directly to the graph writing section rather than creating all the data files again) and continue to process another n data files until your list of data files is all converted to graphs when your perl process can exit.
[reply]
[reply]

In Section Seekers of Perl Wisdom