in reply to Re: Perl memory Memory consumption
in thread Perl memory Memory consumption

Hi,

Thanks for your observations, but due to a NDA in my project I can't publish any code. But looking the memory consumption I can see that it grows about 1 MB/sec. I think the problem is in this part : 
	my $config = Config->new(config_file => "config.xml", debug=>0);





	$bd =  db_out->new ($config->{db_data});





	while (1) { 





  		$records = $bd->stack_out_get(0);





  		if(defined($records)) {





    			while (($key,$msg)= each %$msg ){	





      			process_msg($bd,$msg);





    			}





  		} 





	}

Replies are listed 'Best First'.
Re: Re: Re: Perl memory Memory consumption
by alfatux (Novice) on Nov 23, 2001 at 16:58 UTC
    Sorry, stack_out_get returns a hash with all the resultset of de query, 10 records each like
    id|TXT VARCHAR(2048)|timestamp_a|timestamp_b|app_Code VARCHAR(3)|number (integer)

    TXT lenght it's alway about 250 char, but in a future version could be greater.
[reply]
Re: Re: Re: Perl memory Memory consumption
by jarich (Curate) on Nov 24, 2001 at 10:30 UTC
    First of all let me say that I dearly hope that the conditional in this while loop has a typographical error: 
    while(($key, $msg) = each %$msg) {
    process_msg($bs,$msg);
}

    [download]
    I hope you mean: 
    while(($key, $msg) = each %$record)

    [download]
    If you don't mean this, then this might be some of your problem. Try renaming your hash value.

    You can also make this much easier to read by using a foreach loop (if you only want to handle the msg): 

    foreach my $msg (values %$records) {
    process_msg($bd, $msg);
}

    [download]

    Anyway, I wanted to suggest that you look at Perl's profiling package. It can't exactly help you find out where all your memory is going, but it might help you track down where it's being lost. (Note that you should definately be using "my" for each of your variables to ensure that they don't hang around out of scope.)

    You can read about Perl's profiling package with perldoc Devel::DProf

    The shorts of it are to profile a script run the perl interpreter with the -d switch. So perl -d:DProf test.pl. when it's done, check the results by running dprofpp or dprofpp -T.

    I hope that helps. :)

[reply]
[d/l]
[select]
Re: Re: Re: Perl memory Memory consumption
by orkysoft (Friar) on Nov 24, 2001 at 04:20 UTC

    Does the program grow if you leave out that process_msg subroutine call?

    Also, use strict and my variables where possible, and don't make circular references like $a = \$b; $b = \$a;, or they'll never really get out of scope (they might go out of scope and be unreachable, but since they refer to eachother, they'll never go away)

[reply]
[d/l]
      Hi,

      No, with process_msg the program not gows.

      I've investigating more an I detect that the problem could be in the get_msg funtion that looks like: 
      

sub get_msg {


  my $self = shift;


  my ($param) = @_;


  my $consulta_stack = qq{


                         SELECT *


                         FROM out


                         WHERE app = $param


                         ORDER BY id DESC LIMIT 100;


                        };


  my $sth = $self->{dbh}->prepare($consulta_stack);


  $sth->execute();


  my $msgs = $sth->fetchall_hashref('id');


  $sth->finish;


  return $msgs;


}
      I also observ that when there's no record in the resultset memory consumption is highter than when there's sontehing.
[reply]

In Section Seekers of Perl Wisdom