in reply to Re^5: perl 5.12.3 + threads + memory leak
in thread perl 5.12.3 + threads + memory leak
Sorry, but I don't think I am going to be able to help you.
Despite the amount of code you've posted, you've omitted a lot of the stuff that is likely to be the source of leaks.
Eg. the declarations (and the relative positioning in the source file(s?) of those declarations) for all the mysterious hashes that you use and maintain. If these hashes (eg.%db_connections, %db_ids_in_progress, %grand_hash, %running_threads ) are closed over by your thread procedure(s?), they are very likely sources of leaks, because:
Your code is full of anomalies and dubious coding practices:
my ($cnf,$thread) = @_; if ($thread && !$db_connections{$cnf}{$thread}) { my $control = "dbh".$cnf.$thread; # ^^^^^^^^ $$control = DBI->connect( #^^^^^^^^ "dbi:Pg:database=dirotext;host=domain.com", 'user', 'pass' + ) or die $DBI::errstr;
Which makes "settings stricts etc" dubious.
The fact that you are including $thread into those symbolic names suggests that you are pooling dbi handles across threads which is a definite no-no.
sub dbconn() { my ($cnf,$thread) = @_;
And get away with it by overriding the prototypes by using &dbconn('pg1');
my %response_hash = (); ... return \%response_hash; ... my %tr_res = %{$running_threads{$name}->join()};
Whilst there is nothing wrong with that in as much as it works, the mechanics of what goes on under the covers to allow you to do it, make it unnecessarily prone to leakage.
The reference you receive in the parent is (cannot be) the same reference that you take within the thread. In order to be able to give the parent thread a hash ref that it can dereference, the entire hash has first to be cloned into shared memory space.
And then the first thing you do is clone it again into your local memory space.
Far better to do:
my %response_hash; ## No need to initialise lexicals. ... return %response_hash; ## Return a simple list of values ... my %tr_res = $running_threads{$name}->join(); ## Assign the list to a +local hash.
One caveat of this is that you need to ensure that the thread procedure is given the a list context (so that it can return a list). That means that this (mess):
$running_threads{$threadname} = threads->create(\&send_sms +_message, $db_ids_in_progress{$id}{'uid'}, $db_ids_in_progress{$id}{' +from'}, $db_ids_in_progress{$id}{'to'}, $db_ids_in_progress{$id}{'msg +'}, $id, $db_ids_in_progress{$id}{'dlr_track_id'}, $processing);
Has to become:
( $running_threads{$threadname} ) = threads->create( \&send_sms_message, $db_ids_in_progress{$id}{'uid'}, $db_ids_in_progress{$id}{'from'}, $db_ids_in_progress{$id}{'to'}, $db_ids_in_progress{$id}{'msg'}, $id, $db_ids_in_progress{$id}{'dlr_track_id'}, $processing );
Notice that the variable that will receive the thread handle is in parens, thus giving threads->create() a list context, which it duly passes on to the thread procedure.
The upshot is that the style of the code you've posted along with what you've chosen to omit, strongly suggest that you are one of those that eschew the use of strict (and that usually means you eschew warnings also; as confirmed by your earlier code). The fact that you feel the need to initialise hashes at various points indicates that these are almost certainly global variables with extended scopes.
Whilst you can get away with these archaic (perl4-ish) practices in single threaded code, mixing them with threads is a recipe for mysterious errors and leaks if not crashes.
Whilst it seems that threads currently leak on *nix, I'm guessing that the vast majority of your problems are self-inflicted. The bottom line is that unless you can post a full, strict & warnings clean, globals-free, version of your program that I can at least syntax check the results of changes using Perl, even if I cannot run it, wading through a mass of code like this and trying to guess where and how half the variables are declared is just too much work to consider.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^7: perl 5.12.3 + threads + memory leak
by kamenpetrov (Novice) on Feb 18, 2011 at 06:49 UTC | |
by BrowserUk (Patriarch) on Feb 18, 2011 at 06:59 UTC | |
by kamenpetrov (Novice) on Feb 18, 2011 at 11:44 UTC | |
by BrowserUk (Patriarch) on Feb 18, 2011 at 13:48 UTC |