mpeppler has asked for the wisdom of the Perl Monks concerning the following question:

Some background: I'm writing a DBI driver (pure perl) for a proprietary system. The transport layer to the "database" is over http, and there is a whole perl object framework for interacting with the database. I've got it working pretty nicely, but I need to implement an "interrupt" functionality that communicates back to the server in the case where the client is terminated either by a SIGTERM, SIGINT or by an alarm/timeout.

The framework supports this interrupt functionality, but it must know what request it needs to interrupt, meaning that the $SIG{INT}/$SIG{TERM} subroutine needs to know which DBI statement handle is active at the time, and act accordingly.

My initial thought was to do something like this:

# This is $sth->execute sub execute { my $sth = shift; .... my $req = BigHairyDatabaseInteraction->new(...); local $SIG{INT} = sub { $req->handleInterrupt }; $req->execute; # When $req->excute comes back the entire query # has been processed, and all the rows are in $req. ... }
Initial testing appear to show that this works, but I'm worried that by using a closure like this $req will never get destroyed due to dangling references. I haven't really used closures before, so my experience in that area is rather limited.

Does the above technique seem reasonable, or would you suggest other techniques to achieve the same result?

Thanks!

Michael

Replies are listed 'Best First'.
Re: Closures, object destruction and memory leaks
by sgifford (Prior) on Nov 08, 2003 at 20:44 UTC

    I believe that what you have there will destroy $req when the execute sub exits. At that time, $req will go out of lexical scope, and the local $SIG{INT} will also go out of scope, so there will be no leftover references to $req and it will go away.

    If you hadn't made $SIG{INT} local, $req would stay in scope until it was set to something else.

    Here's a test program I wrote to confirm this.

    #!/usr/bin/perl -w use strict; package TestObject; sub new { my $class = shift; my $self = {}; bless $self,$class; } sub somesub { my $self = shift; print "Called somesub\n"; } sub DESTROY { my $self = shift; print "DESTROY\n"; } package main; print "Entering execute\n"; &execute; print "Execute completed\n"; sub execute { my $req = TestObject->new or die "Couldn't create TestObject\n"; print "Created TestObject\n"; local $SIG{INT} = sub { $req->somesub }; print "Set \$SIG{INT}\n"; sleep(1); print "Done with execute\n"; }

    It produces:

    Entering execute Created TestObject Set $SIG{INT} Done with execute DESTROY Execute completed
    showing that the object is destroyed immediately after execute completes.

    Update: Fix typo in shebang line.

Re: Closures, object destruction and memory leaks
by diotalevi (Canon) on Nov 08, 2003 at 19:55 UTC

    Trying this on 5.8.0 appears to work. I didn't see any warnings in the documentation on %SIG or local() regarding this.

    sub f { local $SIG{INT} = sub { die "a" }; {redo} } $\ = "\n"; print $SIG{INT} ? 1 : 0; eval { f(); }; print $SIG{INT} ? 1 : 0;
Re: Closures, object destruction and memory leaks
by jonadab (Parson) on Nov 09, 2003 at 02:09 UTC
    I'm worried that by using a closure like this $req will never get destroyed due to dangling references.

    Unless there's something you're not telling us, you should be okay. The operative line is here:

    local $SIG{INT} = sub { $req->handleInterrupt };

    By scoping $SIG{INT} dynamically, you ensure that it will pass out of scope at the end of the block. At that time, the coderef it stored will be thrown away, and assuming that's the only reference to the anonymous sub in question, the anonymous sub's reference count should drop to zero, and it will be harvested; consequently, it will drop its reference to $req->handleInterrupt so that that also can be cleaned up, unless something somewhere else is holding another reference.

    Generally speaking, you need to be careful about memory leaks with closures under the same circumstances as with any other references -- primarily when you have circular constructs, e.g., like the following:

    sub createcircle { my ($foo, $bar) = @_; my $qux; my $quux = sub { if (shift==$foo) { $bar } else { $qux->() } }; $qux = sub { $quux->(shift - 1) }; return $quux; }

    $;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}} split//,".rekcah lreP rehtona tsuJ";$\=$ ;->();print$/
Re: Closures, object destruction and memory leaks
by tilly (Archbishop) on Nov 09, 2003 at 03:49 UTC
    I would test this on your client's version of Perl.

    What you describe should work, but I know that there was a bug in some versions of Perl (sorry, I couldn't seem to track it down, nor do I have them to test, but I am pretty sure that Perl 5.005_03 was one of them) where closures like that would never get garbage collected because of a Perl bug.

    I would suggest trying the above with a $req with a DESTROY method that tells you when it goes away. See if it is a problem. If it is, then there isn't much you can do about it other than clean up $req yourself and hope that the memory leak doesn't grow too fast.

      Thanks all - the target platform is perl 5.6+, so I guess the bug tilly mentioned won't be an issue. I've got this working the way I want now (well - the way I want for an alpha relase :-)) and I've decided on the following:
      sub execute { my $self = shift; my $req = BigHairyDatabaseInteraction->new(...); { local $SIG{INT} = sub {$req->{_signal} = 2; $req->handleInterrupt; }; local $SIG{TERM} = sub {$req->{_signal} = 15; $req->handleInterrupt; }; $req->executeRequest(); } # If a TERM/INT signal has been delivered during the request # then pass it on to the program itself: if($req->{_signal}) { kill $req->{_signal}, $$; } ... rest of $sth->execute()
      The idea is that I want to override any signal handlers that the calling program may have installed so that I can interrupt the processing on the server as needed, but I also want to make sure that the caller does eventually receive the signal.

      So far this appears to work really well...

      Michael

        One tip that may make sense.

        Before you do the local, capture the existing handlers in lexical variables and pass them in to $req's handleInterrupt method.

        That way if a particular request type wants to, it can rethrow an exception properly up the chain, rather than just ignoring the poor programmer who is trying to put a signal handler around your code because they really don't want to exit on that signal.

        Doing this makes the signal handler system act a little bit more like a proper exception handling system. Not that much more like, mind you, but somewhat more like it.

      I _think_ the memory leak you mean only comes up with recursive closures:

      my $foo; $foo=sub { $foo->(@_[1..$#_]) if @_ }; $foo->();

      But like you I dont remember the details...


      ---
      demerphq

        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi


        Recursive closures leak memory. That is documented and the fix is to use WeakRef.

        What I am talking about is an actual bug, and I darned well know that it was there in 5.005_03 because I encountered it and had to fix a bunch of code that was leaking memory too fast.