There were some refcounting bugs in versions of perl before 5.8.0 that caused closures to leak, regardless of what you did with them. That's been fixed, so far as we know, so try your code with 5.8.0 and see if you're better with it. (There is, of course, always the possibility that things have been missed--perl's a big system, and refcounting GC systems are pretty error-prone)
Leak that perrin mentioned before was fixed in the latest development
version of Perl, by patch id 18302.
This means 5.8.0 is not enough to avoid that leak, need more recent perl.