The singleton's should as well be cloned and not be able to touch each other.

I said I suspect but cannot prove. Without 1000 SNMP devices (or even one I could call 1000 times) all I can do is speculate. I agree it shouldn't happen, but are there any occasions or circumstances where a variable becomes shared without explicit sharing?

Well, take the example of any parameters you pass to your thread code on the threads->create(), you don't need to share those. But that's a special case. How about your leaker3.pl above. You create your queue in the main thread and never share it:

my $queue = new Thread::Queue; $queue->enqueue( @ips[ 0 .. $howmany ] );

but there you are using it inside the thread subroutine:

while( my $string = $queue->dequeue_nb ) {

If that was shared without explicit instruction, what else was?

Do me a favour and try this. All the non-blocking SNMP code is derived purely from the docs, which I may have misundertood. It compiles clean but is otherwise untested so it may require a few tweaks. It is basically the third option I mentioned above that use threads to rescue the calling code from the binds of the underlying event-driven state machine.

#! perl -slw use strict; use threads; use Thread::Queue; use Storable qw[ freeze thaw ]; use constant { ipCidrRouteProto => "1.3.6.1.2.1.4.24.4.1.7", ipCidrRouteIfIndex => "1.3.6.1.2.1.4.24.4.1.5", ipCidrRouteType => "1.3.6.1.2.1.4.24.4.1.6", ipRouteMask => "1.3.6.1.2.1.4.21.1.11", ipRouteIfIndex => "1.3.6.1.2.1.4.21.1.2", ipRouteType => "1.3.6.1.2.1.4.21.1.8", ipRouteNextHop => "1.3.6.1.2.1.4.21.1.7", ipRouteProto => "1.3.6.1.2.1.4.21.1.9", }; sub warnf{ warn sprintf $_[0], @_ } my $THREADS = $ARGV[ 0 ] || 100; my $Q = new Thread::Queue; ## The main processing goes here sub processResults{ my( $host, $commstr, $result ) = @_; ## do your heavy stuff here } ## simple worker thread sub worker { while( my $result = $Q->dequeue ) { my $result = thaw $result; my $host = delete $result->{ _my_private_host_key }; my $commstr = delete $result->{ _my_private_commstr_key }; processResult( $host, $commstr, $result ); } } ## Callback sub enqueue { ## First arg is the seesion handle--the rest whatever we asked for my( $session, $host, $commstr ) = @_; my $result = $session->var_bind_list(); ## Get the results + hash $result->{ _my_private_host_key } = $host; ## Tack on some ex +tras $result->{ _my_private_commstr_key } = $commstr; $Q->enqueue( freeze $result ); ## And serialise it for qu +euing $session->close; } my @workers = map{ threads->create( \&worker ); } 1 .. $THREADS; ## Avoid loading the heavy stuff until after we've spawned out threads +; require Net::SNMP; open my $ipsFH, '<', 'allips.txt' or die $!; while( my( $host, $commstr ) = split ' ', <$ipsFH> ) { my( $session, $error ) = Net::SNMP->session( -hostname => $host, -community => $commstr, -port => 161, -version => 'SNMPv1', -translate => 1, -debug => 0x00, -nonblocking => 1, -timeout => 30, -retries => 3, ); unless( defined $session ) { warn "Couldn't create session for $host/$commstr: reason: $err +or\n"; next; } my $success = $session->get_entries( -columns => [ ipCidrRouteIfIndex, ipCidrRouteProto, ipCidrRouteType ], ## Have host and commstr passed back to the callback -callback => [ \&enqueue, $host, $commstr ], ) and next; $success = $session->get_entries( -columns => [ ipRouteNextHop, ipRouteIfIndex, ipRouteMask, ipRouteProto, ipRouteType ] ## Have host and commstr passed back to the callback -callback => [ \&enqueue, $host, $commstr ], ) or warnf "Failed to get_entries for $host/$commstr: reason %s\n" +, $session->error(); } close $ipsFH; ## Run the event loop Net::SNMP->snmp_dispatcher(); ## Wait for the work to be done sleep 1 while $Q->pending; ## signal workers to die $Q->enqueue( ( undef ) x $THREADS ); ## And bury them $_->join for @workers;

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

In reply to Re^9: Massive Perl Memory Leak by BrowserUk
in thread Massive Perl Memory Leak by wagnerc

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.