if I : ... then might the object use main thread memory (during dynamic update), even when called from a worker.thread?
Unfortunately no. I wish that shared variables had been implemented that way as it would be
- More intuative.
- Faster.
- Simpler.
The correspondance between "shared" and "global" is very strong. Simple, direct access to Perl's global variables (with user locking) would be conceptually far simpler to grasp, faster (no cloning required; nor to be subverted), and would have less (no) performance impact upon lexical variables, as they would never be shared, so none of the internal locking would be required. And that would benefit everyone including those not using threads. But that horse bolted a long while ago.
As I said above, for dynamic use, I'd use a client-server approach to sharing the CIDR object. (With some help from JadeNB & ikegami) I came up with this implementation which seems to work quite well and can sustain close to 30,000 queries per second from 10 concurrent threads. Which if there is any IO in your threads should be more than sufficient:
#! perl -slw
use strict;
use Math::Random::MT qw[ srand rand ];
use Net::CIDR::Lite;
use threads qw[ yield ];
use threads::shared;
use Thread::Queue;
our $WORKERS ||= 10;
our $N ||= 1000;
sub n2ip { join '.', unpack 'C4', pack 'V', $_[ 0 ] }
sub worker {
my $tid = threads->tid;
srand( $tid );
my( $Q, $sem ) = @_;
for( 1 .. $N ) {
my $ip = n2ip( int( rand 2**32 ) );
$Q->enqueue( "$tid:$ip" ); ## 1
lock $$sem; ## 2
cond_wait $$sem; ## 3
printf "[$tid] ($_) $ip was %s\n", $$sem ? 'found' : 'not foun
+d';
yield; ## 4
}
}
my $Q = new Thread::Queue;
my @sems = map{ my $var :shared; \$var } 0 .. $WORKERS;
my @threads = map{
threads->create( \&worker, $Q, $sems[ $_ ] );
} 1 .. $WORKERS;
my $CIDR = Net::CIDR::Lite->new(
map{
n2ip( int( rand 2**32 ) ) . '/' . ( 1+int( rand 32 ) )
} 1 .. 10
);
print for $CIDR->list;
printf "Enter to run threads"; <>;
while( threads->list(threads::running) > 1 ) {
my( $tid, $ip ) = split ':', $Q->dequeue;
print "Got $ip from $tid";
my $found = $CIDR->find( $ip );
$CIDR->add( "$ip/1" ) unless $found;
${ $sems[ $tid ] } = $found;
lock ${ $sems[ $tid ] };
cond_signal ${ $sems[ $tid ] };
yield;
}
$_->join for @threads;
This is not trivial code (and I haven't commented it), so please ask if there is anything you need clarifying. One thing I will point out is that both yields are necessary for reliable operations.
The essential steps for the client threads are the four I've tagged. The server end is the while loop in the main thread which should be usable pretty much as is. You could also hive that off into its own thread if you have something better for your main thread to be doing.
I spent a couple of hours trying to figure out the 'pack'ing process to create 32-bit integers from string IPs
There are functions in one of the modules that do the conversion of ips to ints and vice versa, but I can never remember the name of the module or the (stupid) names of the functions, and find it quicker to role my own. For IP to int I use:
sub ip2n{ unpack 'V', pack 'C4', split '\.', $_[ 0 ] }
Of course, someone will point out that ips can contain hex or decimal components and that won't deal with them. But the only times I've ever encountered them, have been when some oik is trying to disguise them for nefarious purposes, so having it fail for them is a positive in my book.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
|