in reply to Re: Is a Net::CIDR::Lite object sharable within threads
in thread Is a Net::CIDR::Lite object sharable within threads

"basis of that description--ie.readonly use of the object once initially constructed-"
I did not state that the list is created/updated dynamically during the run, with values supplied by cooperating processes.

if I :

then might the object use main thread memory (during dynamic update), even when called from a worker.thread?

I did like your reply though. I spent a couple of hours trying to figure out the 'pack'ing process to create 32-bit integers from string IPs and such. I thought I would simply create a shared hash with members containing the base network and the mask. Then I could do the comparisons myself and have a much more transparent solution. Unfortunately, the abstraction layer of Perl's implementation is still very opaque to me...

  • Comment on Re^2: Is a Net::CIDR::Lite object sharable within threads

Replies are listed 'Best First'.
Re^3: Is a Net::CIDR::Lite object sharable within threads
by BrowserUk (Patriarch) on Dec 03, 2008 at 14:15 UTC
    if I : ... then might the object use main thread memory (during dynamic update), even when called from a worker.thread?

    Unfortunately no. I wish that shared variables had been implemented that way as it would be

    1. More intuative.
    2. Faster.
    3. Simpler.

    The correspondance between "shared" and "global" is very strong. Simple, direct access to Perl's global variables (with user locking) would be conceptually far simpler to grasp, faster (no cloning required; nor to be subverted), and would have less (no) performance impact upon lexical variables, as they would never be shared, so none of the internal locking would be required. And that would benefit everyone including those not using threads. But that horse bolted a long while ago.

    As I said above, for dynamic use, I'd use a client-server approach to sharing the CIDR object. (With some help from JadeNB & ikegami) I came up with this implementation which seems to work quite well and can sustain close to 30,000 queries per second from 10 concurrent threads. Which if there is any IO in your threads should be more than sufficient:

    #! perl -slw use strict; use Math::Random::MT qw[ srand rand ]; use Net::CIDR::Lite; use threads qw[ yield ]; use threads::shared; use Thread::Queue; our $WORKERS ||= 10; our $N ||= 1000; sub n2ip { join '.', unpack 'C4', pack 'V', $_[ 0 ] } sub worker { my $tid = threads->tid; srand( $tid ); my( $Q, $sem ) = @_; for( 1 .. $N ) { my $ip = n2ip( int( rand 2**32 ) ); $Q->enqueue( "$tid:$ip" ); ## 1 lock $$sem; ## 2 cond_wait $$sem; ## 3 printf "[$tid] ($_) $ip was %s\n", $$sem ? 'found' : 'not foun +d'; yield; ## 4 } } my $Q = new Thread::Queue; my @sems = map{ my $var :shared; \$var } 0 .. $WORKERS; my @threads = map{ threads->create( \&worker, $Q, $sems[ $_ ] ); } 1 .. $WORKERS; my $CIDR = Net::CIDR::Lite->new( map{ n2ip( int( rand 2**32 ) ) . '/' . ( 1+int( rand 32 ) ) } 1 .. 10 ); print for $CIDR->list; printf "Enter to run threads"; <>; while( threads->list(threads::running) > 1 ) { my( $tid, $ip ) = split ':', $Q->dequeue; print "Got $ip from $tid"; my $found = $CIDR->find( $ip ); $CIDR->add( "$ip/1" ) unless $found; ${ $sems[ $tid ] } = $found; lock ${ $sems[ $tid ] }; cond_signal ${ $sems[ $tid ] }; yield; } $_->join for @threads;

    This is not trivial code (and I haven't commented it), so please ask if there is anything you need clarifying. One thing I will point out is that both yields are necessary for reliable operations.

    The essential steps for the client threads are the four I've tagged. The server end is the while loop in the main thread which should be usable pretty much as is. You could also hive that off into its own thread if you have something better for your main thread to be doing.

    I spent a couple of hours trying to figure out the 'pack'ing process to create 32-bit integers from string IPs

    There are functions in one of the modules that do the conversion of ips to ints and vice versa, but I can never remember the name of the module or the (stupid) names of the functions, and find it quicker to role my own. For IP to int I use:

    sub ip2n{ unpack 'V', pack 'C4', split '\.', $_[ 0 ] }

    Of course, someone will point out that ips can contain hex or decimal components and that won't deal with them. But the only times I've ever encountered them, have been when some oik is trying to disguise them for nefarious purposes, so having it fail for them is a positive in my book.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.