in reply to Re^5: Should I use threads? Perl/DHCP/Radius
in thread Should I use threads? Perl/DHCP/Radius

I even tried detach instead of join. Again, it progressively worsens as it reaches the 10th thread. I dont know why.

Can you let me see that code? I might then be able to answer the why.

From what I have read it looks like join blocks the other threads

Where did you read that? join() only blocks the thread is runs in.

I suspect your misunderstand arises from $_->join for @threads.

If the first thread in @threads is last to finish, then the other threads will not be joined until that occurs. But that is because you've asked to join the threads in a specific order. So don't do that.

Again, sight of your code would make life simpler.

and be able to have a hash in the main thread w the status for each MAC I have encountered.

This is the bit I'm not understanding yet.

  1. Why must the main thread retain this information?
  2. What will it do with that information?
  3. When will it do it?

By way of example. This simulates reading a logfile (it just reads from STDIN); spawns a thread for compliant input (your ack lines); those threads "process" the input (sleep and then build a hash for return); and a finisher thread processes the results (prints them to stdout).

The thing to note is the separation of concerns. The main thread only reads the log; the spawned threads only deal with their job and return results; the finisher thread gathers and processes the results. Each thread is free to do what it needs to do, when it need to do it, without any form of synchronisation or locking.

Each thread individually is a simple linear piece of code. It either starts, runs and finishes; or loops until done. No multiplexing; no synchronisation; no artificial events; no breaking up linear code into itsy bitsy chunks in order to ensure that someother concern is dealt with in a timely manner. We let the OS scheduler ensure that when a trhead needs to do something, it gets done just as soon as there is a cpu available to do it. That means that it runs just as well (actually better) on a 16 or 32 core processor as it does on a 2 core processor, without the need for any hardware specific tuning. It just works.

#! perl -slw use strict; use Time::HiRes qw[ sleep ]; use Data::Dump qw[ pp ]; use threads; use threads::shared; my $done :shared = 0; sub checkStuff { sleep rand( 5 ); return { split ' ', $_[0] }; } my $finisher = async { while( sleep 0.1 and not $done ) { for my $thr ( threads->list( threads::joinable ) ) { my $hRef = $thr->join; printf "From tid:%d got %s\n", $thr->tid, pp $hRef; } } }; while( <> ) { next unless /ACK (.+)$/; async \&checkStuff, $1; } sleep 0.1 while threads->list( threads::running ) > 1; sleep 0.1 while threads->list( threads::joinable ); $done = 1; $finisher->join; __END__ [15:14:01.92] c:\test>856740-2 ACK A 1 B 2 C 3 junk ACK A 1 B 2 C 3 ACK A 1 B 2 C 3From tid:2 got { A => 1, B => 2, C => 3 } junk ACK A 1 B 2 C 3 junk junkFrom tid:3 got { A => 1, B => 2, C => 3 } ACK A 1 B 2 C 3From tid:4 got { A => 1, B => 2, C => 3 } junk ^Z From tid:5 got { A => 1, B => 2, C => 3 }


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy

Replies are listed 'Best First'.
Re^7: Should I use threads? Perl/DHCP/Radius
by MonkeyMonk (Sexton) on Aug 25, 2010 at 15:48 UTC
    #!/usr/bin/perl use strict; use LWP::UserAgent; use HTTP::Request; use threads; use threads::shared; use strict; use Time::HiRes qw( gettimeofday tv_interval); use IO::Handle; STDOUT->autoflush(1); # Reg Exp Patterns my $rexphour = '[0-2][0-9]'; my $rexpminsec = '[0-5][0-9]'; my $rexpmac = '([A-Fa-f0-9]{2}:{0,1}){6}'; my $rexpip = '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}'; my $rexppcname = '\(.*?\)'; my $rexpethIface = 'via\s+.*?'; my $rexpPattern = qr/.*?($rexphour):($rexpminsec):($rexpminsec).*?($re +xpip).*?($rexpmac)\s($rexppcname){0,1}.*?($rexpethIface)$/; my $stext = "dhcpd: DHCPACK on "; my %all_threads; my ($line, $cmac, $pcname, $NWIface, $ip); my @joinable; my %clients :shared; my @threads; open(DETAIL,"<dhcpleases") or die "ERROR messages "; seek(DETAIL,1,2); for (;;){ while (<DETAIL>) { chomp; $line = $_; # Get current time my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = loc +altime(time); # Search for DHCP Acks and Parse Time, MAC, IP, Device Name, NW Iface if($line =~ /$stext.*/){ $line =~ /$rexpPattern/; my $secepoch = timelocal($3,$2,$1,$mday,$mon,$year, $wday, + $yday, $isdst); $ip = $4; $cmac = uc($5); $pcname = $7 ; if($pcname eq ""){ $pcname = "?"; } $NWIface = $8; }# if $all_threads{$cmac} = threads->create ( \&checkmacapi, $cmac ); }#while seek(DETAIL, 0, 1); my $finisher = async{ while (1) { foreach my $macthread (threads->list(threads::joinable)) +{ $macthread->join(); } }#while }; }# sub checkmacapi{ my $mac = shift; my $macAuthURL = "http://internaladdressconfidential"; my $t0; my $t1; my $tint; my ($request, $ua, $response, $respcon, $rval); my ($returnkey,$returnval); print "Thread ".threads->tid()." Launched \n"; # print usecTime()." Thread".threads->tid(). ": MAC: ".$mac." Cont +acting $macAuthURL$mac.\n"; $t0 = [gettimeofday]; # Wrap call to API with eval eval { local $SIG{ALRM} = sub { die "Alarm\n" }; alarm(16); my $macr = $macAuthURL.$mac; $request = HTTP::Request->new(GET => $macr); $ua = LWP::UserAgent->new(); $response = $ua->request($request); $respcon = $response->content(); alarm(0); }; # We check the response object's status if($response->status_line =~ /^500/ ){ print "MAC: ".$mac." Network Delay/Error."; { lock (%clients); $clients{$mac} = 0; } } # print usecTime()." Thread".threads->tid().": MAC: ".$mac." Server + response: $respcon \n\n\n" ; # All OK - Ensure that sent mac is returned back on success. if( $response->{_rc} == 200 ){ $response->{_content} =~ s/[\{\} \"]//g; ( $returnkey, $returnval ) = split(/:/,$response->{_content},2 +); print "MAC: ".$mac." returned with status: $response->{_rc} " +; if( $mac eq uc($returnval) ){ { lock (%clients); $clients{$mac} = 1; } } else{ # Malformed URL { lock (%clients); $clients{$mac} = 0; } } } else{ { lock (%clients); $clients{$mac} = 0; } } $t1 = [gettimeofday]; $tint = tv_interval $t0, $t1; print "Thread ".threads->tid()." time :$tint\n"; }
    This is the newest based on your last post with an async block monitoring joinable threads. Noticed a significant change. LAst thread to be launched finished in 4 seconds instead of 8.

      The first thing I notice is that you are starting your finisher thread inside your for ever loop. That means you will start a new finisher every time you loop and never clean up the old ones.

      The second thing is that your finisher thread is in a tight loop. (while(1)). That's bad. It means that thread will consume as much cpu as the scheduler gives it, to the detriment of every other thread (and process) in the system.

      Move the creation of that thread outside (above) the for loop and add a short (0.1) delay and the overall responsiveness of your program (and entire system) will improve.

      Finally, and most importantly, you are joining these threads, but then doing nothing at all with the results. Indeed, you aren't returning anything in the first place. So why bother to join them? Are you intending doing something useful at a later point?


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        BrowserUK,

        I am slowly beginning to understand what you are trying to say. This is my first attempt at threads. I guess when trying to write threads, it requires a change in mentality - breakdown of tasks and split them out into threads. This I think comes with experience. But very nice ideas yours! I think you more than nudged me into threads. Thanks.

        Coming back to your post above: Finisher thread was a stupid mistake. I am now rethinking the tasks.

        1. Plan is to have a main thread reading the log file.
        2. Main thread will also have all encountered MACS to save on lookups if it has already failed earlier.
        3. Split the http check part into a thread.
        4. Possibly include Auth also in above thread. ( Cant reveal much on this )
        5. A separate thread that will monitor all MACs and delete them based on certain conditions based on their network activity

        One question : What is the difference between a thread updating a share hash and exiting (A) and a thread joining the main process with return data (B).

        In my case where I use a shared hash with MAC keys in the main thread , I feel (A) is a better option. Each thread updates the hash and simply leaves.