btoovey has asked for the wisdom of the Perl Monks concerning the following question:

Hello monks,

I have a couple of questions about perl threads. I have read the available tutorials online and have searched google, but my questions still remain.

a) If I use an smp system on Linux, can I expect threads to run on different processors like kernel threads? How exactly is the determination made which processor which thread runs on?

b) I have the following logic that I would like to code. Is there anyone that can help turn it into valid perl code?:

Run 30 threads that continuously check the internet to see if an event is true. If it is, another 30 threads which have already been spawned immediately take action.

c) Is there any timing issues that you can see that I will run into?

Thanks in advance for your posts, Brian

This code is so wrong :)

#!/usr/bin/perl -w use threads; # threading routines use threads::shared; use Thread::Queue; our $queue : shared = Thread::Queue->new; for (1..30){ $thr = threads->new(\&thread_sub); } for (1..30) { $thr1 = threads->new(\$thread_sub1); } $thr->join; $thr1->join; sub thread_sub { while ($test == 0) { if (certain test is true) {$test = 1;} } #### Now here is where I am getting really god damned lost {my $action = $queue->enqueue;} } sub thread_sub1 { $action = $queue->dequeue; #??? totally lost here print "I am going to do your bidding master now that the test is + true\n"; }

Replies are listed 'Best First'.
Re: multi-threaded questions
by zentara (Cardinal) on Dec 08, 2006 at 14:07 UTC
    I don't know about smp distributing threads, I guess the kernel will decide? But here are some general comments on threads.

    When joining, the thread must return from it's code block, either by reaching the end of the block, or with an explicit return.

    # bad, you are only joining 1 # $thr->join; # $thr1->join; #better foreach my $thread ( threads->list ) { + $thread->join; + } # or you could stuuff the threads into hashes # like my %thrs; for(1..30){ $thrs{$_}{'thread'} = threads->new(\&thread_sub); } #then join them by looping thru the hash keys

    You are probably going to need to use shared variables to control interaction between the threads, but I'm not very familiar with enqueue

    sub thread_sub { while ($test == 0) { if (certain test is true) { $test = 1; #maybe test should be shared return; } } # Now here is where I am getting really god damned lost {my $action = $queue->enqueue;} }

    I'm not really a human, but I play one on earth. Cogito ergo sum a bum
      Thanks for the tip on doing the join - I was wondering why I was leaving threads hanging
Re: multi-threaded questions
by renodino (Curate) on Dec 08, 2006 at 16:26 UTC
    a) If I use an smp system on Linux, can I expect threads to run on different processors like kernel threads?
    Assuming your Linux is using the right threading kernel/library, yes. ithreads is based on pthreads on most POSIX-like systems. Assuming the provided pthreads implementation is SMP capable, ithreads will be as well.

    How exactly is the determination made which processor which thread runs on?
    Thats a pthreads/OS kernel issue. I believe some platforms provide APIs to bind individual threads to run on specific CPUs, but I don't know that they're widely implemented.

    One note about your app: 30 + 30 threads may appear to consume a lot of vmem on Linux (and Windows too). Consider tuning the default thread stack size to a small value (see Re: Use more threads. and the threads CPAN note).


    Perl Contrarian & SQL fanboy
      Thanks for your response! I will research how my distro handles pthreads.
      Do you have an idea if I am doing the que stuff properly?
        See embedded comments:
        use Thread::Queue; # # Thread::Queue is already shared (in fact, # its just a shared array) # our $queue = Thread::Queue->new; sub thread_sub { while ($test == 0) { if (certain test is true) {$test = 1;} } # # Not certain of your intent here, but assuming you're # just trying to send some data to another thread, then # you just need to pass the data as a parameter; # Note that doing so will return immediately wo/ # any ack. from the recv'r. Also note that # if $action is a ref to something, then that # something must also be threads::shared # # {my $action = $queue->enqueue;} $queue->enqueue($action); } sub thread_sub1 { # # this is fine. # $action = $queue->dequeue; #??? totally lost here print "I am going to do your bidding master now that the test is + true\n"; }
        Blatant plug:

        If you need full duplex communications between threads, you might peek at Thread::Queue::Duplex or Thread::Queue::Multiplex.


        Perl Contrarian & SQL fanboy
Re: multi-threaded questions
by Anonymous Monk on Dec 09, 2006 at 01:00 UTC
    So I came up with a proof of concept based on the suggestions here. Please let me know if i am doing anything wrong. Thanks to all who posted:

    #!/usr/bin/perl use threads; use Thread::Queue; use threads::shared; our $queue = Thread::Queue->new; my $test = "0"; my $stop_var : shared = 0; for (1..30){ $thr = threads->new(\&thread_sub1); } for (1..30){ $thr1 = threads->new(\&thread_sub); } foreach $thr ( threads->list ) { $thr->join; } foreach $thr1 ( threads->list ) { $thr1->join; } sub thread_sub { my $tid = threads->self->tid(); my $sec=1; until ($sec == 0) { ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) += localtime(time); # print "$sec\n"; } lock($stop_var); if ($stop_var ==0) { $stop_var = 1; for (1..30){ $queue->enqueue($tid); } } } sub thread_sub1 { my $tid = threads->self->tid(); print "Order thread number $tid spawned\n"; $action1 = $queue->dequeue; print "I have gotten through the queue my lord: $action1\n"; }
      You don't need 2 join subs. As you can see, all threads are in main's thread->list. You probably should 'use warnings' and 'use strict' too, it will help you keep from confusing variables.
      my @list = threads->list; print scalar(@list),"\n"; foreach my $thr ( threads->list ) { $thr->join; }

      I'm not really a human, but I play one on earth. Cogito ergo sum a bum
      Again, as your intentions are less than clear, I'm "assuming" intent.

      1. The "self->" in "threads->self->tid()" isn't needed (its an unfortunate meme many of us inherited from older threads docs).

      2. If your intent is to kill the worker threads whenever any master thread detects an event, then you're fine.

      However, I suspect you ultimately intend to keep those worker threads around to field queued events until the daemon is terminated. If my assumption is correct, then you've got a problem: queueing 30 msgs does not guarantee that all 30 threads will get a copy of the msg. It's entirely possible (indeed, quite probable) that the first few threads may get back to dequeueing before the last waiting threads get a chance to dequeue().

      Which means you either have to

      • use 30 individual queues, one per worker, and enqueue() to all 30 queues when dispatching an event
      • (as previously mentioned) use Thread::Queue::Multiplex aka TQM.

      TQM provides a publish/subscribe capability, so that your master (as publisher) need only post a single msg to a single queue, and all worker threads (as subscribers) will get their own copy of the msg.


      Perl Contrarian & SQL fanboy