http://qs1969.pair.com?node_id=1133644

kamrul has asked for the wisdom of the Perl Monks concerning the following question:

I have a code similar to the below. I have one main script which is calling another module named initial.pm. initial.pm opens up connection with an AMQP server (In my case RabbitMQ)and using Net::AMQP::RabbitMQ library for establishing the connection. Everything works fine except when I try to join my threads I get segmentation fault. I think the Net::AMQP::RabbitMQ is not thread safe. But this is only being used by the main thread. Im pretty sure you can reproduce the error if you just copy and past the codes below. main.pl
#!/usr/bin/perl use Cwd qw/realpath/; use File::Basename qw/dirname/; use lib 'lib'; use threads; use threads::shared; use initial; my @threads = (); my $run :shared = 1; my $init = load initial($name); $SIG{'TERM'} = sub { $run = 0; }; threads->create(\&proc1); threads->create(\&proc2); while($run){ sleep(1); print "I am main thread\n"; } $_->join() for threads->list(); sub proc1 { while($run){ sleep(1); print "I am child thread 1 \n" } } sub proc2 { while($run){ sleep(1); print "I am child thread 2 \n"; } }
lib/initial.pm
package initial; use Net::AMQP::RabbitMQ; use Cwd qw/realpath/; use File::Basename qw/dirname/; my $mq; my $stop = 0; sub load { my $class = shift; my $self = {}; connectq(); bless $self,$class; return $self; } sub connectq { $mq = Net::AMQP::RabbitMQ->new(); my ($host,$port,$user,$pass) = ('localhost','5672','guest','guest' +); $mq->connect($host, { user => $user, password => $pass, port => $port, timeout => 10, }); $mq->channel_open(1); $mq->consume(1, 'logger'); } 1;
If I call the initial class after creating the threads I dont see the error. It seems the threads are copying all the previously initiated variables into it when it is being created. In my case I dont need to access my $init variable from any other threads. So is there a way to tell perl not to copy a specific variable while creating new threads ?

Replies are listed 'Best First'.
Re: Perl seg fault while joining threads
by BrowserUk (Patriarch) on Jul 07, 2015 at 22:04 UTC
    So is there a way to tell perl not to copy a specific variable while creating new threads ?

    I don't have Net::AMQP::RabbitMQ, so this is untested, but if your description is accurate, it ought to work.

    Just re-structure your code slightly:

    #!/usr/bin/perl use Cwd qw/realpath/; use File::Basename qw/dirname/; use lib 'lib'; use threads; use threads::shared; use initial; my @threads = (); my $run :shared = 1; sub proc1 { while($run){ sleep(1); print "I am child thread 1 \n" } } sub proc2 { while($run){ sleep(1); print "I am child thread 2 \n"; } } threads->create(\&proc1); threads->create(\&proc2); my $init = load initial($name); $SIG{'TERM'} = sub { $run = 0; }; while($run){ sleep(1); print "I am main thread\n"; } $_->join() for threads->list();

    In general, it is better to start your threads before you create/allocate anything that is only used by your main thread.

    See threads: spawn early to avoid the crush. for more details.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
    I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!
      There are some places in my code where I cant really create the thread before the instantiating class. Is there any other way ?
        Is there any other way ?

        Yes. If you spawn a thread early, it will not copy anything created after it is spawned. If you then arrange for it to spawn the threads that you need to spawn later, they will inherit the cleanliness of their parent. Ie. Won't duplicate anything created after that first thread was spawned.

        The code below looks complicated, but it is really quite straight forward. If you wrap it into a module and call it before you call your initial module, the api can be pretty transparent.

        This just demonstrates the technique; you can do the wrapping to suit your code/style:

        #! perl -slw use strict; use threads; use threads::shared; use Thread::Queue; my $run :shared = 1; sub func1 { print "func1: @_"; sleep 1 while $run; } sub func2 { print "func2: @_"; sleep 1 while $run; } my $Qspawn = new Thread::Queue; async { while( my $parms = $Qspawn->dequeue ) { my( $type, @args ) = split $;, $parms; if( $type == 1 ) { $Qspawn->enqueue( threads->new( \&func1, @args ) ); } elsif( $type == 2 ) { $Qspawn->enqueue( threads->new( \&func2, @args ) ) } else { warn "Unknown type: $type"; } } }->detach; ## now create/allocate your main thread stuff # require initial; # initial->import( entrypoints ); ## if needed # my $init = ... ## When you want to start func1 $Qspawn->enqueue( join $;, 1, 1, 'two', 3.3 ); my $thread1 = $Qspawn->dequeue; ## other stuff ## Now spawn func2 $Qspawn->enqueue( join $;, 2, 2, 'four', 8.8 ); my $thread2 = $Qspawn->dequeue; ## do stuff ## tell func1 & func2 to finish $run = 0; ## Join them $_->join for $thread1, $thread2; ## Let the spawner thread clean itself up exit; __END__ C:\test>spawner.pl func1: 1 two 3.3 func2: 2 four 8.8

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!

        Is there a reason that the call to 'initial' needs to be in the main thread? Could you not just put it in it's own thread and spawn that first?

Re: Perl seg fault while joining threads
by stevieb (Canon) on Jul 07, 2015 at 21:50 UTC

    It's prudent to inform everyone when you've cross-posted, so that people can review what responses you got at the other site before re-inventing any wheels.

    -stevieb

A reply falls below the community's threshold of quality. You may see it by logging in.