in reply to Re: Queuing in multithread context
in thread Queuing in multithread context

Thanks !!

I was considering my approach wasn't the proper or best one. You perfectly understood what I was trying to achieve thus you made a important point.

I've already been able to work without thread in a sequential way and ended implementing threads to hurry the stuff and not depending on tasks between separated servers.

I'm still figuring out how I could use the queue as you pointed but at least it sounds like a more relevant direction...

Replies are listed 'Best First'.
Re^3: Queuing in multithread context
by SimonPratt (Friar) on Jan 20, 2015 at 12:40 UTC

    This is a very basic representation of how your threading model might look:

    use 5.16.2; use threads; use Thread::Queue; my @serverList = qw(one two three four five six); my $serverQueue = Thread::Queue->new( @serverList ); $serverQueue->end; my @threads = map { threads->new( \&worker ) } 1..4; $_->join for @threads; say 'Back to main - Finished!'; sub worker { my $tid = threads->tid; while ( my $server = $serverQueue->dequeue_nb ) { say "I'm thread ID $tid and I'm processing server $server to c +opy a whole bunch of files"; sleep 1; } say "Thread ID $tid finished"; }

      Thank you again for your guidance,

      problem is to me that I'm unable to find a pattern where I can achieve a working implementation of your sample. This is a great sample of queueing proved working but I just don't get it, maybe I just completely mistaken the purpose of the Thread::Queue; ... I understood it could hold threads (like tasks) waiting for the completion of one to dequeue a second..

      To be honest what is losing me in your example is that you never "queue" any item in $serverQueue... there is something I don't get in here, maybe this is a bit too advanced for my understanding

        Added some comments to help you out:

        use 5.16.2; use threads; use Thread::Queue; my @serverList = qw(one two three four five six); # Array to hold list + of servers that need to be updated my $serverQueue = Thread::Queue->new( @serverList ); # Create a Thread +::Queue, passing in the array of servers as items to queue (each serv +er is queued as a single string) $serverQueue->end; # Nothing else will be queued, so notify the queue +of this by ending it my @threads = map { threads->new( \&worker ) } 1..4; # Create a suitab +le number of threads that wont overload IO on the local server $_->join for @threads; # Block here until all the threads have complet +ed say 'Back to main - Finished!'; # End of program sub worker { my $tid = threads->tid; # Get the ID number of the thread while ( my $server = $serverQueue->dequeue_nb ) { # Dequeue the ne +xt item from the queue, in a non-blocking fashion (will undef and fin +ish loop once the queue is empty) say "I'm thread ID $tid and I'm processing server $server"; sleep 1; } say "Thread ID $tid finished"; }