anlamarama has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I am new at perl, using it for a month. It's a great language!

I need some help for threads, am trying to do threading but could not do that.

I have an script, which fetches data from internet, it's main structure is like this:
for ($i=0; $i<1000000; $i++) { $content = get("http://www.somewebsite.com/news.php?id=$i"); print "Error!" unless defined $content; ... doing some trimming with regex ... }
It actually does fetching more than 1000000 times. so, it takes long time. This is why i am trying to learn threads. I will use 10 threads.

However, I have $i value and am using it in the loop, so if I create 10 threads, it is doing same loop 10 times. I need something, which will do like;

for first thread :

http://www.somewebsite.com/news.php?id=1
http://www.somewebsite.com/news.php?id=11
http://www.somewebsite.com/news.php?id=21
http://www.somewebsite.com/news.php?id=31
...

for second thread

http://www.somewebsite.com/news.php?id=2
http://www.somewebsite.com/news.php?id=12
http://www.somewebsite.com/news.php?id=22
http://www.somewebsite.com/news.php?id=32

for third thread

http://www.somewebsite.com/news.php?id=3
http://www.somewebsite.com/news.php?id=13
http://www.somewebsite.com/news.php?id=23
http://www.somewebsite.com/news.php?id=33

.. and so on..

I have created 10 different sub routines and loops. Every loop is doing a part of fetching like;

first sub 1-100000
second sub 100000-200000
third sub 300000-400000

but it is not an elegant way of course. I would like to learn the correct format. i tried many things but could not success.

I also would like to change number of threads, so I guess, I also need to create a for loop for threads to do so. Then i can also change number of threads, but how?

Thanks from now on,

Replies are listed 'Best First'.
Re: Using threads in a for loop
by ikegami (Patriarch) on Sep 11, 2008 at 22:33 UTC

    but it is not an elegant way of course. I would like to learn the correct format.

    Have you considered a work queue?

Re: Using threads in a for loop
by NetWallah (Canon) on Sep 12, 2008 at 05:11 UTC
    if you need working thread code to get you started, it is available here.

    The code allows you to specify how many threads you want to start, and provides a mechanism to feed the threads data, and extract results.

         Have you been high today? I see the nuns are gay! My brother yelled to me...I love you inside Ed - Benny Lava, by Buffalax