Threaded recursive program seg faults

fx has asked for the wisdom of the Perl Monks concerning the following question:

Hi all,

I compiled perl, v5.6.1 built for i686-linux-thread using the CPAN module but am having problems with a program I am writing using threads.

There is one subroutine that downloads a webpage and looks for our web address. If a link to another webpage is found in the document the program starts a new thread using the subroutine to look for our address in this new page.

Pseduo code:

sub getWebPage
{
  my $url = $_[0];
  my $page = get($url);
  ...
  foreach (website link found in $page)
  {
    new Thread \&getWebPage, website;
  }
  ...
}
[download]

but there are two problems
(1) the subroutines called from the newly created threads do not work
and
(2) it seg faults

Can anyone offer some advice?

Comment on Threaded recursive program seg faults Select or Download Code

Replies are listed 'Best First'.
Re: Threaded recursive program seg faults by kschwab (Vicar) on Jul 29, 2001 at 09:40 UTC
Perl threads are currently "experimental". See README.threads for more info. Most likely, you don't want to use perl threads for anything but research. This will get better, but it's not there as of now. See LWP::Parallel for a non-threaded solution to pulling webpages in parallel.	[reply]
Re: Threaded recursive program seg faults by fx (Pilgrim) on Jul 29, 2001 at 22:26 UTC
The list of web pages to download is generated on the fly by examaning the current web page's data. This way only one web site address becomes available at a time. In this situation I am thinking that a parallel attempt would fail. Thoughts? Another question: is there any way I can find out why it is seg faulting? Perhaps it is the threads, perhaps it's something to do with the code... I've acutally had a few successfully completed runs of the code without fault, but when ran again on the same initial URL it fails.	[reply]
Re: Re: Threaded recursive program seg faults by kschwab (Vicar) on Jul 29, 2001 at 22:44 UTC
Read the README.threads link from my earlier node. Specifically: `Debugging Use the -DS command-line option to turn on debugging of the multi-threading code. Under Linux, that also turns on a quick hack I did to grab a bit of extra information from segfaults. If you have a fancier gdb/threads setup than I do then you'll have to delete the lines in perl.c which say #if defined(DEBUGGING) && defined(USE_THREADS) && defined(__linux_ +_) DEBUG_S(signal(SIGSEGV, (void(*)(int))catch_sigsegv);); #endif` [download]	[reply] [d/l]
Re: Threaded recursive program seg faults by trantor (Chaplain) on Jul 30, 2001 at 11:15 UTC
As others pointed out, threads are still experimental as of Perl 5.6.1. Also rememeber that, even when threads will be perfectly functional, programs cannot be automagically made mutli-threaded. Global variables not used wisely and non-reentrant functions are almost always causes of trouble. If, in your code, `get()` is not reentrant, you will certainly experience data consistency problems or even segfaults. -- TMTOWTDI	[reply] [d/l]