The system you are describing sounds strangely similar to one we were trying to build: Apache2/mod_perl2 + Perl iThreads, parallel searches to multiple threads, collecting data, returning it back to the user. The differences we had included spawning all threads at ChildInit and putting them into a wait loop, then signalling them when we wanted them to work. They'd go do stuff, put the data in a shared hash, then return to the wait loop.

What we found is that Perl is not thread-safe. Let me repeat that: PERL IS NOT THREAD-SAFE! I say this with emphasis because after several long nights hunting down segfaults in the system, turning on debugging, adding evals to trap errors, banging my head, I came across this in the log: 

Bizarre ARRAY copy in caller() at Common::Logger::log_stuff ...

[download]
Common::Logger is our internal logging package. log_stuff is the logging function. It calls CORE::caller(). This blew up. Evidently, the thread boundary is a bit like an event horizon and caller can't cross it without potentially breaking. I saw this message a few more times before we ditched threads entirely.

I strongly encourage you to investigate the possibility of dead threads fouling your app. Then look at alternatives to a threaded model. We moved to a multi-tiered Apache platform. We have an Apache "front-end" that talks to an Apache "back-end" (which in turn can talk to itself; we have layers of parallel and serially parallel processing).

When we mentioned to merlyn, TheDamien, and Nick Clark that we were looking at the threaded approach when we were at OSCON 2005, they all pretty much wished us luck. I do the same for you now.

Ivan Heffner
Sr. Software Engineer, DAS Lead
WhitePages.com, Inc.

In reply to Re: mod_perl, threads, and blank pages by Codon
in thread mod_perl, threads, and blank pages by richard5mith

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.