Re^4: ithreads picks fight with LWP::Agent; everybody loses (Works for me!)

Thanks for the code and for your help. You raise a couple of points.

First, the premature thread termination -- yeah, I spotted that. In the real code this failure script is based off of, I do indeed take care to make sure all threads have finished before termination, so I was quite certain that wasn't the source of my trouble and left it out of this code. (Besides, on my server, the code seg faults before it ever becomes an issue.) Sorry, I should have made that clearer.

Second, your code. I had to tweak it slightly to run (had to set stack size to 4*4096, and s/uleep/usleep/), but once it ran, it gave me a more or less instant seg fault, though in much less detail than I'm seeing in my own code:

Started: 0 Ended: 0 Segmentation fault

Though the "instant segfault" feature should prove valuable to my debugging, since it beats waiting for my own code to finally get around to deciding to die.

(Whoops, I just realized I never specified that. In both my failure script and the real code, the crash isn't instantaneous; it runs a little while first. Sorry, that was relevant and should have been mentioned in my initial posting.)

Finally, thanks for the 1.67 recommendation; I'll definitely give that a try.

Comment on Re^4: ithreads picks fight with LWP::Agent; everybody loses (Works for me!) Download Code

Replies are listed 'Best First'.
Re^5: ithreads picks fight with LWP::Agent; everybody loses (Works for me!) by BrowserUk (Patriarch) on Feb 27, 2008 at 07:52 UTC
(had to set stack size to 44096, and s/uleep/usleep/)* Sorry about the `uleep`, I changed it in the browser from the Win32::Sleep I used here. I'm surprised you needed to increase the stack size? Your threads don't do much that would consume stack. Or is that a requirement of a 64-bit processor? The main change I made in that version, beyond the logging, was to bypass perl's cleanup. It is usually during this phase that the mysterious, and for the most part inconsequential, "free to wrong pool" errors usually occur. Hence the reason you have to wait so long. The other change was to use shared variables. I suspect that it is this that is the cause of the premature segfault. You could try omitting them and see if bypassing the cleanup with `POSIX::_exit();` helps. The problem here is that if you join the threads, the cleanup is triggered early and the problem will arise. If you detach them, your main thread has nothing to wait on. That's what I used the shared vars and monitor thread to achieve. Generally, I don't expect that iThreads have been exercised much on 64-bit builds. Finally, I wouldn't recommend threads or forks for this type of usage. Both are to heavy for this purpose. An IO::Select or a LWP::Parallel will use far less resources for too much greater effort. Best of all, they are more likely to work in your environment. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^5: ithreads picks fight with LWP::Agent; everybody loses (Works for me!)
by BrowserUk (Patriarch) on Feb 27, 2008 at 07:52 UTC

(had to set stack size to 4*4096, and s/uleep/usleep/)

Sorry about the uleep, I changed it in the browser from the Win32::Sleep I used here.

I'm surprised you needed to increase the stack size? Your threads don't do much that would consume stack. Or is that a requirement of a 64-bit processor?

The main change I made in that version, beyond the logging, was to bypass perl's cleanup. It is usually during this phase that the mysterious, and for the most part inconsequential, "free to wrong pool" errors usually occur. Hence the reason you have to wait so long.

The other change was to use shared variables. I suspect that it is this that is the cause of the premature segfault. You could try omitting them and see if bypassing the cleanup with POSIX::_exit(); helps.

The problem here is that if you join the threads, the cleanup is triggered early and the problem will arise.

If you detach them, your main thread has nothing to wait on. That's what I used the shared vars and monitor thread to achieve.

Generally, I don't expect that iThreads have been exercised much on 64-bit builds.

Finally, I wouldn't recommend threads or forks for this type of usage. Both are to heavy for this purpose. An IO::Select or a LWP::Parallel will use far less resources for too much greater effort. Best of all, they are more likely to work in your environment.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

[reply]
[d/l]
[select]