Re^4: Thread::Pool::Simple || !

BrowserUK - please check your attitude! It is unbecoming of a monk of your stature. Your paraphrase is wrong, actually damn wrong.

I have spent more than a little time trying to find out what is happening when the threads die. I know they die because they cease to log "anything". Other threads working on the same queue continue about their merry way. The thread(s) that die simply disappear akin to Amelia Earhart. And like the disappearance of Amelia Earhart, it may take more than a couple of decades to figure it out.

I am guessing that you have more than enough experience to understand how difficult it can be to find intermittent bugs. This is especially true if your own code isn't the sole issue. Over the last 30 years, I have worked on more hardware, OS, and development languages and framework than I care to remember. Without exception, they all at time to time exhibited behaviors that cannot readily, or even after a lot work, be explained.

At some point, I as a human, will throw in the towel on getting to the bottom of a problem if I can take a path around it or mitigate it in some means. I have decided that I will not, not will not bother to, expend any additional hours of my life that I cannot get back to research, test and identify the root cause of my issue.

CPAN is far from perfect. I do not like the amount of cruft that is out there in it. But it is in the wild. In the wild you will find snakes, pirate and liars as well as things of great beauty. I have found many modules in CPAN that I absolutely adore and use as if they were part of the perl CORE because they bring benefit to me that I want, need and appreciate. Even the modules written by snakes, pirate and liars are helpful to show me how "NOT" to do something.

I do wish that the perl community would do something about having a means to obtain a perspective on how well a module(s) does or doesn't work other than one off comments on CPAN. I would be more than happy to help. For my own sanity, I generally do not use a module if I can't find enough evidence of its consumption by others and indications that the code is maintained to merit considering it.

However, given the amount of time that CPAN has been in existence and the fact that this issue has existed since shortly after its establishment, I think it unlikely that it will be addressed soon. And if you already haven't looked, please do take a look at CPAN and see if you find any modules, flakey or otherwise put there by me. I have not put anything on CPAN because I could not get it up to my standards in the time that I have. I "will not" put anything on CPAN or elsewhere that isn't ready for consumption and that I won't support myself or via a support group.

Capiche?

lbe

Is this a rant? I hope not. I don't allow myself to rant -- in public. :)

Comment on Re^4: Thread::Pool::Simple \|\| !

Replies are listed 'Best First'.
Re^5: Thread::Pool::Simple \|\| ! by BrowserUk (Patriarch) on Jul 18, 2012 at 19:31 UTC
Ask yourself this. Would you accept a text editor that worked well, but occasionally died throwing away unsaved changes; provided it restarted itself and loaded up the last saved copy? Because that is exactly analogous to what you are looking to do with your threads. And are suggesting you might put on CPAN. With respect to tracking down your transient error(s); show me the code. I bet I can track it down. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply]
Re^6: Thread::Pool::Simple \|\| ! by learnedbyerror (Monk) on Jul 18, 2012 at 20:49 UTC
All problems are not the same. No, I would not accept that as a text editor. However, if I am looking at something that generates aggregate statistics and missing a single point and even a hundred of them and not affecting the precision of the result by more than by 0.0001% and the overall accuracy of the algorithm is +/- 0.1%, then yes. It is perfectly acceptable. Help I appreciate. A personal indictment on what I care about - not so much. In this forum, posters ask for critique on perl, nothing more, nothing less. Critical critique in an answer to a question is fine. Extraneous critique or judgement not germaine to the question is superfluous - uneccesary and needless, and may call into question that credibility of the person speaking. Given that it is election season here in the U.S., there are plenty of superfluous statements, observations and critiques available. And as your signature line so aptly states: "In the absence of evidence, opinion is indistinguishable from prejudice." lbe	[reply]
Re^7: Thread::Pool::Simple \|\| ! by BrowserUk (Patriarch) on Jul 18, 2012 at 21:17 UTC
All problems are not the same. No, I would not accept that as a text editor. However, if I am looking at something ... And that is the gist of my argument against a generic thread pool module. Each thread pool application has a different set of requirements, making it impossible to serve them all -- or even a substantial subset -- with a single, simple API. And by the time you have extended the API to cater for sufficient variations for it to be considered a general purpose module, the API has become so complex that using it is as much, if not more effort, than hand-rolling your own pool. Which at the simplest level can be done in around 10 lines of code. that generates aggregate statistics and missing a single point and even a hundred of them and not affecting the precision of the result by more than by 0.0001% and the overall accuracy of the algorithm is +/- 0.1%, then yes. It is perfectly acceptable. Acceptable for your application; but how many others? And, wouldn't it also be acceptable to your application if it didn't miss any points? Wouldn't fixing whatever it is causing the loss of those points not only be acceptable to your application, but also mean that it might make your module useful for other applications with more critical requirements? A personal indictment on what I care about ... Extraneous critique or judgement not germaine ... Okay. I was trying to bring this back around to the code, but you seem to want to get into the meta discussion. So be it. What "Extraneous critique or judgement"? Please quote. Please explain why you have taken the quote as questioning your credibility; rather than as technical critique of the ideas you expressed in your post? With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply]
Re^5: Thread::Pool::Simple \|\| ! by bulk88 (Priest) on Jul 18, 2012 at 23:10 UTC
I am guessing that you have more than enough experience to understand how difficult it can be to find intermittent bugs. This is especially true if your own code isn't the sole issue. Over the last 30 years, I have worked on more hardware, OS, and development languages and framework than I care to remember. Without exception, they all at time to time exhibited behaviors that cannot readily, or even after a lot work, be explained. At some point, I as a human, will throw in the towel on getting to the bottom of a problem if I can take a path around it or mitigate it in some means. I have decided that I will not, not will not bother to, expend any additional hours of my life that I cannot get back to research, test and identify the root cause of my issue. Then surly you know how to fix this problem. Use a C debugger, attach to the OS thread, look at the callstack, and fix it. You probably need a DEBUGGING build of perl so your C debugger works cleaner. No OS thread can "freeze" itself or deschedule itself without help from the OS. If the thread busy waiting sucking cpu, ps/top/task manager will tell you. If your threads are "disappearing" (I can't tell if you mean they disappear from the OS, or they simply freeze indefinitely) without a trace, chances are very high your leaking resources and ram too. Since your use Unix, have your tried setting signal SEGV/BUS/FPE/ABRT/other CPU exceptions from perl to see if your thread is throwing one of those signals? You haven't shown any code, or explained what specific CPAN modules and C libraries and XS modules you are using. I'll make a wild ass guess and say you probably have what is called a race condition in a 3rd party C library (access vio or thread sync/mutex deadlock), or you are doing network I/O with no timeout. I'm not sure exactly how process/user resource limits work on Linux (I'm a Win32 person). I've read that the Linux kernel doesn't know what threads are, each thread is a separate process in the same memory sandbox. So maybe one of your OS threads/ithreads hit the OS per thread resource limit and thats why it disappears (if disappearing is what happens on Linux when 1 thread in a multi threaded process hits a per thread resource limit). If you showed code, I bet BrowserUk could track it down as he claims in Re^5: Thread::Pool::Simple \|\| !.	[reply]
Re^5: Thread::Pool::Simple \|\| ! by BrowserUk (Patriarch) on Jul 18, 2012 at 19:21 UTC
BrowserUK - please check your attitude! {Check} S'fine thanks. I was trying to help you help yourself. You don't want it. S'fine too. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply]