Prugnaud has asked for the wisdom of the Perl Monks concerning the following question:

Hello,
We are using PERL 5.8 and some multithread functions.
Unfortunately we have stability issues.
Has anyone heard about similar issues and if yes which workaround solutions have been used?
Let me know if you need more information on the issues.
Many Thanks in advance,
JC
**************************
Here is an update of the question:*
**************************
First of all thanks a lot for all the comments you already made.
At last, I can come back with more details and more ...questions.
I aplogise in advance for this long email.
1/ Perl 5.8.2 was built from source, using gcc 2.95.3, on a SUN E420 (Solaris 7) with multithreading support and ithreads options selected.
The following modules were added in order to get the scheduler.pl script to run:

HTML::Tagset
HTML::Parser
URI
LIBWWW
Date::Calc
Tie::DBI
MIME::Lite
DBI * This issues a warning that multithreaded Perl is not suitable for production environments *
DBD:Oracle


So when setting up the script this warning message came up.
Sould we ignore this meesage?
Because on the other hand when looking at the PERL doc on this site http://www.perldoc.com/perl5.8.0/pod/perlthrtut.html#Status
it is said "The ithreads code has been available since Perl 5.6.0, and is considered stable".
Who should we trust?

2/ Here is the error message generated:

Attempt to free unreferenced scalar: SV 0x12289c at /opt/ActivePerl-5. +8.2/lib/site_perl/5.8.2/sun4-solaris-thread-multi/DBI.pm line 626. Attempt to free unreferenced scalar: SV 0x12289c at /opt/ActivePerl-5. +8.2/lib/5.8.2/sun4-solaris-thread-multi/IO/Socket.pm line 46.

3/ I am going to try to generate a report using perl -d as kindly suggested by Zaxo.

4/ When a error happens within the programme the following function is executed:

sub mydie{ kill "SIGTERM", $$; # suicide! }

Could someone tell me if it means that all the child threads are immediately killed or only the process where the kill function.
If this mydie function is used in the parent process:
- will it free the ressources allocated to the child thread?
- Will it terminated eventually the child thread as well?

5/ The Threads need to write in a table in an Oracle Database.
Even though a DB connection has been opened in the main programme, we need for each Thread to reopen.
Is it a normal behavior?

6/ Looking at the code below do you foresee any likely issue?
The points that concern me are:
- The database connection is never properly closed.
Then database connection should remain pending till Oracle clear them.
- It is possible to go more than 2 times in the main loop.
But the maximum thread is 2 so after two be created what happens?
Does the program hang on the down() method of the do_job function until one is released.
- In the do_job function, the Thread is created and then the method detach() is called.
Will this call be sequential? Isn't there a risk the thread be cleared before the thread process be properly finished?
- It might be possible to have 2 threads running. Suppose one is going in the exception case of the the_job function.
Then what happens there? Is the Kill function killing the current child or everything?
- Supposing the programme runs correctly till the end.
Then when the instruction "$childs->down($maxchilds)" is executed, will the memory used by the Threads be released?

my ($data, $dbh); my %locks : shared = (); my $childs = new Thread::Semaphore(2); my $db; # Database-Hash # my %results; # Hash for results-db my %jobs; # additional information # if marked here, job will not be tryed again today my %jobs_done : shared = (); # associate hash with Database $db = tiedb(); my $jobs = 0; while(1){ # which job to do next? my ($id,$time) = get_next_job_Id_And_Frequency(); /* look up into +a table */ last unless $id; $jobs++; do_job($id, $time); } # cleaning up $childs->down($maxchilds); close LOCK or mydie("cant close $lockfile: $!"); unlink $lockfile or mydie("cant delete $lockfile: $!"); sub do_job { my ($id, $time) = @_; $childs->down(); my $thread = threads->create(\&the_job, $id, $time); $thread->detach(); } sub the_job { lock(%jobs_done); $jobs_done{$id} = 1; // Some Job is made here If Failure then kill "SIGTERM", $$; # suicide! $childs->up(); # increment semaphore } sub tiedb{ my %db; tie %db, 'Tie::DBI', { db => $db_source, table => $db_table, key => 'id', user => $db_user, password => $db_passwd, CLOBBER => 1, AUTOCOMMIT => 1, # DEBUG => 1, }; # set DATE-format from Database my $dbh = tied(%db)->dbh; my $handle = $dbh->prepare($db_setdate); $handle->execute(); $handle->finish(); return \%db; }


THANKS!

20040128 Edit by Corion: Added code tags

Replies are listed 'Best First'.
Re: MultiThread possible issues - basic questions - Thanks
by rob_au (Abbot) on Jan 28, 2004 at 08:50 UTC
    With regards to your questions regarding DBI thread safety and code implementations, I would strongly recommend incorporating the creation and destruction of your database handle into your job code - This is because of the behaviour of the DBI in threaded environments. In the interpreter threads (iThreads) environment, the new thread is created through the cloning of the interpreter and assignment of a single interpreter per application thread. Subsequently, as described in the DBI documentation, if the database handle and drivers are loaded and handles created before the thread is created then the new interpreter thread will only receive a cloned copy of the DBI, the drivers and the handles rather than a denovo copy. When this occurs however, the internal pointer data within the handles will refer to the DBI and drivers in the original interpreter, which in turn causes the interpreter thread to croak upon the use of any method call using handles that don't belong to the current thread.

    The error messages which you are seeing (Attempt to free unreferenced scalar ...) are in fact the result of the the interpreter thread croaking upon employ of methods within the database driver handle and subsequently attempting to free resources which have been cloned from the parent thread.

    In short, DBI handles cannot be shared between interpreter threads and as such each thread should create it's own database handle upon creation.

     

    perl -le "print unpack'N', pack'B32', '00000000000000000000001010110111'"

Re: MultiThread possible issues - basic questions - Thanks
by tilly (Archbishop) on Jan 28, 2004 at 19:32 UTC
    Answers:
    1. I'm inclined to trust DBI. 2 reasons. One, I'm still a little dubious about the stability of Perl's threading. Two (and more importantly), even if Perl's threading model is fine, any binary module that you load may or may not be threadsafe. Odds are that the DBI authors are suggesting that either DBI or the DBD::* driver that you use may not be threadsafe. I wouldn't dismiss that.
    2. I don't know the reason for this. rob_au's theory sounds as good as any.
    3. That wasn't a question, so I'm not answering it. ;-)
    4. Part of the answer is "no", part is "yes", and part is "maybe". Any externally allocated interactions (eg with a database) may or may not get cleaned up properly when you just go. Any internal memory allocation should get cleaned up when you go. As for who dies, I think that that depends on the threading model that the operating system uses. In particular do all threads share the same process ID or not? Different implementations of POSIX threads differ on this. I suspect that the version that you get on Solaris by default is one of the threading models where you can have multiple process IDs per thread, which suggests that you may only kill some of your threads. But read the documentation on your operating system's threading model before quoting me on a vague memory about something that I haven't worked with.
    5. Yes. If multiple database handles share a socket, then the database has no way to sort out what is happening when multiple threads babble simultaneously on that socket. So each thread needs its own socket and hence its own database handle.
    6. The code that you have written is not all Perl. For instance // is not a comment, and If Failure Then... is some other language entirely. Beyond that, my big concerns are internal to DBI, and what might happen in things like calls to do_job. (eg If you spawn threads there, then you have trouble.)