in reply to Sharing DBI between threads

The author of DBI recommends against using it with a threaded perl. He thinks it's such a bad idea that when you try to build it, you get this warning: 
*** You are using a perl configured with threading enabled.
*** You should be aware that using multiple threads is
*** not recommended for production environments.

[download]

Replies are listed 'Best First'.
Re^2: Sharing DBI between threads
by BrowserUk (Patriarch) on Jan 21, 2010 at 14:39 UTC

    Every program is a threaded program, even if it only has one thread.

    If DBI works correctly in all of those, then if some of them create a second thread: 

    use DBI;
use threads;

async { sleep 1 while 1 }->detach;

# the rest of the script.

    [download]

    it'll continue to work correctly. Provided you don't use DBI within that other thread!

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
Re^2: Sharing DBI between threads
by bagent (Novice) on Jan 21, 2010 at 23:01 UTC
    Then what technology would you suggest for project like this? Is there any interpreter that has stable production ready support for multithreading? I had a look at ruby and python. Ruby seems like a good compromise between Perl with its extremely rich syntax and Python with its oversimplified syntax. However ruby's port of WWW::Mechanize is in alpha release as well as curb (curl bindings). Also it is new and has very poor documentation in contrast with Perl. However, ruby's syntax is much more clear in contrast with Perl's, Perl has really unsafe features - for example it allows interchange of HASH references and integers without any warning even with `use strict;` - something that even C doesn't allow. Also need for using all this complex references for nested arrays/hashes seems very unclear for me as mostly PHP programmer. I couldn't use PHP for this project because it seems PHP lacks any support for threads at all. Now I see threads in other languages are also experimental. Finally, I know that the best ways for this job would be to use Async I/O however this is much more complex solution. So, I am basically in search for technology that has something like WWW::Mechanize and good production ready support for threading.
[reply]

      The problem is not Perl, and the problem is not thread support in Perl. Ikegami explained the real problem in Re: Sharing DBI between threads: A single database connection can not know which of the various threads sends commands, thus the database gets confused or damaged. You need a dedicated database connection for each single thread, period. (And you need it in every language that allows you to use threads and a database connection.)

      DBI WARNS not to use a threaded perl in a production environment. This does not mean DBI does not work with a threaded perl. It means that there MAY BE some issues with a threaded perl. You should perhaps read that warning as a better-safe-than-sorry warning: "it will very probably work as well, but we can not guarantee that as well as for a non-threaded perl". Every recent perl for Windows (with faked fork() support) is threaded, as are most perls built by the various Linux distributions. And all of those perls run pretty well with the DBI.

      Perl has really unsafe features - for example it allows interchange of HASH references and integers without any warning even with `use strict;`

      This sounds pretty strange, like a misunderstanding of some essential concepts of Perl. If not, could you show us a working example, and a copy of perl -V that exhibits that problem?

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
[reply]
[d/l]
[select]
        it allows interchange of HASH references and integers

        I'm guessing that he means: 

        print $href, '-', 0+$href;;
HASH(0xacaa0) - 707232

        [download]

        But as you say misunderstands that you can't (easily) go the other way. Also, perhaps he forgets that C allows you to cast a pointer to an integer; but more significantly cast an integer to a pointer (to anything).

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

        > This sounds pretty strange,
        > like a misunderstanding of
        > some essential concepts of Perl.
        > If not, could you show us a working example,
        > and a copy of perl -V that exhibits that problem?

        #!/usr/bin/perl -w
use strict;

my %results = (
        'results'       =>      [
                [10, 13, 14]
        ],
);

# Add array reference to integer number without a warning,
# result is a big number, 10896017 on my architecture
print 1 + $results{'results'}->[0], "\n";

# print number of elements in array + 1
print 1 + scalar(@{$results{'results'}->[0]}), "\n";

# forgot @{, produces big number (10896017)
# very easy to do mistake
# can lead to hours of bughunting
print 1 + scalar($results{'results'}->[0]), "\n";

        [download]
        Not sure why `perl -V` is needed but here it is: 
        par@maxlance ~/projects $ perl -V
Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.31-gentoo-r6, archname=x86_64-linux-threa
+d-multi
    uname='linux maxlance.homelinux.org 2.6.31-gentoo-r6 #9 smp wed de
+c 23 03:27:57 msk 2009 x86_64 amd athlon(tm) 64 x2 dual core processo
+r 5400+ authenticamd gnulinux '
    config_args='-des -Darchname=x86_64-linux-thread -Dcccdlflags=-fPI
+C -Dccdlflags=-rdynamic -Dcc=x86_64-pc-linux-gnu-gcc -Dprefix=/usr -D
+vendorprefix=/usr -Dsiteprefix=/usr -Dlocincpth=  -Doptimize=-O2 -pip
+e -march=athlon64 -Duselargefiles -Dd_semctl_semun -Dscriptdir=/usr/b
+in -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dinst
+allman1dir=/usr/share/man/man1 -Dinstallman3dir=/usr/share/man/man3 -
+Dman1ext=1 -Dman3ext=3pm -Dinc_version_list=5.8.0 5.8.0/x86_64-linux-
+thread-multi 5.8.2 5.8.2/x86_64-linux-thread-multi 5.8.4 5.8.4/x86_64
+-linux-thread-multi 5.8.5 5.8.5/x86_64-linux-thread-multi 5.8.6 5.8.6
+/x86_64-linux-thread-multi 5.8.7 5.8.7/x86_64-linux-thread-multi  -Dc
+f_by=Gentoo -Ud_csh -Dusenm -Dusethreads -Di_ndbm -Di_gdbm -Di_db -Du
+srinc=/usr/include -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemulti
+plicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=define uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='x86_64-pc-linux-gnu-gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE
+ -DTHREADS_HAVE_PIDS -fno-strict-aliasing -pipe -Wdeclaration-after-s
+tatement -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gd
+bm',
    optimize='-O2 -pipe -march=athlon64',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-stri
+ct-aliasing -pipe -Wdeclaration-after-statement -I/usr/include/gdbm'
    ccversion='', gccversion='4.3.4', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1
+6
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
+ lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='x86_64-pc-linux-gnu-gcc -O2 -pipe -march=athlon64', ldflags ='
+ -L/usr/local/lib64'
    libpth=/usr/local/lib64 /lib64 /usr/lib64
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.9.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.9'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynami
+c'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64'


Characteristics of this binary (from libperl): 
  Compile-time options: MULTIPLICITY PERL_IMPLICIT_CONTEXT
                        PERL_MALLOC_WRAP THREADS_HAVE_PIDS USE_64_BIT_
+ALL
                        USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
                        USE_PERLIO USE_REENTRANT_API
  Built under linux
  Compiled at Jan 14 2010 11:10:33
  @INC:
    /etc/perl
    /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi
    /usr/lib64/perl5/vendor_perl/5.8.8
    /usr/lib64/perl5/vendor_perl
    /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi
    /usr/lib64/perl5/site_perl/5.8.8
    /usr/lib64/perl5/site_perl
    /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi
    /usr/lib64/perl5/5.8.8
    /usr/local/lib/site_perl
    .
par@maxlance ~/projects $

        [download]
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom