Re^2: using parallel processing to concatenate a string, where order of concatenation doesn't matter

Thanks, that's intriguing, but my perl doesn't seem to like that code.

hartman@ds0207:~/pfmArena> perl test2.pl
Bogus Parallel::Queue: "fork" and "thread" are exclusive at test2.pl l
+ine 17
hartman@ds0207:~/pfmArena> perl -e '@ARGV=("test2.pl"); while (<>) {di
+e $_ if $. == 17}'
    my $mgr = Parallel::Queue->construct( 'thread' );
hartman@ds0207:~/pfmArena>

hartman@ds0207:~/pfmArena> perl -V | grep -i thread
    osname=linux, osvers=2.6.16, archname=i586-linux-thread-multi
    config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusr
+binperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Duseshrplib=tr
+ue -Doptimize=-O2 -march=i586 -mtune=i686 -fmessage-length=0 -Wall -D
+_FORTIFY_SOURCE=2 -g -Wall -pipe'
    usethreads=define use5005threads=undef useithreads=define usemulti
+plicity=define
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
+-DDEBUGGING -fno-strict-aliasing -pipe -Wdeclaration-after-statement 
+-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGI
+NG -fno-strict-aliasing -pipe -Wdeclaration-after-statement'
    libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -
+Wl,-rpath,/usr/lib/perl5/5.8.8/i586-linux-thread-multi/CORE'
                        PERL_MALLOC_WRAP THREADS_HAVE_PIDS USE_ITHREAD
+S
    /usr/lib/perl5/5.8.8/i586-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.8/i586-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.8/i586-linux-thread-multi
hartman@ds0207:~/pfmArena> perl -V | grep -i ithread
    usethreads=define use5005threads=undef useithreads=define usemulti
+plicity=define
                        PERL_MALLOC_WRAP THREADS_HAVE_PIDS USE_ITHREAD
+S
hartman@ds0207:~/pfmArena>
[download]

Any ideas?

Comment on Re^2: using parallel processing to concatenate a string, where order of concatenation doesn't matter Download Code

Replies are listed 'Best First'.
Re^3: using parallel processing to concatenate a string, where order of concatenation doesn't matter by diotalevi (Canon) on Oct 18, 2006 at 15:52 UTC
Tell it to use threading by default: `use Parallel::Queue 'thread';`. ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊	[reply] [d/l]
Re^4: using parallel processing to concatenate a string, where order of concatenation doesn't matter by tphyahoo (Vicar) on Oct 20, 2006 at 17:40 UTC
Can't... get it... to work... I tried as you suggested... hartman@ds0207:~/Desktop/misc/idealoSubversionArena/parallelListProces +sing> ./testpq.pl Can't locate object method "new" via package "Threads" (perhaps you fo +rgot to load "Threads"?) at /usr/lib/perl5/site_perl/5.8.8/Parallel/Q +ueue.pm line 197. hartman@ds0207:~/Desktop/misc/idealoSubversionArena/parallelListProces +sing> head testpq.pl #!/usr/bin/perl use threads::shared; use Parallel::Queue 'thread'; print concatenate_parallel( ['a' .. 'z'], 4 ) . "\n"; sub concatenate_parallel { my $result :shared; my @input = map { my $string = $_; [download] Also tried other permutations with spelling variations, but I feel like I'm stumblinb around in the dark. Parallel::Queue indicates that forking works, but threading is "still in progress". Are you able to run this code? I am really keen to make progress on this, because I am hoping to use this as a basis for `threadedreduce` which would serve as an abstraction layer for all sorts of list processing that could potentially be speeded up by using threading, but without changing anything in the "meat" of the program (which would be a function builder that uses `threadedreduce` or `nonthreadedreduce`, each having the same effect). See Could there be ThreadedMapReduce (and/or ForkedMapReduce) instead of DistributedMapReduce? and my followup, with `threadedreduce` stub code, at Re^2: Could there be a ThreadedMapReduce (instead of DistributedMapReduce)?	[reply] [d/l] [select]
Re^5: using parallel processing to concatenate a string, where order of concatenation doesn't matter by ikegami (Patriarch) on Oct 20, 2006 at 19:40 UTC
Fix the module as follows: Line 197: Change `Threads->new` to `Thread->new`. Line 292: Insert `require Thread;` before `require Thread::Semaphore;`. These fixes have not been tested, just like the module itself: Threads are unstested. Forks work; threads are still in progress. The code may work but I have not tested it yet. Update: Added spelling fix.	[reply] [d/l] [select]
Re^5: using parallel processing to concatenate a string, where order of concatenation doesn't matter by diotalevi (Canon) on Oct 20, 2006 at 19:25 UTC
Well... take out your debugger, look at the source of the module and make it go. Simple. ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊	[reply]