Thread-safe: just don't share handles simultaneously
[download]

You could follow a suggestion BrowserUK gave to me - put the non-thread safe bits of your code in another script, then use open() (or IPC::Open2 if you need bi-di communication) in the original script to communicate with the second script in another process, thus bypassing any thread safety issues.

My issue was with Crypt::SSLeay but the solution should work well for you too.

Thank you!

This solution works great, so thats what I am using. It may be a little inefficient 'using' $threads * Net::SSH2, but so far resources is not an issue

-Jesper

use Net::OpenSSH::Parallel;

my $pssh = Net::OpenSSH::Parallel->new;
while (my $element = <MYINPUTELEMENTS>) {    
    if ($element =~ /(\d+)(\.\d+){3}/) {        
        $pssh->add_host($element, user => $user, password => $password
+);
    }
}

$pssh->all(cmd => { stdout_file => 'gss-%HOST%' },  'ls -l / | wc -l')
+;
$pssh->all(cmd => { stdout_file => 'lcs2-%HOST%' }, 'ls -l /etc | wc -
+l');

$pssh->run;

my %errors = $pssh->get_errors;
for my $host (keys %errors) {
    print "task failed for host $host: $errors{$host}\n"
}
[download]

Thanks for the answer, however parallel would slow things down considerably. I need to control the number of threads spawned, thus I would need to pass it into pssh in batches of $threads. Running parallel I would then need to wait for all of them to finish before sending a new batch. The completion time of the commands on each node can vary with factor 10 or more.

I'll keep this in mind for a future project tho, the code is significantly easier than the threading. :-)

-Jesper

I need to control the number of threads spawned, thus I would need to pass it into pssh in batches of $threads.

Net::OpenSSH::Parallel already does that for you. For instance, if you want to limit the number of SSH connections to 10, you can use:

my $pssh = Net::OpenSSH::Parallel->new(connections => 10);
[download]

Before you move to using processes, you could try this (untested) alternative formulation and see what happens:

#!/usr/bin/perl -w
use strict;
use threads;
use Thread::Queue;

my $Q = new Thread::Queue;
my @pool = map threads->create( \&worker_parse, $Q ), 1 .. 10;

open (MYINPUTELEMENTS, "./list.txt");

while( my $element = <MYINPUTELEMENTS> ) {
    if( $element =~ /(\d+)(\.\d+){3}/ ) {
        $Q->enqueue( $element );
    }
}
close (MYINPUTELEMENTS);

$Q->enqueue( (undef) x 10 );
$_->join for @pool;

sub worker_parse {
    my $Q = shift;
    require 'Net::SSH2';

    while( my $inIP = $Q->dequeue ) {
        chomp( $inIP );
        my $ssh2 = Net::SSH2->new();

        if ($ssh2->connect($inIP)) {
            if ($ssh2->auth_password('user','pass')) {
                my $gssChannel = $ssh2->channel();
                $gssChannel->exec('ls -l / | wc -l');
                my $gssData;
                my $gssLen = $gssChannel->read($gssData,8192);
                chomp($gssData); print "gss: $gssData - ";
                $gssChannel->close;

                my $lcsChannel = $ssh2->channel();
                $lcsChannel->exec('ls -l /etc | wc -l');
                my $lcs2Data;
                my $lcs2Len = $lcsChannel->read($lcs2Data,8192);
                chomp($lcs2Data); print "lcs2: $lcs2Data\n";
                $lcsChannel->close;
                $ssh2->disconnect;
            } else {
                print "Authentication Failed\n";
            }
        } else {
            print "Connection Failed\n";
        }
    }
}
[download]

Thanks, but unfortunally it still breaks at random iterations during the execution with the same error as before. But good idea, I hadn't tried that. :-)

-Jesper

Worth a try.

I'm beginning to suspect that it is one of the math/crypto libraries that is used by many/all of these SSL modules that is the root source of the thread-unsafeness.

Update: Indeed, it seems to be a bug or design deficiency in the GnuTLS library: google

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

I'm running this on perl 5.12.4 on Ubuntu 11.10. Please help me save the little hair

Make sure your Ubuntu version of Perl is compiled to use threads.... I seem to recall Ubuntu may install a non-threaded Perl as default.

#!/usr/bin/perl
use warnings;
use strict;
use threads;

my $thr;
for(1..10){
   $thr = threads->new( \&sub1 )->detach; # Spawn the thread
}

while(1){ 

  my $thread_count = threads->list();
  print "\n\n\n\t\t",'num threads ', $thread_count, "\n\n\n";
  print "Hit control c to exit \n\n\n\n";
  sleep 1;
 }

exit;

sub sub1 {
use Net::SSH2;

# assuming a user named 'z' for demonstration 
# connecting to localhost, so you need your sshd running

# see maillist archives at
# http://lists.sourceforge.net/lists/listinfo/ssh-sftp-perl-users
# for deeper discussions

my $self = threads->tid();

my $ssh2 = Net::SSH2->new();
$ssh2->connect('localhost') or die "Unable to connect Host $@ \n";
$ssh2->auth_password('z','ztester') or die "Unable to login $@ \n";

#shell use
my $chan = $ssh2->channel();
$chan->blocking(0);
$chan->shell();
print $chan "ls -la\n";
print "thread $self : $_" while <$chan>;

print $chan "who\n";
print "thread $self  : $_" while <$chan>;

print $chan "date\n";
print "thread $self  : $_" while <$chan>;
$chan->close;
}
[download]

Hi there!

It is compiled with threads, and threads are working when not using Net::SSH2. The mind boggles :-)

I need some sort of thread control, so Thread::Queue or Thread::Pool is needed. Currently using Thread::Queue with the suggestion to spawn processes in the above, and it is working fine.

I am going to try this one out as well, slightly modified for thread control - It will certainly be more resourcefriendly if I can get it to work, so thank you very much for your answer.

-Jesper