Re: Parallel SSH

Where did you read that Fork Manager works badly with SSH?

About 18 months ago, I was working on a database backup tool that made heavy use of both Parallel::ForkManager and ssh. I did not see any problems with using the two together.

The basic algorithm was: (pseudo-code)

use Parallel::ForkManager;
use Net::SSH::Perl;

$pm = new Parallel::ForkManager(10);

foreach my $server (@big_list)
{
    my $pid = $pm->start and next;
    
    ssh('root@'.$server, '/opt/bin/some_backup_script');
    
    $pm->finish;
}

print "All backups done";
[download]

Comment on Re: Parallel SSH Download Code

Replies are listed 'Best First'.
Re^2: Parallel SSH by salva (Canon) on Apr 11, 2011 at 12:31 UTC
The problem with Net::SSH::Perl and fork is that it does not support opening the connection from some process and then reusing it from its children.	[reply]
Re^3: Parallel SSH by chrestomanci (Priest) on Apr 11, 2011 at 15:00 UTC
OK, I see the difference. In my experience with the backup script above, I was forking first, and then opening ssh connections in the child processes. You are investigating opening ssh connections, and then forking afterwards. I think you will always find problems with this, however you do it. The problem is that when you fork both the parent and the child retain all open file descriptors including network sockets. For some of those file descriptors it is probably harmless, or even desirable behaviour that both parent and child keep the file descriptor, but in the case of a state-full protocol such as ssh, it will almost certainly lead to problems. Like any well engineered security product, ssh (RFC 4251) will include protection against replay attacks, most likely via some sort of sequence number. If you start and ssh connection, and then fork then both parent and child will inherit copies of the connection object, with a sequence number. If both parent and child then use their connections to talk to the server, then the ssh demon on the server will see the sequence number go backwards which would normally only happen if a cracker was attempting a replay attack, so the server will close or otherwise reject the connection.	[reply]
Re^4: Parallel SSH by salva (Canon) on Apr 12, 2011 at 08:45 UTC
It can be done in other ways. Actually, Net::OpenSSH does support sharing the SSH connection with children and other processes (the merit should be attributed to the underlaying OpenSSH).	[reply]