I'm glad that you found a solution. I had been working on this before you made your post. I thought that I'd post it just in case it gives you some ideas.
I used Parallel::Runner
to run 5 scrapers
#!/usr/bin/perl
use strict;
use warnings;
use Parallel::Runner;
open STDOUT, '>', 'ssh.log'
or die "Couldn't open ssh.log: $!\n";
my $runner = Parallel::Runner->new(5);
$runner->run( sub { system("perl scraper1.pl") });
$runner->run( sub { system("perl scraper2.pl") });
$runner->run( sub { system("perl scraper3.pl") });
$runner->run( sub { system("perl scraper4.pl") });
$runner->run( sub { system("perl scraper5.pl") });
$runner->finish;
Then I used this to check the log
#!/usr/bin/perl
use strict;
use warnings;
use Net::SSH::Perl;
connectToServer();
sub connectToServer {
my $host = 'remote_server';
my $username = 'user';
my $password = 'password';
my $cmd = 'cat /path/to/ssh.log';
my $ssh = Net::SSH::Perl->new(
$host,
protocol => '2,1',
debug => 1,
);
$ssh->login( $username, $password );
my ( $stdout, $stderr, $exit ) = $ssh->cmd($cmd);
print $stdout, "\n";
}
|