adornach has asked for the wisdom of the Perl Monks concerning the following question:

A couple disclaimers:

* I'm not a developer nor a sys admin but had to learn PERL to get some stuff done

* I have about 2-3 weeks of self taught experience with PERL and using remote servers

So, with that out of the way, here's my situation.

I've created some few scraping scripts that I've installed on a remote Linux server (Ubuntu Hardy) which I've been executing manually after establishing an SSH connection from my local (home) laptop to the server. The problem I'm having is that the SSH sessions really chug up my machine as I have to leave the scripts and the session running for a couple days. During these periods, my lappy so is so freaking slow and having to do my actual job (UI design) becomes really tedious. So, is there another way to execute those scripts remotely without using SSH so I can continue using my laptop as normal and just periodically check the remote db to see what data has been collected?
  • Comment on Is SSH the only way to execute remote, one off PERL scripts?

Replies are listed 'Best First'.
Re: Is SSH the only way to execute remote, one off PERL scripts?
by Corion (Patriarch) on Jul 24, 2010 at 08:59 UTC

    You can run the scripts remotely detached, for example using GNU screen or simply by running them as nohup:

    nohup myscript.pl 2>&1 >myscript.log &

    I think you can even do that inline using ssh:

    ssh somehost 'nohup myscript.pl 2>&1 >myscript.log &'
Re: Is SSH the only way to execute remote, one off PERL scripts?
by salva (Canon) on Jul 24, 2010 at 09:01 UTC
    The problem I'm having is that the SSH sessions really chug up my machine
    Then you are doing something wrong, having some SSH connections open should be unnoticeable.

    Post your code and we will be able to tell you how to improve it or show better alternatives.

Re: Is SSH the only way to execute remote, one off PERL scripts?
by JavaFan (Canon) on Jul 24, 2010 at 11:09 UTC
    So, is there another way to execute those scripts remotely without using SSH so I can continue using my laptop as normal and just periodically check the remote db to see what data has been collected?
    Of course. Probably the most used way to start a remote Perl program is done using HTTP, and something like CGI on the remote side. But there are dozens of other possibilities. Almost any network connection triggers something on the remote site.
Re: Is SSH the only way to execute remote, one off PERL scripts?
by intel (Beadle) on Jul 24, 2010 at 15:30 UTC
    If you're using Net::SSH::Perl then you're re-creating ssh on your machine, and all the encryption that entails in Perl, though Math::BigInt and Pari.

    If you're using Net::OpenSSH you get the same functionality using ubuntu's implementation of SSH, which makes things a lot faster.

    But, again, you should post your code, as always here at PerlMonks.

      Thanks everyone for the replies.

      A quick update:

      I was indeed doing something wrong. I was printing out error and status messages for each step in my parsing loops. For a big scrape job like the one I'm in the midst of, this could be hundreds of thousands of lines which was causing Terminal to chew up RAM.

      So, I've commented out all but a couple critical lines and immediate symptom solved.

      However, I think the nohup suggestion will actually solve the problem. Not having to keep a terminal window and SSH session open after executing a script sounds like a dream!

      Thanks again!

        I'm glad that you found a solution. I had been working on this before you made your post. I thought that I'd post it just in case it gives you some ideas.

        I used Parallel::Runner to run 5 scrapers

        #!/usr/bin/perl use strict; use warnings; use Parallel::Runner; open STDOUT, '>', 'ssh.log' or die "Couldn't open ssh.log: $!\n"; my $runner = Parallel::Runner->new(5); $runner->run( sub { system("perl scraper1.pl") }); $runner->run( sub { system("perl scraper2.pl") }); $runner->run( sub { system("perl scraper3.pl") }); $runner->run( sub { system("perl scraper4.pl") }); $runner->run( sub { system("perl scraper5.pl") }); $runner->finish;
        Then I used this to check the log
        #!/usr/bin/perl use strict; use warnings; use Net::SSH::Perl; connectToServer(); sub connectToServer { my $host = 'remote_server'; my $username = 'user'; my $password = 'password'; my $cmd = 'cat /path/to/ssh.log'; my $ssh = Net::SSH::Perl->new( $host, protocol => '2,1', debug => 1, ); $ssh->login( $username, $password ); my ( $stdout, $stderr, $exit ) = $ssh->cmd($cmd); print $stdout, "\n"; }