Best module to execute administrator commands on external operating systems (Linux)

thanos1983 has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

Maybe the title is not so clear, or might produces confusion. Any alternative suggestions for the title are always welcome.

I am trying to create a script that it executes an administrator command on several external operating systems (Linux Servers).

I was looking on the web for the best possible Perl Module to execute commands externally and I think the best way is through SSH. Well I found the following the following modules that can apply ssh commands Net::Telnet, Net::SSH, Net::OpenSSH and Net::SSH::Perl.

Update: In the conf.ini file and also on the net.pl script

I found very useful to use the dump_log and input_log options for debugging purposes, I will post the contents under.

End of Update

I even wrote a script based on Net::Telnet because I thought it was the best. But I found out that can not execute sudo commands. Unless if I am doing something wrong.

Bellow I provide a full working code for testing purposes:

conf.ini (configuration file) Updated

[Device 1]
IP        = change
user      = change
psw       = change
prompt    = /[\$%#>] $/
port      = 22
dump_log  = dump_log.txt
input_log = input_log.txt 
hostname  = ubuntu

[Device 2]  
IP        = change
user      = change
psw       = change
prompt    = /[\$%#>] $/
port      = 22
dump_log  = dump_log.txt
input_log = input_log.txt 
hostname  = ubuntu
[download]

net.pl (Net::Telnet script) Updated

#!/usr/bin/perl
use strict;
use warnings;
use IO::Pty ();
use Data::Dumper;
use Net::Telnet ();
use Config::IniFiles;
use Fcntl qw(:flock);

$| = 1;

my $path = "conf.ini";
my $command = "sudo ntpdate ntp.ubuntu.com";

sub devices {
    
    open my $fh , '<' , "".$path."" 
    or die "Could not open file: ".$path." - $!\n";

    flock($fh, LOCK_SH)
    or die "Could not lock '".$fh."' - $!\n";

    tie my %ini, 'Config::IniFiles', ( -file => "".$path."" )
    or die "Error: IniFiles->new: @Config::IniFiles::errors";

    close ($fh)
    or die "Could not close '".$fh."' - $!\n";

    print Dumper(\%ini);

    my @keys = keys (%ini);
    my @data;
    my @loop;

    foreach my $hash (@keys) {

    @loop = clk_sync( $ini{$hash}{IP},
              $ini{$hash}{user},
              $ini{$hash}{psw},
              $ini{$hash}{prompt},
              $ini{$hash}{port},
              $ini{$hash}{dump_log},
              $ini{$hash}{input_log} );

    push(@data,@loop);

    }

    print Dumper(\@data);

    return @data;

} # end sub complex

my @list = devices();

sub spawn { 

    my (@cmd) = @_; 
    my ($pid, $pty, $tty, $tty_fd);
    
    ## Create a new pseudo terminal.
    $pty = new IO::Pty
        or die $!;

    ## Execute the program in another process.
    unless ($pid = fork) {  # child process
        die "problem spawning program: $!\n" unless defined $pid;

        ## Disassociate process from its controlling terminal.
        use POSIX ();
        POSIX::setsid()
            or die "setsid failed: $!";

        ## Associate process with a new controlling terminal.
        $pty->make_slave_controlling_terminal;
        $tty = $pty->slave;
        $tty_fd = $tty->fileno;
        close $pty;

        ## Make standard I/O use the new controlling terminal.
        open STDIN, "<&$tty_fd" or die $!;
        open STDOUT, ">&$tty_fd" or die $!;
        open STDERR, ">&STDOUT" or die $!;
        close $tty;

        ## Execute requested program.
        exec @cmd
            or die "problem executing $cmd[0]\n";
    } # end child process

    $pty;
} # end sub spawn

sub clk_sync {

my $host      = shift;
my $user      = shift;
my $passwd    = shift;
my $prompt    = shift;
my $port      = shift;
my $dump_log  = shift;
my $input_log = shift;
my ($buf, $match, $pty, $ssh, @lines);

## Start ssh program.
    $pty = spawn("ssh",
         "-l", $user,
         "-p", $port,
         "-e", "none",
         "-F", "/dev/null",
         "-o", "PreferredAuthentications=password",
         "-o", "NumberOfPasswordPrompts=1",
         "-o", "StrictHostKeyChecking=no",
         "-o", "UserKnownHostsFile=/dev/null",
         $host);

## Create a Net::Telnet object to perform I/O on ssh's tty.
    $ssh = new Net::Telnet (-fhopen                 => $pty,
                -prompt                 => $prompt,
                -telnetmode             => 0,
                -dump_log               => $dump_log,
                -input_log              => $input_log,
                -timeout                 => 20,
                -output_record_separator => "\r",
                -cmd_remove_mode => 1);

## Wait for the password prompt and send password.
    $ssh->waitfor(-match => '/password: ?$/i',
          -errmode => "return")
    or die "problem connecting to \"$host\": ", $ssh->lastline;
    $ssh->print($passwd);

## Wait for the shell prompt.
(undef, $match) = $ssh->waitfor(-match => $ssh->prompt,
                -match => '/^Permission denied/m',
                -errmode => "return")
    or return $ssh->error("login failed: expected shell prompt ",
              "doesn't match actual\n");
return $ssh->error("login failed: bad login-name or password\n")
    if $match =~ /^Permission denied/m;

## Run commands on remote host.
#@lines = $ssh->cmd("hostname");
@lines = $ssh->cmd("sudo ntpdate ntp.ubuntu.com");
#@lines = $ssh->cmd("uptime");

$ssh->close;

return @lines;

}

__END__

$VAR1 = {
          'Device 1' => {
                          'IP' => 'IP',
                          'user' => 'USER',
                          'psw' => 'PASSWORD',
                          'prompt' => '/[\\$%#>] $/',
                          'port' => 'PORT',
                          'hostname' => 'ubuntu'
                        },
          'Device 2' => {
                          'IP' => 'IP',
                          'user' => 'USER',
                          'psw' => 'PASSWORD',
                          'prompt' => '/[\\$%#>] $/',
                          'port' => 'PORT',
                          'hostname' => 'ubuntu'
                        }
        };
$VAR1 = [
          ' 04:23:41 up  4:25,  3 users,  load average: 0.12, 0.30, 0.
+42
',
          'Operating:~',
          ' 04:23:42 up  4:25,  3 users,  load average: 0.12, 0.30, 0.
+42
',
          'Operating:~'
        ];
[download]

In case that I remove the hash tag (#) on line 136 and execute the script with the (sudo ntpdate ntp.ubuntu.com) command then the output is.

$VAR1 = {
          'Device 1' => {
                          'IP' => 'IP',
                          'user' => 'USER',
                          'psw' => 'PASSWORD',
                          'prompt' => '/[\\$%#>] $/',
                          'port' => 'PORT',
                          'hostname' => 'ubuntu'
                        },
          'Device 2' => {
                          'IP' => 'IP',
                          'user' => 'USER',
                          'psw' => 'PASSWORD',
                          'prompt' => '/[\\$%#>] $/',
                          'port' => 'PORT',
                          'hostname' => 'ubuntu'
                        }
        };
$VAR1 = [
          ' 04:23:41 up  4:25,  3 users,  load average: 0.12, 0.30, 0.
+42
',
          'Operating:~',
          ' 04:23:42 up  4:25,  3 users,  load average: 0.12, 0.30, 0.
+42
',
          'Operating:~'
        ];

command timed-out at synchronization.pl line 136
[download]

input_log file output:

Warning: Permanently added '[IP]:22' (ECDSA) to the list of known host
+s.
user@IP's password: 
Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-34-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

0 packages can be updated.
0 updates are security updates.

Last login: Fri Aug 15 15:52:47 2014 from host-185-126.IP
]0;user@OS: ~user@OS:~$ sudo ntpdate ntp.ubuntu.com
[sudo] password for OS:
[download]

Based on what I see it expects for a second password input, but there is not prompt in the terminal to insert the password.

I also created a script in Net::SSH::Perl to test my luck.

Sample of code and output is provided under:

ssh.pl (NET::SSH::PERL script)

#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
use Net::SSH::Perl;
use Fcntl qw(:flock);
use Config::IniFiles;

my $path = 'conf.ini';

$| = 1;

sub devices {
    
    open my $fh , '<' , "".$path."" 
    or die "Could not open file: ".$path." - $!\n";

    flock($fh, LOCK_SH)
    or die "Could not lock '".$fh."' - $!\n";

    tie my %ini, 'Config::IniFiles', ( -file => "".$path."" )
    or die "Error: IniFiles->new: @Config::IniFiles::errors";

    close ($fh)
    or die "Could not close '".$fh."' - $!\n";

    print Dumper(\%ini);

    my @keys = keys (%ini);
    my @data;
    my @loop;

    foreach my $hash (@keys) {

    @loop = clk_sync( $ini{$hash}{IP},
              $ini{$hash}{user},
              $ini{$hash}{psw},
              $ini{$hash}{prompt},
              $ini{$hash}{port} );

    push(@data,@loop);

    }

    print Dumper(\@data);

    return @data;

} # end sub complex

my @list = devices();

sub clk_sync {

my $host   = shift;
my $user   = shift;
my $passwd = shift;
my $prompt = shift;
my $port   = shift;

my $ssh = Net::SSH::Perl->new( $host , 
                   port => $port ,
                   protocol => 2 ,
                   interactive => 1 ,
                   batchMode => 1 ,
                   RhostsAuthentication => 1,
                   debug => 0 );

$ssh->login($user,$passwd);

my $cmd = "pwd";
#my $cmd = "sudo ntpdate ntp.ubuntu.com";

my($stdout, $stderr, $exit) = $ssh->cmd($cmd);

$ssh->cmd("exit");

return $stdout;

}

__END__

$VAR1 = {
          'Device 1' => {
                          'IP' => 'IP',
                          'user' => 'USER',
                          'psw' => 'PASSWORD',
                          'prompt' => '/[\\$%#>] $/',
                          'port' => 'PORT',
                          'hostname' => 'ubuntu'
                        },
          'Device 2' => {
                          'IP' => 'IP',
                          'user' => 'USER',
                          'psw' => 'PASSWORD',
                          'prompt' => '/[\\$%#>] $/',
                          'port' => 'PORT',
                          'hostname' => 'ubuntu'
                        }
        };
$VAR1 = [
          '/home/Operating
',
          '/home/Operating
'
        ];

$VAR1 = [
          undef,
          undef
        ];
[download]

So my question is, does anybody know any module capable of executing administrator commands on external operating systems?

Final Update with answer

I was not able to find a solution with Net::Telnet this does not mean that is not possible, but for my current level of knowledge was not possible. So I moved to Net::OpenSSH. Where I found a really nice and simple example p5-Net-OpenSSH / sample / expect.pl.

Final Update 2 with better answer and output sample:

So with no further delays this is the code for multiple devices. For sample purposes the code runs on my localhost.

#!/usr/bin/perl
use Expect;
use strict;
use warnings;
use Data::Dumper;
use Net::OpenSSH;
use Config::IniFiles;
use Fcntl qw(:flock);

$| = 1;

# To see the complete debuging process
#$Net::OpenSSH::debug = ~0;
my $path = 'conf.ini';
my $timeout = 20;
my $debug = 0;

sub devices {
    
    open my $fh , '<' , "".$path."" 
    or die "Could not open file: ".$path." - $!\n";

    flock($fh, LOCK_SH)
    or die "Could not lock '".$fh."' - $!\n";

    tie my %ini, 'Config::IniFiles', ( -file => "".$path."" )
    or die "Error: IniFiles->new: @Config::IniFiles::errors";

    close ($fh)
    or die "Could not close '".$fh."' - $!\n";

    my @keys = keys (%ini);
    my %data = ();
    my %final = ();

    foreach my $hash (@keys) {

    %data = clk_sync( $ini{$hash}{host},
              $ini{$hash}{user},
              $ini{$hash}{psw},
              $ini{$hash}{port},
              $hash );
    
    @final{keys %data} = values %data;
    }

    return %final;

} # end sub complex

my %results = devices();

print Dumper(\%results);

sub clk_sync {

# alternative of shift my ($host,$user,$passwd,$port,$device) = (@_);

    my $host   = shift;
    my $user   = shift;
    my $passwd = shift;
    my $port   = shift;
    my $device = shift;

    my %opts = ( passwd => $passwd, 
         port   => $port, 
         user   => $user );

    my $ssh = Net::OpenSSH->new( $host, %opts );
    $ssh->error and
    die "Couldn't establish SSH connection: ". $ssh->error;

    my ($stop, $pid_stop) = $ssh->open2pty("sudo -k; sudo service ntp 
+stop")
    or die "open2pty failed: " . $ssh->error . "\n";

    my $expect_stop = Expect->init($stop);
    $expect_stop->raw_pty(1);
    $debug and $expect_stop->log_user(1);

    $debug and print "waiting for password prompt\n";
    $expect_stop->expect($timeout, ':')
    or die "expect failed\n";
    $debug and  print "prompt seen\n";

    $expect_stop->send("$passwd\n");
    $debug and print "password sent\n";

    $expect_stop->expect($timeout, "\n")
    or die "bad password\n";
    $debug and print "password ok\n";

    my $result_stop = ();

    while(<$stop>) {
    # removing trailing characters such as \n
    chomp $_;
    # removing blank space before and after the string
    $_ =~ s/^\s+|\s+$//g;
    print "\$_: ".$_."\n";
    $result_stop = $_;
    }

    my ($ntpd, $pid_ntpd) = $ssh->open2pty("sudo -k; sudo ntpd -gq")
    or die "open2pty failed: " . $ssh->error . "\n";

    my $expect_ntpd = Expect->init($ntpd);
    $expect_ntpd->raw_pty(1);
    $debug and $expect_ntpd->log_user(1);

    $debug and print "waiting for password prompt\n";
    $expect_ntpd->expect($timeout, ':')
    or die "expect failed\n";
    $debug and  print "prompt seen\n";

    $expect_ntpd->send("$passwd\n");
    $debug and print "password sent\n";

    $expect_ntpd->expect($timeout, "\n")
    or die "bad password\n";
    $debug and print "password ok\n";

    my $result_ntpd = ();

    while(<$ntpd>) {
    # removing trailing characters such as \n
    chomp $_;
    # removing blank space before and after the string
    $_ =~ s/^\s+|\s+$//g;
    print "\$_: ".$_."\n";
    $result_ntpd = $_;
    }

    my ($start, $pid_start) = $ssh->open2pty("sudo -k; sudo service nt
+p start")
    or die "open2pty failed: " . $ssh->error . "\n";

    my $expect_start = Expect->init($start);
    $expect_start->raw_pty(1);
    $debug and $expect_start->log_user(1);

    $debug and print "waiting for password prompt\n";
    $expect_start->expect($timeout, ':')
    or die "expect failed\n";
    $debug and  print "prompt seen\n";

    $expect_start->send("$passwd\n");
    $debug and print "password sent\n";

    $expect_start->expect($timeout, "\n")
    or die "bad password\n";
    $debug and print "password ok\n";

    my $result_start = ();

    while(<$start>) {
    # removing trailing characters such as \n
    chomp $_;
    # removing blank space before and after the string
    $_ =~ s/^\s+|\s+$//g;
    print "\$_: ".$_."\n";
    $result_start = $_;
    }

    my @send = ();
    
    push (@send,$result_stop,$result_ntpd,$result_start);

    chomp(@send);

    my %hash_out = ( $device => [ $result_stop , $result_ntpd , $resul
+t_start ] );

    #print Dumper(\%hash_out);

    return %hash_out;
}

__END__

$_: * Stopping NTP server ntpd                                        
+   [ OK ] 
$_: ntpd: time slew +0.000728s
$_: * Starting NTP server ntpd                                        
+   [ OK ] 
$_: * Stopping NTP server ntpd                                        
+   [ OK ] 
$_: ntpd: time slew -0.000542s
$_: * Starting NTP server ntpd                                        
+   [ OK ] 
$VAR1 = {
          'DEVICE 2' => [
                          '* Stopping NTP server ntpd                 
+   [ OK ]',
                          'ntpd: time slew -0.000542s',
                          '* Starting NTP server ntpd                 
+   [ OK ]'
                        ],
          'DEVICE 1' => [
                          '* Stopping NTP server ntpd                 
+   [ OK ]',
                          'ntpd: time slew +0.000728s',
                          '* Starting NTP server ntpd                 
+   [ OK ]'
                        ]
        };
[download]

Configuration file (conf.ini)

[DEVICE 1]
host      = 127.0.0.1
user      = username
psw       = password
port      = 22

[DEVICE 2]  
host     = 127.0.0.1
user     = username
psw      = password
port     = 22
[download]

Seeking for Perl wisdom...on the process of learning...not there...yet!

Comment on Best module to execute administrator commands on external operating systems (Linux) Select or Download Code

Replies are listed 'Best First'.
Re: Best module to execute administrator commands on external operating systems (Linux) by McA (Priest) on Aug 15, 2014 at 06:20 UTC
Hi, have a look at Rex. Probably it fits your usage pattern. Regards McA	[reply]
Re^2: Best module to execute administrator commands on external operating systems (Linux) by thanos1983 (Parson) on Aug 15, 2014 at 08:16 UTC
Hello McA, Thank you for sharing I did not know about that module. I will create a test code and I will let you know if it is working. Thank you for your time and effort reading and replying to my question. Seeking for Perl wisdom...on the process of learning...not there...yet!	[reply] [d/l] [select]
Re: Best module to execute administrator commands on external operating systems (Linux) by jellisii2 (Hermit) on Aug 15, 2014 at 11:22 UTC
If you have sufficient permissions on the remote system to create accounts, can you create an account that already has proper permissions to do what you need to? I have in the past had to do similar things, and eventually gave up due to time constraints. I had the systems admin create an account that could do all that I needed to do and authenticated with a private key exclusively. It ended up being a much cleaner solution than trying to pass sudo onto a remote command line or using Expect to try and get stuff done.	[reply]
Re^2: Best module to execute administrator commands on external operating systems (Linux) by thanos1983 (Parson) on Aug 15, 2014 at 14:09 UTC
Hello jellisii2, That is true I was thinking the same solution but my problem is that I want to parse the output of the return and get the offset that I will use it on my calculations. I was even thinking create a local script which will do the process and send me the output, but by doing so in all machines I will not be able to make an automated system. I was thinking of putting this script to execute before and after my schedule experimentation so I can add or subtract the time stamps on my results. Well I manage to move a bit further with my problem. I will update now my findings so far. I hope that someone will have an answer to my problem. Seeking for Perl wisdom...on the process of learning...not there...yet!	[reply] [d/l] [select]
Re: Best module to execute administrator commands on external operating systems (Linux) by mr_mischief (Monsignor) on Aug 15, 2014 at 14:40 UTC
I realize you may want to reinvent this particular wheel. However, have you looked at Ansible, mssh, or Fabric? For that matter the sort of thing you're wanting to do is a tiny subset of functionality for Chef, Salt, Puppet, CFEngine, or cdist. I would hate to think you're going to home brew an alternative to those.	[reply]
Re^2: Best module to execute administrator commands on external operating systems (Linux) by thanos1983 (Parson) on Aug 15, 2014 at 18:01 UTC
Hello mr_mischief, Wow I had no clue about all this interesting solutions that you proposed. Yo be honest I want to do something so simple but it has ended up being so difficult. I think that I am on the correct way though I thought about it and maybe there is a solution to my problem. Well I might need some time to implement it, because I am not an expert in programming but I will get there I think. :D Thank you for your time and effort to assist me. Seeking for Perl wisdom...on the process of learning...not there...yet!	[reply] [d/l] [select]