cmv has asked for the wisdom of the Perl Monks concerning the following question:

Wise Monks-

I have the need to tail a bunch of files on remote machines, all at the same time, and print the intermixed lines as they come in (ala xtail). My hands are tied, in that I can't use any perl modules that are not already installed on the existing machine.

Luckily, the current machine has IO::Select, so my current plan is to open file descriptors that are reading the output pipe from the ssh commands, and use select to tell me when there is input on any of them.

I can get this select strategy to work with commands run on the local machine, but anytime I try to ssh that exact same command to a remote machine, I get the following error:

tcsetattr: I/O error

and the ssh connection closes. Attached is my current test script. Any help is very much appreciated!

Thanks

-Craig 

use strict;
use warnings;

use IO::Select;

my $cmd1 = "cat /etc/passwd";

my $sshcmd = "/usr/bin/ssh -t -i \$HOME/.ssh/mypubkey -o 'StrictHostKe
+yChecking no' user\@host $cmd1";

# Prove command works...
my $out = `$sshcmd`;
print STDERR "OUT:\n$out";

#open(IFILE, "$cmd1 |") || die "Bad open";            # This will work
open(IFILE, "$sshcmd |") || die "Bad open";           # This does not
my $select = IO::Select->new();
$select->add(\*IFILE);

my @ready;
while ( @ready = $select->can_read(5) ) {
    foreach my $fh (@ready) {
        my $line = <$fh>;
        STDERR "line=$line";
        if(eof($fh)) {
            $select->remove($fh);
            close($fh) || warn "Close problem on fh: $fh";
        }
    }
}

[download]

Replies are listed 'Best First'.
Re: IO::Select & ssh problem
by ikegami (Patriarch) on Sep 17, 2009 at 17:44 UTC

    Don't know about your problem. What's the error? (include $! in your error message).

    The following isn't very safe: 

    my $cmd1 = "cat /etc/passwd";
my $sshcmd = "/usr/bin/ssh -t -i \$HOME/.ssh/mypubkey -o 'StrictHostKe
+yChecking no' user\@host $cmd1";
open(IFILE, "$sshcmd |") || die "Bad open";

    [download]
    my @cmd = (cat => (
    '/etc/passwd',
));

my @sshcmd = ('/usr/bin/ssh' => (
    -t => (),
    -i => "$HOME/.ssh/mypubkey",
    -o => 'StrictHostKeyChecking no',
    'user@host',
    @cmd
));

open(my $fr_ssh, '-|', @sshcmd)
    or die("Can't fork and pipe: $!\n");

    [download]
[reply]
[d/l]
[select]
      ikegami++ Thanks for your response.

      The output from $! is "Illegal seek"

      Am I doing something I'm not supposed to be doing? Can anyone else get this to work?

      Thanks

      -Craig

      Update:

      Also, I'm also seeing an error when I run the following code: 
      ...
my @cmd = (cat => ( '/etc/passwd' ));

open(IFILE, '-|', @cmd) || die "Bad ssh open";
...

      [download]
      The error is: Can't use an undefined value as filehandle reference at ...

      The perl version being run is: v5.6.1 built for sun4-solaris-64int

      However, this code does run fine on perl 5.8

      Update 2:

      However, this seems to work: 
      ...
my @cmd = (cat => ( '/etc/passwd' ));
open (IFILE, '-|') || exec @cmd;
...

      [download]
[reply]
[d/l]
[select]
Some Progress, but still NEED HELP!
by cmv (Chaplain) on Sep 21, 2009 at 14:44 UTC
    Folks-

    I'm suspecting that this is a problem having to do with both the perl version, and the ssh versions (near end and far end).

    The attached script seems to be working for perl5.8, when speaking between a host running "OpenSSH_5.1p1, OpenSSL 0.9.7l 28 Sep 2006", and a far end running "OpenSSH_5.2p1-lucent-1".

    However, it still fails when trying to run under perl 5.6.1, when speaking between a host running "Sun_SSH_1.1.1, SSH protocols 1.5/2.0, OpenSSL 0x0090700f", and a far end running "Sun_SSH_1.1.1".

    Update: It seems as though it has to do with the ssh version on the target machine. I've verified that things are working from a Sun_SSH_1.1.1 to a OpenSSH_5.2p1 host.

    When I compare the -v outputs of the working versus non-working runs, the thing that is catching my attention is the "debug1: channel 0: write failed" message. Here is a brief excerpt of what I am seeing (in readmore's):

    Read more... (2 kB)

    I also notice in the not working case, where the bytes transferred statistics only show data being transferred on stderr. I'm currently trying to figure out how I can access this information. Any pointers are appreciated.

    Also, if there is an alternative way of solving this problem, please offer a suggestion. I may not be able to get this to work.

    Thanks

    -Craig 

    use strict;
use warnings;

use IO::Select;

my $cmd = '"cat /etc/passwd"';
my $login = 'usr@host';
my $pkey = "$ENV{HOME}/.ssh/privateKey";

my @shell = ( '/usr/bin/ksh' => (
    -c => $cmd,
));

my @sshcmd = ('/usr/bin/ssh' => (
    -t => (),
    -v => (),
    #-vv => (),
    #-vvv => (),
    -i => $pkey,
    -o => 'StrictHostKeyChecking no',
    $login,
    @shell,
));

open (IFILE, '-|') || exec @sshcmd;

my $select = IO::Select->new();
$select->add(\*IFILE);

my @ready;
while ( @ready = $select->can_read(5) ) {
    foreach my $fh (@ready) {
        my $line = <$fh>;
print STDERR "line=$line";
        if(eof($fh)) {
            $select->remove($fh);
            close($fh) || warn "Close problem on fh: $fh";
        }
    }
}

close(IFILE);

END{
    print STDERR "\$!: $!\n";
}

    [download]
[reply]
[d/l]
[select]
Solution: Don't know why it works...
by cmv (Chaplain) on Sep 21, 2009 at 18:42 UTC
    Folks-

    The following code seems to work between Sun_SSH_1.1.1 boxes. I don't know why it works, but it does. The secret ingredient was to add a system call that sent something to STDOUT before the system call (or exec) that does the ssh.

    I'm guessing a bug in sshd on the target host. Any thoughts?

    Thanks

    -Craig 

    use strict;
use warnings;

use IO::Select;

my $cmd = '"cat /etc/passwd"';

my $login = 'usr@host';
my $pkey = "$ENV{HOME}/.ssh/privateKey";

my @shell = ( '/usr/bin/ksh' => (
    -c => "$cmd",
));

my @sshcmd = ('/usr/bin/ssh' => (
    -t => (),
    -i => $pkey,
    -o => 'StrictHostKeyChecking no',
    $login,
    @shell,
));

if(my $child = open (IFILE, '-|')) {
    print STDERR "I'm the parent\n";
    print STDERR "Child pid=$child\n";
}else{
    select(STDIN); $|++;
    select(STDOUT); $|++;
    select(STDERR); $|++;
    print STDERR "I am the child\n";
    print "LINE 1...\n";
    print "LINE 2...\n";
    print "LINE 3...\n";
    print "LINE 4...\n";
    my $ret;
    $ret = system('echo hi');
    print STDERR "CHILD 1: ret=$ret\n";
    $ret = system(@sshcmd);
    print STDERR "CHILD 2: ret=$ret\n";
    print "LINE 5...\n";
    print "LINE 6...\n";
    print "LINE 7...\n";
    print "LINE 8...\n";
}

my $select = IO::Select->new();
$select->add(\*IFILE);

my @ready;
while ( @ready = $select->can_read(5) ) {
    foreach my $fh (@ready) {
        my $line = <$fh>;
print STDERR "line=$line";
        if(eof($fh)) {
            $select->remove($fh);
            close($fh) || warn "Close problem on fh: $fh";
        }
    }
}

close(IFILE);

    [download]
[reply]
[d/l]

      By the way, all your LINE lines are being printed to STDERR, since you made that handle the default using select.

      Easiest solution: replace 

          select(STDIN); $|++;
    select(STDOUT); $|++;
    select(STDERR); $|++;

      [download]
      with 
      use IO::Handle;

    STDOUT->autoflush();
    STDERR->autoflush();

      [download]
      or just 
      $| = 1;

      [download]

      It's useless to set autoflush on an input handle, and STDERR is already autoflushed.

[reply]
[d/l]
[select]
        ikegami++

        Thanks for pointing this out. I was casting around for anything that might help me, and it didn't occur to me what was actually happening.

        Although I was able to make the above script work, when I replaced the cat /etc/passwd line with a tail -f growingfile line, another problem occurred. The initial output from the tail comes out, but then nothing after that, even as I watch the file grow.

        I'm disappointed that I can't get this to work correctly, and can't figure out what is actually happening. If I can't make any progress today, I'll implement another method. Any suggestions would be appreciated.

        Thanks

        -Craig

[reply]
[d/l]
[select]
Re: IO::Select & ssh problem
by cmv (Chaplain) on Sep 22, 2009 at 20:18 UTC
    Folks-

    After lots of twiddling, and some CB help (thanks ssandv++ clinton++), here is the version that seems to work best. Still have some strange things going on (like stdout seems to be in raw mode, and sometimes I control-c doesn't work), but overall it seems to be doing what I expect.

    Use this when you need to run tail -f on a file that exists on a bunch of hosts, and want the output interleaved correctly.

    Please make comments and/or give me alternative ways of doing this (under perl5.6 with no other modules but IO::Select).

    Thanks

    -Craig 

    use strict;
use warnings;

use IO::Select;

$|++;

my $pkey = "$ENV{HOME}/.ssh/PrivateKey";

my @logins = ( qw ( usr@host1 usr@host2 usr@host3) );

my $cmd = "'tail -f /tmp/growingfile'";

my @shell = ( '/usr/bin/ksh' => ( -c => "$cmd" ));

my @FDs;

# Create ssh tails as FDs on child processes...
foreach my $login (@logins) {
print STDERR "login=$login\n";
    my @sshcmd = ('/usr/bin/ssh' => (
        -t => (),
        #-v => (),
        -i => $pkey,
        -o => 'StrictHostKeyChecking no',
        $login,
        @shell,
    ));

    my $fd;
    if(my $child = open ($fd, '-|')) {
        print STDERR "I'm the parent\n";
        print STDERR "PARENT: \$!: $!...\n";
        print STDERR "Child pid=$child\n";
    }else{
        $|++;
        print STDERR "I am the child\n";
        exec @sshcmd;
    }
    push(@FDs, \*$fd);
}

my $sel = new IO::Select(@FDs);

while(my @ready = $sel->can_read) {
    foreach my $fh (@ready) {
        if(eof($fh)) {
            $sel->remove($fh);
            close($fh);
            next;
        }
        my $line = <$fh>;
        print "$fh: $line";
    }
}

    [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom