odd escape chars in strings obtained by SSH

danmcb has asked for the wisdom of the Perl Monks concerning the following question:

Here's a little secure shell puzzler for you monks.

I'm using Net::SSH::Expect to connect to a remote machine, run commands and grab their output, for testing purposes. I wrapped the whole thing in an object, which has an "exec" method which is like that of N:S:E, except that it will remove the prompt, return either an array ref or a string depending on context - and chomp newlines.

Ah, new lines. Chomp just doesn't seem to work here. I check the value of $\, and then start looking at the output using unpack to hexdump. There is some odd stuff here - I guess this is to do with ssh operation, butI can't figure it, or find anything about it online ...

Here's some code and what I get. My main module:

use TestRobot;

my $tr = TestRobot->new( 'config.yml' );
$tr->open;
my $s = $tr->exec('ls -l');
print $s;
#print "\n***\n";
#print unpack('H*', $/);
#print "\n***\n";
#print unpack('H*', 'abc');
#print "\n***\n";
#print unpack('H*', $s);
#print "\n***\n";

$tr->close;
[download]

and here is the exec method in TestRobot:

sub exec {
  my ( $self, $cmd )  = @_;
  croak 'cannot exec unless state is open' unless $self->is_open;
  croak 'need a command to execute' unless defined $cmd && $cmd ne '';
  my $r = $self->{ssh}->exec( $cmd );

  #my $q = chomp $r; print "$q gone\n";

  # remove trailing prompt and surrounding whitespace from output
  my $u = $self->{config}{test_server}{username};
  $r =~ s/\[$u@.*$//g;
  if ( wantarray ) {
    my @a = split "\n", $r;
 #   chomp( @a );
    return \@a;
  } else {
    return $r;
  }
}
[download]

(the chomp removes zero chars, by the way). Finally, here is what I get when I uncomment those prints, and redirect the output to a file. Not that the odd ^[[ sequences don't show up when you run it in a terminal!

[00mtotal 24
drwxrwxr-x  2 testrobot testrobot 4096 Oct 11 12:33 [00;34mbin[00m
drwxrwxr-x  3 testrobot testrobot 4096 Oct 11 12:33 [00;34mtesting[0
+0m
drwxrwxr-x  3 testrobot testrobot 4096 Oct 11 11:53 [00;34mwork[00m
[m]0;testrobot@svnsepia:~
***
0a
***
616263
***
1b5b30306d746f74616c2032340a64727778727778722d782020322074657374726f62
+6f742074657374726f626f742034303936204f63742031312031323a3333201b5b303
+03b33346d62696e1b5b30306d0a64727778727778722d782020332074657374726f62
+6f742074657374726f626f742034303936204f63742031312031323a3333201b5b303
+03b33346d74657374696e671b5b30306d0a64727778727778722d7820203320746573
+74726f626f742074657374726f626f742034303936204f63742031312031313a35332
+01b5b30303b33346d776f726b1b5b30306d0a1b5b6d1b5d303b74657374726f626f74
+4073766e73657069613a7e07
***
[download]

What I would like to do is filter this down do just raw ASCII, because otherwise it is going to make running tests on the output very messy. But how? options to ssh? (I do call "stty raw -echo" after connecting, as suggested in the docs.

As always, all help is gratefully received.

Comment on odd escape chars in strings obtained by SSH Select or Download Code

Replies are listed 'Best First'.
Re: odd escape chars in strings obtained by SSH by Fletch (Bishop) on Oct 11, 2007 at 14:41 UTC
Those are terminal colour codes from some version of `ls`. Try setting the terminal type on the other end to `DUMB` or run `ls --color=never` to tell it not to output color escapes.	[reply]
Re^2: odd escape chars in strings obtained by SSH by dah (Sexton) on Oct 11, 2007 at 16:51 UTC
Or ask ssh not to allocate a tty (terminal).	[reply]
Re^2: odd escape chars in strings obtained by SSH by lykich (Initiate) on Jun 25, 2021 at 18:27 UTC
It is true, have strings from expect and can not clear from different symbols. It working $ssh->send("ls --color=never -A1"); Thanks.	[reply]
Re: odd escape chars in strings obtained by SSH by graff (Chancellor) on Oct 11, 2007 at 17:50 UTC
As mentioned above, your "ls" is trying to be clever, and you just need to disable that -- but that's not the only source of the escape characters you're getting. The .bashrc (or maybe .profile ?) for the user account is setting the command prompt string to something "fancy" (maybe bold font?) as well. If you can edit the .bashrc for this user account on the target machine, you can control both the "ls" behavior and the command prompt string: `alias ls='/bin/ls --color=never' PS1='$ '` [download] (or maybe you want the prompt string to be something particular other than '$ ' -- it's up to you). If you can't edit that .bashrc file, just issue those two command lines first thing when you connect, and then only pay attention to the lines that are returned after that point. `if ( wantarray ) { my @a = split "\n", $r; # chomp( @a ); return \@a; } else { return $r; }` [download] (the chomp removes zero chars, by the way) Well, in the case where "wantarray" returns true, of course chomp does nothing, because `split /\n/` removes all the line-feed characters.	[reply] [d/l] [select]
Re^2: odd escape chars in strings obtained by SSH by Fletch (Bishop) on Oct 11, 2007 at 18:44 UTC
And it's a good practice when running something like this (i.e. via something Expect-ish) to "sanitize" the remote environment. Things such as setting your prompt to a (simple) static value and sending the `PATH` you're expecting to be be in effect makes looking for prompts and debugging so much easier and more dependable. Save the angry fruit salad with 4 different typefaces for interactive use.	[reply]
Re^2: odd escape chars in strings obtained by SSH by danmcb (Monk) on Oct 12, 2007 at 08:24 UTC
Thanks for the suggestions. I have indeed aliased ls and set a simple prompt as suggested. I still do have, however, one escape sequence appended to the end of output (even a simple command like 'pwd'). It looks like this: `ESC]0;username@hostname:~^G` ESC is hex 1b, and ^G is the bell char (07). I'm still trying to figure if this is coming from SSH, the term, or from Expect. I'm going to hack it out for now ...	[reply] [d/l]
Re^3: odd escape chars in strings obtained by SSH by toma (Vicar) on Oct 12, 2007 at 16:17 UTC
This looks like it may be the shell prompt. Try setting the shell environment variable `PS1=''` on the remote machine. It should work perfectly the first time! - toma	[reply] [d/l]
Re^2: odd escape chars in strings obtained by SSH by danmcb (Monk) on Oct 12, 2007 at 07:46 UTC
hah! good point about the chomp. The one on the array is indeed redundant. There is another one there, one the scalr before it gets split, that also removes none.	[reply]