Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

scp progress logging

by egilchri (Novice)
on Mar 15, 2006 at 22:55 UTC ( [id://536982]=perlquestion: print w/replies, xml ) Need Help??

egilchri has asked for the wisdom of the Perl Monks concerning the following question:

I'm invoking scp from Perl, and I'm happy with using it this way. However, I'd really like to stream the progress information to a file, so I can monitor that file to check on scp progress. But from Googling around, I've become convinced that the nifty progress info doesn't get created when I invoke scp from a script, probably because scp has a call to isatty. Any quick and dirty way to tap into the progress info? I tried %scp -r html egilchri@egilchri-pc: >> /tmp/log  2>&1 but log ends up being empty. I'd really like to see all though percentages whizzing by.

Replies are listed 'Best First'.
Re: scp progress logging
by traveler (Parson) on Mar 15, 2006 at 23:08 UTC
    Since you seem to be on *nix, can you open /dev/tty as the standard output/input? Then you wouldn't have to watch the file, you could watch the screen, if that is good enough.
      Let me elaborate a bit on what I'm doing. A web application kicks off a backend process. It's this backend process that does all the scp activity. For instance, scp of a huge tar file. What I'd like to do is have my web application be able to ask the backend, "So, how's it coming along?" I figure if scp was dumping its progress into a file, I could display that log file back to the user of the web application.
Re: scp progress logging
by zentara (Archbishop) on Mar 16, 2006 at 13:48 UTC
    I've become convinced that the nifty progress info doesn't get created when I invoke scp from a script, probably because scp has a call to isatty. Any quick and dirty way to tap into the progress info?

    I've seen those types of programs when trying to make progressbars for various apps run from Tk. One of the things I've had success with, is running the program thru IPC::Open3, and watching the output on the filehandles. A sneaky trick is NOT to open the program itself with IPC::Open3, but open a "/bin/bash". Then print to the STDIN FH , your scp command (don't forget a newline :-) ). Then you should be able to read the STDOUT and STDERR that would be appearing in the bash shell. Regex that output for your progress data.

    There are a few variations on this trick, like try a piped-open instead of IPC::Open3. Maybe

    open(FH, "/bin/bash -c $scp_string 2>&1 |");
    It should provide you with "hours of keybanging fun" while you find the right combination. :-)

    I'm not really a human, but I play one on earth. flash japh
      Neither of those two methods worked, so I started playing around with IO::Pty. I reason that this is all about tricking scp into thinking it's being invoked from a tty, and I gather that's what a pseudo tty is. The following script executes successfully, but I don't know how to get access to the progress info, if it is indeed going anywhere. Plus, I really don't need/want the thing to exec, since I'd rather that my program waited for the command to return, before proceeding.
      #!/usr/bin/perl use IO::Pty; my $cmd = "scp"; my @args = ("-v", "big.tar", "egilchri\@other_machine:"); &do_cmd ($cmd, @args); sub do_cmd { my ($cmd,@args) = @_; my $pty = IO::Pty->new or die "can't make Pty: $!"; defined (my $child = fork) or die "Can't fork: $!"; return $pty if $child; POSIX::setsid(); my $tty = $pty->slave; close $pty; STDIN->fdopen($tty,"<") or die "STDIN: $!"; STDOUT->fdopen($tty,">") or die "STDOUT: $!"; STDERR->fdopen(\*STDOUT,">") or die "STDERR: $!"; close $tty; $| = 1; exec $cmd,@args; die "Couldn't exec: $!"; }
        Yeah, it sounds like you are pretty close. Here is a snippet (from unknown origin) that may help get the output from the pty. Maybe you can figure out how to combine your scripts?
        #!/usr/bin/perl -w # Description: Fool a process into # thinking that STDOUT is a terminal, when in fact # it may be a file or a pipe. This can be useful # with programs like ps and w on linux... which # will trunc their output to the width of the # terminal, and, if they cannot detect the terminal # width, use a default 80 columns. Wouldn't it be # nice to say "ps -aux | grep etcshadow", and get # output that looks like when you just say "ps # -aux"? Well, that's the idea. use warnings; use strict; use IO::Pty; die "usage: ptyexec command [args]\n" unless @ARGV; my $pty = IO::Pty->new; my $slave = $pty->slave; open TTY,"/dev/tty" or die "not connected to a terminal\n"; $pty->clone_winsize_from(\*TTY); close TTY; my $pid = fork(); die "bad fork: $!\n" unless defined $pid; if (!$pid) { # $slave->close(); open STDOUT,">&=".$pty->fileno() or die $!; exec @ARGV; }else{ $pty->close(); while (defined (my $line = <$slave>)) { print $line; } } #cleanup pty for next run $pty->close();

        I'm not really a human, but I play one on earth. flash japh
Re: scp progress logging
by zer (Deacon) on Mar 16, 2006 at 03:26 UTC
    perhaps if you piped the process... get the output directly?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://536982]
Approved by pileofrogs
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others studying the Monastery: (6)
As of 2024-04-18 19:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found