hoyt has asked for the wisdom of the Perl Monks concerning the following question:

I have a perl script that basically moves directories from one server to another. In it is a line using rsync that does this: 
my $rsync_command = 'rsync -vrlte "ssh -p '.$pref_values->{'SSH_PORT'}
+.' -l '.$pref_values->{'SSH_USER'}.' -o StrictHostKeyChecking=no" --p
+rogress';
my $cmd = `$rsync_command "$full_dir_name"/* $pref_values->{'SSH_SERVE
+R'}:$home_dir_name`;

[download]
I'm moving some large directories over the internet, so oftentimes this command takes 20 or 30 minutes. I'd like to somehow grab the output of the rsync command and update a database with the progress. Right now, I'm only adding a line that says started, and a line that says done. On occasion the script crashes, or the connection is interrupted, etc. So when I see Started, I have to log into the target server and check the directory size to see if it's still growing. The output of this command when I run it manually looks like this: 
File1
          1,282 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=23/24)
File2
          1,064 100%  346.35kB/s    0:00:00 (xfr#2, to-chk=22/24)
File3
      3,506,176  10%  121.86kB/s    0:04:17  ^C

[download]
Is there anyway to poll this command and get the output while it's running? I know I can grab it at the end and insert a summary, but I want to be able to look in my database and see that I'm 10% of the way through File3, it's going at 121 kB/s and there's 4:17 left on that file. Thanks!

Replies are listed 'Best First'.
Re: Retrieving the Progress of a Command
by kroach (Pilgrim) on Sep 13, 2016 at 21:14 UTC
    You can use open to connect to a command with a pipe and get its output. Commandline progress bars work by printing the carriage return character ("\r"), to set the cursor back to the start of line and overwriting the text. To capture each update in progress, you can set the input record separator to "\r". 
    use feature qw(say);

my $rsync_command = 'rsync ... --progress';
open my $RSYNC, '-|', $rsync_command or die "Failed to start rsync: $!
+";
local $/ = "\r";
while (<$RSYNC>) {
    if (/\A\s/) { # progress lines start with whitespace
        say 'Progress: ', $_;
    }
    else {
        print 'File: ', $_; # file lines have newlines at the end
    }
}
close $RSYNC;

    [download]
    The code to differentiate between a filename and a progress line could be upgraded a bit but this is the general approach I would use to monitor progress bars.
[reply]
[d/l]
      That makes sense. I'll try to adapt that and see how I can make it work here. Thanks!
[reply]

Back to Seekers of Perl Wisdom