RaduH has asked for the wisdom of the Perl Monks concerning the following question:

Hi there,

At some point in my Perl script I am downloading a huge file (a tgz 500MB to 1GB in size) using a form of secure copy (I have used Net:SSH2, SFTP, right now I am doing a scp inside a system call, not sure what the final variant will be). Because this happens on a remote computer (on my computer my script invokes a script on a remote computer and that script is performing the download) and because this does take a while, I would like to be able to send back to my computer progress reports (say every 10% of the download).

Using scp there's real-time feedback on the screen and thus I can follow the download percent by percent but this happens on the remote computer and not where I can see it. Any ideas how I could solve this problem efficiently? I can spin in a loop and check the size of the file against the size of the entire file (which I am sure should be easy to figure) but this seems like a lot of spinning, especially for a 1GB tarball. Any other options out there and I don't know about? It doesn't have to be scp, any king od secure copy will do. In case it makes a difference, this does not run in a webpage, it's simple, plain Perl, no other strings attached

Thanks!

Replies are listed 'Best First'.
Re: Long download progress report
by Corion (Patriarch) on Nov 16, 2007 at 17:06 UTC
[reply]
Re: Long download progress report
by gamache (Friar) on Nov 16, 2007 at 17:19 UTC
    Here is how I would do it:
    • Fork a child to run the SCP (or other file transfer) process and exit.
    • From the parent, loop on the following:
      • Is the child done? If so, set progress = 100% and exit loop.
      • Set progress = current_filesize / full_filesize.
      • Sleep for N seconds.
[reply]
Re: Long download progress report
by BrowserUk (Patriarch) on Nov 16, 2007 at 19:24 UTC

    Something like this (totally untested) code might work for you assuming that the progress reporting on teh remote computer is output to either stdout or stderr: 

    use threads;

my $fh;
my $pid;
async {
    $pid = open $fh, 
        "scp dvader@empire.gov:file1.txt dvader@deathstar.com:somedir 
+2>&1 |" 
        or die $!;
}->detach;

while( <$fh> ) {
    print;
}
close $fh;
kill 'quit', $pid;

    [download]

    This runs the command asynchronously through a forking open, redirects all the output to a pipe and then displays the output on the screen.

    Note: I've never used scp, so the syntax and indeed the whole idea is just speculation.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

Back to Seekers of Perl Wisdom