PerlRob has asked for the wisdom of the Perl Monks concerning the following question:

Hey Monks,

I'm attempting to track the progress of a file upload to an FTP server.

I'm currently opening the file, reading 2,000,000 bytes into a scalar, writing those bytes to a local output file, then using the append() method of the Net::FTP module to append the output file to the one on the server.

This works, but is there any way to eliminate the middle step and just write directly to the file on the FTP server?

Here's my code: 
open(INFILE, "<", $fileToRead);
binmode INFILE;

my $ftp = Net::FTP->new("ftp.example.com");
$ftp->login('username', 'password');
$ftp->binary();

while (read INFILE, my $data, 2000000)
{
    open(OUTFILE, ">", $fileToWrite);
    binmode OUTFILE;
    print OUTFILE $data;
    close(OUTFILE);
    $ftp->append($fileToWrite);
    # Here's where I will print the progress...
}

$ftp->quit();

[download]

Replies are listed 'Best First'.
Re: Track file upload progress
by Tanktalus (Canon) on Jun 20, 2008 at 23:37 UTC

    The docs don't seem to state it, but you can actually pass in a filehandle instead of a local filename. However, it appears you then *must* specify the remote file name.

    my $ftp = Net::FTP->new("ftp.example.com");
$ftp->login('username', 'password');
$ftp->binary();

while (read INFILE, my $data, 2000000)
{
    open my $datafh, '<', \$data;
    $ftp->append($datafh, $fileToWrite);
    # Here's where I will print the progress...
}

$ftp->quit();

    [download]
    Or something like that. (UNTESTED) Good luck :-)

[reply]
[d/l]
Re: Track file upload progress
by ikegami (Patriarch) on Jun 21, 2008 at 00:30 UTC
    Totally untested. 
    use strict;
use warnings;

BEGIN {
   package Handle::ReadProgress;

   use Symbol      qw( gensym );
   use Tie::Handle qw( );

   our @ISA = 'Tie::Handle';

   sub wrap {
      my $outer_fh = gensym();
      tie *$outer_fh, @_;
      return $outer_fh;
   }

   sub new {
      my ($class, $inner_fh, $callback) = @_;
      
      return bless({
         inner_fh => $inner_fh,
         callback => $callback,
         progress => 0,
      }, $class);
   }

   sub READ {
      my ($self) = @_;

      my $bytes_read = read($self->{inner_fh}, $_[1], $_[2], $_[3]);

      if (defined($bytes_read)) {
         my $progress = ( $self->{progress} += $bytes_read );
         $self->{callback}->($outer_fh, $progress);
      }

      return $bytes_read;
   }

   sub READLINE {
      my ($self) = @_;

      my $line = readline($self->{inner_fh});

      if (defined($line)) {
         my $progress = ( $self->{progress} += lenght($line) );
         $self->{callback}->($outer_fh, $progress);
      }

      return $line;
   }

   sub CLOSE   { close   $_[0]{inner_fh}        }
   sub BINMODE { binmode $_[0]{inner_fh}, $_[1] }
   sub EOF     { eof     $_[0]{inner_fh}        }
}


use IO::Handle qw( );

{
   open(my $fh, "<", $fileToRead)
      or die;

   binmode $fh;

   my $ftp = Net::FTP->new("ftp.example.com");
   $ftp->login('username', 'password');
   $ftp->binary();

   my $size = -s $fh;

   $fh = Handle::ReadProgress->wrap($fh, sub {
      my ($fh, $progress) = @_;
      printf("\r%d%%", int($progress / $size) * 100);
      STDOUT->flush();
   });

   print('0%');
   STDOUT->flush();

   $ftp->store($fh, $fileToWrite);

   print("\rdone.\n");
}

    [download]

    Update: Fixed missing TIEHANDLE (by adding new).

[reply]
[d/l]
[select]
Re: Track file upload progress
by pc88mxer (Vicar) on Jun 20, 2008 at 23:44 UTC
    You could do this which eliminates the need for executing separate append operations for each read:
    1. Open an ftp connection to the remote server
    2. Initiate an append operation and open the data connection to the ftp server.
    3. Start your read loop and write what you read to the data connection. Also perform whatver tracking of the data here.
    4. When done reading, close the data connection. (Crossing your fingers before closing the connection would be advisable here.)
    Basically you would be customizing the _store_cmd() routine in Net::FTP with your own process. Sub-classing Net::FTP actually might not be a half-bad way to do this.

    Update: Adding a call-back to Net::FTP for progress tracking shouldn't be too hard. It already prints out hash marks using this code in _store_cmd(): 

        if ($hashh) {
      $count += $len;
      print $hashh "#" x (int($count / $hashb));
      $count %= $hashb;
    }

    [download]
    Just replace that with your own code.
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom