I'm trying to split a large file into smaller chunks, and then use threads to upload the chunks to an FTP server. This is to speed up the transfer with multiple connections uploading a piece of the file. Here's what I've tried so far but its not working when using threads, the program seems to hang or block, and the file on the FTP server is 0 size. Any help would be appreciated. 
#!/usr/bin/perl -w
use strict;
use warnings;
use Net::FTP;
use Config;
$Config{useithreads} or die('Recompile Perl with threads to run this p
+rogram.');
use threads;

my $file = 'c:\sdwork\dla495.txt';

my $ftp = Net::FTP->new("ftpserver") or die "can't get ftp object\n";
$ftp->login("fakeusr", "secret") or die "can't login to ftp server\n";
$ftp->binary();
$ftp->cwd('/home/fakeusr/tmp');

open (FH, "<$file") or die "Could not open source file. $!";
binmode(FH);

my $i = 0;

my $start_byte = 0;
my @THRD_LIST;
my $thrd;

while (1) {
   my $chunk;
   print "process part $i\n";
   
   $i ++;
   if (!eof(FH)) {
      my $bytes_read = read(FH, $chunk, 10000);  #bytes
      print "bytes read: $bytes_read\n";
      
      #test with threads
      $thrd = threads->create (\&transfer_chunks, $start_byte, $chunk)
+ or die "Failed to start the thread: $@\n";
      push (@THRD_LIST, $thrd);
      
      #test without threads
      #transfer_chunks($start_byte, $chunk);
      
      $start_byte += $bytes_read + 1;
      print "start byte: $start_byte\n";
   } 

   last if eof(FH);
}

# start the threads
for $thrd (@THRD_LIST) {
  $thrd->join();
}

$ftp->quit;

sub transfer_chunks {
   my ($start_byte, $chunk) = @_;
   # convert perl string into a filehandle
   open(TEST, '<', \$chunk);
   
   $ftp->restart($start_byte);
   $ftp->put(*TEST, 'testfile.txt');
   ##$ftp->append(*TEST, 'testfile.txt');
   close(TEST);
}

[download]

In reply to Split large file and upload to ftp server with multiple threads by mdc76

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.