Your node got me wondering at what point forking more workers fails to improve performance. This is admittedly very environment-specific, but I found it interesting nonetheless. I put together the following simple script as a test:

Environment:

use strict; use warnings; use Net::FTP; my $kids; my @kids = (); my @fileList = glob("*.pdf"); open(OUT, ">ftp_test.out") or die; for $kids (1..30){ my @tempList = @fileList; my $start_time = time; while(my @files = splice(@tempList, 0, ($#fileList + 1) / $kids)){ my $pid = fork; push(@kids, $pid); if($pid == 0){ #kid &FTP_SUB(@files); exit; } } waitpid($_, 0) foreach (@kids); my $elapsed_time = time - $start_time; print OUT "Elapsed time with $kids kids:\t$elapsed_time seconds\n"; print "Elapsed time with $kids kids:\t$elapsed_time seconds\n"; } sub FTP_SUB { my @files = @_; my $start_time = time; my $end_time = undef; my $host = '...'; my $user = 'user'; my $pwd = 'pwd'; my $ftp = Net::FTP->new($host, Debug => 0); unless($ftp->login($user,$pass)){ warn "$$:\tCannot login " . $ftp->message; } unless($ftp->cwd("ftp_test")){ warn "$$:\tCannot change directory " . $ftp->message; } foreach my $file (@files){ unless($ftp->put($file)){ warn "$$:\tPut failed " . $ftp->message; } } $ftp->quit; $ftp = undef; }
__OUTPUT__ Elapsed time with 1 kids: 177 seconds Elapsed time with 2 kids: 92 seconds Elapsed time with 3 kids: 64 seconds Elapsed time with 4 kids: 56 seconds Elapsed time with 5 kids: 40 seconds Elapsed time with 6 kids: 34 seconds Elapsed time with 7 kids: 28 seconds Elapsed time with 8 kids: 27 seconds Elapsed time with 9 kids: 24 seconds Elapsed time with 10 kids: 23 seconds Elapsed time with 11 kids: 22 seconds Elapsed time with 12 kids: 20 seconds Elapsed time with 13 kids: 19 seconds Elapsed time with 14 kids: 18 seconds Elapsed time with 15 kids: 19 seconds Elapsed time with 16 kids: 17 seconds Elapsed time with 17 kids: 15 seconds Elapsed time with 18 kids: 17 seconds Elapsed time with 19 kids: 16 seconds Elapsed time with 20 kids: 16 seconds Elapsed time with 21 kids: 15 seconds Elapsed time with 22 kids: 14 seconds Elapsed time with 23 kids: 13 seconds Elapsed time with 24 kids: 12 seconds Elapsed time with 25 kids: 15 seconds Elapsed time with 26 kids: 13 seconds Elapsed time with 27 kids: 13 seconds Elapsed time with 28 kids: 13 seconds Elapsed time with 29 kids: 13 seconds Elapsed time with 30 kids: 13 seconds

You can see that the initial forks provided quite significant performance boosts. However, the improvement began to drop quickly once I was launching more than 7 children. Eventually there was no improvement whatsoever (though no cost, either).

Things to keep in mind:


In reply to Re^3: Performance of FTP Transfer by jrsimmon
in thread Performance of FTP Transfer by shijumic

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.