Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

A coworker has a VB script transferring local files to a remote server over DSL and they are going up at like 15 - 20 files A SECOND. File sizes are anywhere from 50 - 150 K or so. He said something about using a stream or socket (?) and maybe a windows dll. The best I could do with a home grown perl ftp script was 1 or two files a second. For that matter, Net::Ftp wasnt much better, if at all. How can I use perl to transfer files at insane speeds like this?

Replies are listed 'Best First'.
Re: Insane ftp speeds
by kvale (Monsignor) on Jun 12, 2002 at 21:38 UTC
    Hmm, without knowing more about your DSL setup, it is hard to say anything definite. As a data point, I wrote a perl program running on Linux 2.4 that skims Usenet (NNTP protocol) and easily saturates my DSL data pipe at 160KB/sec. The best I have ever done on Windows 98 using proprietary programs on the same machine is 140KB/sec for any protocol. The perl code involved no special tricks.

    So from personal experience, I think it it is unlikely that perl's socket interface is much worse than the Windows implementation.

    One possible difference is that you and he have different upload capacities in your DSL setups - they can easily vary by two orders of magnitude, depending on what you pay for. Also, your friend seems to have an upload capacity of 2 MByte/sec which translates to 20Mbits/sec, for which a 10baseT Ethernet connection would be a bottleneck.

    But if your perl code is still suspect after checking out that both your hardware setups are of similar speed, post the relevant portions of your code here so that we can comment on them.

    -Mark
[reply]
Re: Insane ftp speeds
by newrisedesigns (Curate) on Jun 12, 2002 at 22:28 UTC

    Is this actual speed, or just the text report?

    I've found many of my files upload quickly, but report 0.00sec, 624000kb/s, which I know isn't the case.

    Maybe he's just trying to persuade you to the dark side...

    John J Reiser
    newrisedesigns.com

[reply]
Re: Insane ftp speeds
by orkysoft (Friar) on Jun 12, 2002 at 22:29 UTC

    Have you verified his files arrive intact? It might be because his program is broken, and doesn't actually transfer the files. It can also be a bad measurement. Also, read the other repl(y|ies) in this thread.

    Lur: "But if this cape shrinks, consider your species extinct!"

[reply]
Re: Insane ftp speeds
by Anonymous Monk on Jun 17, 2002 at 16:31 UTC
    Me again (guess I need to sign up). The VB and Perl scripts were run against each other in the same office and VB clearly spanked me. My code is listed below, but (I think) it is pretty straight forward. I was told that the VB was running as a system process or something and that that gave it a higher priority and larger bottleneck or something. (?). net::ftp mentions something about increasing write and/or read buffer size. Would that improve my speed? 
    sub upload_process{
        $ftp = Net::FTP->new("$remote_server") or &error;
        if($ftp->login("$FORM{username}", "$FORM{password}")){
            for(@list){
                if($ftp->ok()) {
                    $ftp->put("N:/$local_home$_", "$start_folder$_");
                }else{
                    print $ftp->code()." ".$ftp->message()."\n";
                    $ftp->quit;
                }
             }
        }
        $ftp->quit;        
}

    [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom