in reply to Re: Net::FTP takes a very long time in Passive mode
in thread Net::FTP takes a very long time in Passive mode

Thanks so much for all that detail. I'm sorry to have to tell you that the machine doing the transfer to the Windows machine isn't a LINUX box, it's actually Mac OS X. There are actually three webservers involved.

The process is:

  1. Script on OS X webserver uploads and parses files.
  2. Script on OS X webserver FTPs to internal LINUX box (all goes well)
  3. Script on OS X webserver FTPs to NT/Firewall box (long delay)

So I need to find out how to tweak those values on the Mac server.


($_='kkvvttuubbooppuuiiffssqqffssmmiibbddllffss')
=~y~b-v~a-z~s; print
  • Comment on Re^2: Net::FTP takes a very long time in Passive mode

Replies are listed 'Best First'.
Re^3: Net::FTP takes a very long time in Passive mode
by hsinclai (Deacon) on Oct 16, 2004 at 06:52 UTC
    Ah.

    Well on OSX that should be done with sysctl, which should be fairly analogous to the old FreeBSD sysctl. There are less values than Linux but you can set these at least: 
    net.inet.tcp.sendspace: 32768                                         
+                           
net.inet.tcp.recvspace: 57344                                         
+                           
net.inet.tcp.path_mtu_discovery: 1

    [download]

    You should be able to use (as root) this form 
    sysctl -w net.inet.tcp.recvspace=40960

    [download]

    or in /etc/sysctl.conf set the value with this form 
    net.inet.tcp.recvspace=40960

    [download]
    so any values are maintained across reboots.

    So I suspect the thing to look for is your default send window being too large. If that was the case the delay could be caused by excessive packet reassembly problems, with more hops to the NT probably making it worse. I would also look on the NT box at the firewall logs when you connect .. PS I don't know how close those commands are but it should get you close..
[reply]
[d/l]
[select]
      I said I'd get back to you on Monday ... it's Wednesday now but what the hell... OK I set the values, just as you said, with 
      sudo sysctl -w net.inet.tcp.recvspace=40960

      [download]

      Etc., but it doesn't seem to have made any difference.

      The upload takes just over two minutes for a one-line text file.

      But I turned on Debugging in Net::FTP -- should have done that before, shouldn't I? And found something interesting. 

      Net::FTP=GLOB(0xfc84)>>> ALLO 5
Net::FTP=GLOB(0xfc84): Timeout at ftptest.pl line 17
Net::FTP=GLOB(0xfc84)>>> PASV
Net::FTP=GLOB(0xfc84)<<< 227 Entering passive mode (202,6,74,98,243,97
+)
Net::FTP=GLOB(0xfc84)>>> STOR adsf.txt
Net::FTP=GLOB(0xfc84)<<< 125 Data connection already open; Transfer st
+arting.
Net::FTP=GLOB(0xfc84)<<< 226 Transfer complete.

      [download]

      It seems it's trying to do a non-passive transfer, (line 17 is the "put()" command), timing out after two minutes, then issuing the PASV command and doing it instantly ... what's that all about?

      Update: I issued the PASV command myself before the "put()" command, but it made no difference. It still timed out and then did its own PASV command.

      Update-Update: I realised a couple of seconds ago that I could just set the timeout to some low value and it would instantly time out then move on to the PASV thing. But I'd still like to know what's going on...


      ($_='kkvvttuubbooppuuiiffssqqffssmmiibbddllffss')
      =~y~b-v~a-z~s; print
[reply]
[d/l]
[select]
        Wow. Great observation about the passive stuff on the debug session. There is your timeout and the cause for the delay..

        The N::F documentation also mentions setting PASSIVE mode in the environment.. so early in the script .... $ENV{'FTP_PASSIVE'} = 1;
        Will that do anything?

        About those earlier values for sysctl - actually what I suggested (it must not have been all that clear, so sorry) was to see what the OSX system was set at with respect to TCP window sizes, etc. And then tweak from there.

        The numbers in my OP were just placeholder numbers.. or values on systems here. So the goal was to try to go from a known bad state to a hopefully improved one. Also valuable would be to check the settings on the systems that were FTPing the files quickly, compare and match those settings on the faulty system, and see if it made a difference.

        Another long shot - on that OSX system there isn't an exported environment variable ftp_proxy floating around is there?

        ... "put()" command), timing out after two minutes, then issuing the PASV command and doing it instantly ... what's that all about?

        Have you found a bug in Net::FTP?

[reply]
[d/l]
      Thanks a lot, again. I'll report back on Monday.

      ($_='kkvvttuubbooppuuiiffssqqffssmmiibbddllffss')
      =~y~b-v~a-z~s; print
[reply]

In Section Seekers of Perl Wisdom