Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Morning,

I've trying to use Net::FTP to programatically retrieve files from an FTP server. The program I've got logs in to the FTP server every 30 minutes or so, lists the directory contents, and then downloads any files it needs and closes the connection.

The issue I have is that the code runs continuously, and, after monitoring the ports that Net::FTP and IO::Socket::INET open, it eventually runs out of ports to listen on!

Net::FTP calls IO::Socket::INET like this:

my $sock = IO::Socket::INET->new( Listen    => 5,
                                  Proto     => 'tcp',
                                  Timeout   => $ftp->timeout,
                                  LocalAddr => $ftp->sockhost);

Which creates a listening socket on the next port available. However, it never re-uses the same port twice, so the port numbers go up, and up and up... Until eventually the code fails to be able to create a listen port.

Anyone got any ideas on what can be done about this??

Ta,

Julian

Replies are listed 'Best First'.
Re: Net::FTP and Ports
by demerphq (Chancellor) on Feb 06, 2004 at 11:45 UTC

    I see a ReuseAddr and a ReusePort attribute that is acceptable to IO::Socket::INET->configure(), perhaps changing it to 

    my $sock = IO::Socket::INET->new( Listen    => 5,
                                  Proto     => 'tcp',
                                  Timeout   => $ftp->timeout,
                                  LocalAddr => $ftp->sockhost
                                  ReusePort => 1 
                                 );

    [download]

    Will work? BTW, I would do this via a subclass on Net::FTP so you don't affect anything else using it.


    ---
    demerphq

      First they ignore you, then they laugh at you, then they fight you, then you win.
      -- Gandhi
[reply]
[d/l]
[select]

      Thanks for the feedback, my initial thought was that it's a socket de-allocation problem as well, but I'm struggling to replicate the problem.

      The platform is Windows XP Professional, running Activestates' ActivePerl 5.6.1 Build 635. Net::FTP version is 2.65 which seems to be the newest available for this build. The whole lot is compiled into a free standing Win32 Service using Perl Dev Kit PerlSvc v3

      Playing around testing, I've tried the Reuseport and Reuseaddr options, but these throw back a "Macro Not implemented" result, which makes sense if you think about it as the RFC's state that only one service can run on any given port at any time, and forcing ReUsePort would cause IO::Socket to re-allocate the port immediately (not just when it's been freed). Incidentally, by creating multiple instances of IO::Socket::INET, I've discovered that the max sockets I am allowed to create is 255. However, in testing by performing quick recursive FTP connects/lists/transfers/disconnects to a local FTP server I can't replicate the problem despite performing over 12,000 iterations in the space of a few minutes!

      This leads me either to the conclusion that something in my production code isn't correctly de-allocating the ports - or the WinXP API is occasionally allocating me a port that it then allocates to someone else consecutively!

      Any ideas...?

[reply]

        To be honest my idea would be to just lose the service and change to a scheduled task. I have many scripts running now that basically do what you describe, but run as scheduled tasks and not as services, and they seem to work just fine. Unless there is a really good reason to run as a service (ive yet to hear one in my job but....) then setting up a repeating scheduled task (not using the at command but the scheduled tasks facility available through the control panel) should do you nicely.

        Beyond that anything I could offer would be completely off the top of my head conjecture and unlikely to be useful.
        :-(
        Sorry.


        ---
        demerphq

          First they ignore you, then they laugh at you, then they fight you, then you win.
          -- Gandhi
[reply]
[d/l]
Re: Net::FTP and Ports
by castaway (Parson) on Feb 06, 2004 at 11:45 UTC
    I don't have any concrete ideas, but my co is using Net::FTP in production environments to do this sort of thing, and has never run across this problem. Currently running on Linux (SuSE), HP-UX, and AIX (clients), with HP-UX, AIX and Windows (IIS) as servers. My guess would be its an OS problem, where something in the setup isnt letting the closed ports time out properly.

    Maybe you can show your client/server details, along with perl and Net::FTP versions?

    C.

[reply]
Re: Net::FTP and Ports
by ctilmes (Vicar) on Feb 06, 2004 at 12:08 UTC
    I have to agree with some of the others. I don't think this is a problem with Net::FTP. You have experienced a problem with your code and are speculating on what you think is a problem in Net::FTP.

    Show the actual code (make a small program that illustrates the problem) you are using to interact with Net::FTP.

    Without seeing your code, this is just a guess, but perhaps you aren't closing the FTP connection each time? Either destroy the Net::FTP object and re-create it with each iteration, or make sure you are calling 'login'/'quit' each time.

[reply]

      Now this is where it gets interesting... I've already added tracking to see where the ports get opened and closed, and I am definitely constructing and destructing (destroying?) correctly:

      WARNING:
Opened port 1769 for FTP at c:/Perl/site/lib/Net/FTP.pm line 803.
WARNING:
Closed port 1769 for FTP at c:/Perl/site/lib/Net/FTP.pm line 929.

      The code I'm using a little bit lengthy because I'm wrapping Net::FTP within another package to present a consistent interface to another system. Also, it's not the number of iterations that causes the problems as now I've added a counter onto the system to see how many iterations it gets to before crashing out, and it's breaking the 255 barrier easily, indicating that it is most likely closing sockets fine.

      I've come across this before on Linux some time ago, with code that used to run for weeks and then suddenly fail to make a connection.

[reply]

        Dang and Blast it.

        Turns out it's a problem with ActiveState's PerlSvc Freestanding Exe builder which for some reason limits the number of sockets that can be opened (from startup to destruction) to 169!

        Upgrading from V2.1 to V5.3 seems to have fixed the problem

        Thanks to everyone for their help,

        Julian

[reply]
Re: Net::FTP and Ports
by chimni (Pilgrim) on Feb 06, 2004 at 10:59 UTC

    Why would you need IO::Socket::INET
    Would not the following suffice. 
    my $dir='/whatever';
use Net::FTP;

my $ftp = Net::FTP->new("severname fqdn")|| die("Could not connect: $@
+\n");
$ftp->login("user","password")||die("Wrong username or password "
);
        $ftp->binary();
        $ftp->cwd("$dir")||die("Could not change path ");
        #@aa=$ftp->ls("$dir");
        $ftp->get("$file_name")|| die("File not found $file_name\n");
        $ftp->quit;

    [download]

    That should do it
    Regards,
    Rajdeep
[reply]
[d/l]
      Oh you so missed the point...read the question again...


      kodo
[reply]

Back to Seekers of Perl Wisdom