cknowlton has asked for the wisdom of the Perl Monks concerning the following question:

I am searching for clairity in my pursuit of Perl wisdom. In particular, What exactly does the "Timeout" option in the "new (HOST [,OPTIONS])" Constructor in Net::FTP wait for?

In the description, at Perldoc.com. It gives a generic definition:

"Timeout - Set a timeout value (defaults to 120)"

Is it an idle timeout, a login auth timeout, or what?

I have a software application that is in constant development, and there are many users in remote locations testing it. To keep all the test machines in sync with the latest development version, I've written a perl script using Net::FTP which compares their files against files on my "update" machine, which replaces any files that have been updated, on a daily basis.

The Problem I'm seeing, is that sometimes a file transfer will fail in different places after a pause that I am attributing to a what looks like timeout value somewhere in the process. I have increased all of my server timeouts, and it is still terminating prematurely sometimes on slow connections. So I'm wondering if the "Timeout" option in Net::FTP will help. Any ideas?


Thanks!

-Carlos

Replies are listed 'Best First'.
Re: Net::FTP Timeout
by pg (Canon) on Dec 16, 2002 at 18:31 UTC
    The timeout value you set is 'passed' to IO::Socket::INET (Net::FTP @ISA IO::Socket::INET, if you open up the package, you will realize this), and will be applied to operations against that socket.

    Not in doc for Net::Ftp, but in doc for IO::Socket::INET, you can find the detailed explanation on timeout.

    Update:Carlos, I looked at those testings you mentioned, in your reply to Zapawork. Yes, those test cases are valid. Let me explain:
    1. First of all the timeout value is absolutely not only for socket creation, or accept, but for 'various' operations as stated in doc for IO::Socket::INET, so it is also for things like read/write/send/recv, which are behind your get/put from a Net::FTP view.
    2. The purpose of timeout is to leave space for retry and recovery. You said that you set timeout to 120 seconds, you pulled out the cable in middle of get operation, and the get operation timed out after 120 seconds from the moment you pulled out the cable. That's exactly what you should expect. If it is counted from the beginning of the operation, it does not make any sense. For example, you set timeout to 120 seconds, you start getting a huge file, at the 130th second, you pull out the cable, if timeout is counted from the beginning of the get operation, then it will timeout right away, there is no space for retry or whatsoever. this is not what you want, right?
    3. You also did another test, in which you pluged the cable back before timeout, and the transfer is successful. I believe you already know the reason. Yes, you are right, that's the space for retry/recovery, and the recovery was successful before the timeout.

      Thanks for the update, I feel a serene sense of enlightenment! =')

      I'm also going to check out that 'rsync' utility that others have pointed out. (thanks guys!)


      -Carlos
Re: Net::FTP Timeout
by Marza (Vicar) on Dec 16, 2002 at 18:14 UTC

    Timeout is the amount of time you want to give the command to work. If it hits the limit, it dies

    It has a purpose if you know something should not take long. For example an FTP to a local server versus one of a slow wan link. You would increase the timeout for the slow wan and most likely decrease for the local server

    If you want more network programming, take a look a Lincoln Steins "Network Programming with Perl"

Re: Net::FTP Timeout
by Zapawork (Scribe) on Dec 16, 2002 at 19:38 UTC
    Hi Carlos,

    To hopefully add a bit of clarity. I have the stein book in front me, which is a must read to say the least. The timeout is a timeout on the socket operation itself. Which means, it will attempt to create a socket to the host you specify BUT if a socket is not created in 120 seconds then the operation itself will time out and the operation will return an error.

    Hopefully you are catching such an error e.g.

    my $ftp = Net::FTP->new(HOST) or die "Can't connect: $@\n";

    and if you want to you can get the client to retry it's connection based on that error. The other thing to think about is if you are getting these errors often you might want to grab a network sniffer to determine if your network links are being overloaded or if the server is getting overloaded with connections and is becoming slower to respond.

    In either case you might want to set the clients to connect in groups by time to prevent a small scale denial of service attack on your server, I don't know how many clients you have, and from consuming all possible bandwith.

    Hope this helps!,
    Dave -- Saving the world one node at a time

      Thanks for your reply. I have a question in reference to your first paragraph regarding the nature of socket creation: Is a socket created only uppon connection, or is it done continually throughout the connection?

      I ran a little test where I began a connection, and started a file transfer:

      $ftp_a = Net::FTP->new($remhostname, Timeout => 120, Debug => 1) || &error && die "Cant connect to $remhostname\n"; $ftp_a->login ($remusername,$rempassword) || &error && die "Cant login to $remhostname\n"; $ftp_a->binary (); $ftp_a->cwd ("$home\/$fpath") || warn "Cant find Directory $home/$fpath\n"; $ftp_a->get("$fname") or &error && die $ftp_a->message; $ftp_a->quit;

      After the file transfer had begun, I disconnected the network cable, and watched as the script was terminated at 120 seconds from the time I removed the cable.

      I then ran the test again with the Timeout value set to 900. Again, the script terminated at the Timeout value. If the connection was restored at any time within the timout period, the transfer would continue as normal.

      I don't have a lot of experience with network programming, but this seems to imply that the timeout is for more than just the connection. However, I still don't know specifically what.


      Best Regards,

      -Carlos

      "If at first you don't succeed, skydiving is not for you!"
Re: Net::FTP Timeout
by waswas-fng (Curate) on Dec 17, 2002 at 00:55 UTC
    You may want to look at www.rsync.org for a file mirroring tool for unix. It is nice, It does checksums and only transfers the portions of files that have changed also it is fairly secure. It also comes with many unix flavors now. There may not need to be any reason to reinvent the wheel here. just my 2 cents.

    -Waswas
      I didn't even finish reading your post before I started thinking why aren't you using rsync? Most CPAN mirrors are using rysnc now...

      Cheers,

      -J