hibernian has asked for the wisdom of the Perl Monks concerning the following question:

Dearest Monk collective,

Having a trifle bit of hassle with Net::FTP and it's reluctance, on seemingly random occasions, to delete remote files. Let's have a looksie at some code, shall we?: 


sub get_file {

        # Similar system to sending the file over.  Give it a
        # different name when downloading incase it is parsed whilst
        # only partially loaded.

        my $file = shift;
        my $tmpfile = "DOWNLOADING-".$file;
        my $size_remote;
        my $size_local;

        $tabspaces++;
        logger("Downloading $file as $tmpfile");

        # Get size of the remote file that we wish to download.

        $size_remote = $ftp->size($file) or drop_dead("Could not obtai
+n size of $file - $!");
        logger("Size of $file is $size_remote");

        $ftp->get($file,$tmpfile)
                or drop_dead("Error downloading $tmpfile - $!");

        $size_local = -s $tmpfile;

        if ($size_local != $size_remote) {
                drop_dead("File size differs ( $size_remote <-> $ $siz
+e_local) - Error in download");
        } else {
                logger("Successful download ...");
        }

        logger("Renaming local $tmpfile as $file");
        rename($tmpfile,$file)
                or drop_dead("Could not rename local $tmpfile to $file
+ - $!");

        logger("Deleting remote file $file");
        $ftp->delete($file)
                or drop_dead("Could not delete remote $file - $!");

        $tabspaces--;
}

[download]
Nine times out of ten, this works with no problems. Ten times out of ten, the file is downloaded. The problem is with the delete() operation. Occasionally this fails, Net::FTP coming back with the utterly distressing "Bad File Descriptor", which makes faff all sense to me. I could understand the problem had I not been able to get() the file originally, but that is downloaded without any issues.

Have I a flaky Net::FTP?

Big thanks in advance for your offerrings of advice ...

Replies are listed 'Best First'.
Re: Net::FTP Bad File Descriptor - Much Weirdness
by gbarr (Monk) on Dec 03, 2001 at 18:23 UTC
    Have you tried turning on a debug trace ?

    I guess it is possible that Net::FTP is wrongly sending an ABRT at the end of the transfer. In which case the server may have closed the connection because there is nothing to abort.

[reply]
      From looking at FTP.pm, I cannot see where ABOR would be getting called in this context as the get() is always successful.

      I will look into your suggestion and try a debug though, thanks.

[reply]
Re: Net::FTP Bad File Descriptor - Much Weirdness
by hibernian (Acolyte) on Dec 03, 2001 at 17:18 UTC
    Things are a little desperate when you reply to your own questions: Okay, I've looked at the source to FTP.pm and note that when deleting, we simply pass the 1st argument (hopefully a filename) to the ftp DELE command, therefore the fault is either with the FTP server (SCO Openserver, WU-FTPD v2.1 - ouch ... patch that baby!) or with the fact that we're having to do this over a passive session over port-forwarding connection. Anybody have any issues with using ftp over a passive session?
[reply]
Re: Net::FTP Bad File Descriptor - Much Weirdness
by Fastolfe (Vicar) on Dec 03, 2001 at 22:20 UTC

    A couple of things:

    1. Net::FTP does not place errors in $!. This variable is usually reserved for system error messages, not application errors. Consider using the message() and/or code() methods to get the last message returned by the FTP server. (Note: This isn't tested, but since Net::FTP inherits from Net::Cmd, I would expect this would work.)
    2. You're doing a size check after $ftp->get, second-guessing the success/failure response of the get() method. Is there a reason you're doing that? Have you ever had a situation where the file was apparently successfully transferred yet the sizes were different (implying a truncated download)? I'm kind of curious. This check seems redundant to me, but you may have experiences that really do show a problem (perhaps something that should be communicated to the Net::FTP authors). Note that I might anticipate different file sizes if you're transferring a file using ASCII mode between systems with incompatible newline conventions. This shouldn't be considered a failure.
[reply]
[d/l]
[select]
      Hi there,

      Both points were very helpful - particularly 1) which explains the odd "Bad File Descriptor" message which had me tearing my virtual (I don't have any) hair out.

      As for point 2), yes, I was having problems with files not uploading completely and yet get() would not return with an error, hence the size check. I know already that every file being downloaded is gzipped, therefore I specify the transfer to be binary mode, in which case, the size of the file reported by the ftp server remotely will match (on a successful transfer) the file downloaded.

      However, what did enlighten me was the failure of our back-up ISDN link to the remote server (we were using ftp over a VPN over the Internet). It transpires that the intructions were being sent over the Internet, and yet the data was being received via the ISDN link.

      Yesterday, the ftp client would connect but could not receive any data. Now I need wave a large stick at whoever configured the firewalls.

      I imagine this is where the problem of the short files is coming from - the ISDN linking is either timing out or falling over. Most irritating.

      Thanks again,

      -Gregor-

      -- Gregor Anderson, Edinburgh. http://www.deletia.org/

[reply]
        Still, get() should not be returning a true value unless the transfer was completed successfully (if you ask me). This sounds like a bug with Net::FTP. I might contact the author and describe what happened. It's possible that this is actually intended behavior, so that you can resume the FTP transfer later. But there must be a better way of determining this than checking the size of the file by hand. As I mentioned, an ASCII transfer makes no guarantee that the size of the file will be the same on one side as the other.

        Glad you got it figured out though.

[reply]
Re: Net::FTP Bad File Descriptor - Much Weirdness
by scain (Curate) on Dec 03, 2001 at 21:51 UTC
    hibernian,

    This may not be terribly helpful, but have you looked at the problematic files? Do they have funny characters in the name, do they have different file permissions, have you tried to do this from the command line as the user that is evoking the script? These are all typical debugging procedures, but I thought I'd throw it out there to see if it would help.

    Scott

[reply]
      Hi Scott,

      All very helpful but I've checked all the obvious problems like these already.

      Thanks, though.

      -- Gregor Anderson, Edinburgh. http://www.deletia.org/

[reply]

Back to Seekers of Perl Wisdom