http://qs1969.pair.com?node_id=181428

George_Sherston has asked for the wisdom of the Perl Monks concerning the following question:

Can anyone suggest why the following runs at the command line but dies on $ftp->ls when run from the web browser? I'm completely baffled. But this really is what's happening - exact same file, does what it says on the can when run from the command line, dies in the browser. Did I mention I was baffled?
#!/usr/bin/perl -w use strict; use lib '/home/httpd/lib'; use CGI qw/:standard :cgi-lib/; print header; use jbr::Carp qw(fatalsToBrowser warningsToBrowser); warningsToBrowser(1); use Net::FTP; $| = 1; my $ftp = Net::FTP->new("ftp.xxxxx.xxx", Debug => 0); $ftp->login('xxxxxxx','xxxxxx'); $ftp->cwd('/tst') or die; my @files = $ftp->ls or die; print h2 "deleting old pages..."; for (@files) { print p $_; $ftp->delete($_); }


§ George Sherston

Replies are listed 'Best First'.
Re: Net::FTP via CGI mystery
by runrig (Abbot) on Jul 13, 2002 at 01:56 UTC
    • Run it in debug mode
    • Print the status of FTP statements that fail (and the error messages are not in "$!"; you have to read both the Net::FTP and Net::Cmd docs).
    On connect the error message is in "$@", for Net::Cmd (which a Net::FTP object is a subclass of), I forget, its some method.
Re: Net::FTP via CGI mystery
by rah (Monk) on Jul 13, 2002 at 01:57 UTC
    Have you tried cranking up the debug level on Net::FTP? A 2 or 3 will give you a blow by blow account of the FTP session. Maybe that will give you some idea why it dies on the ls.

    Also, and this may be insignificant, shouldn't
    $ftp->ls be
    $ftp->ls() ?
    I don't know how this could make it behave differently on the command line vs. http, but that might be worth a try.

      I tried increasing the debug level as you suggest, and it leaves me more baffled, but in a more specific way. First, I boiled down the script to the absolute minimum necessary to recreate the problem. It's now:
      #!/usr/bin/perl -w use strict; use Net::FTP; #use CGI::Carp qw(fatalsToBrowser warningsToBrowser); my $ftp = Net::FTP->new("ftp.xxxxx.xxx"); $ftp->pasv; $ftp->login('xxx','xxx'); my @files = $ftp->ls('/tst') or die "could not do ls"; print "Content-Type: text/html\n\n"; print "$_\n" for @files; print "done\n";
      Now, it still works fine when run from the command line, and fails when run from the browser. When it fails in the browser I just get an internal server error, because I deleted CGI::Carp, but reviewing the error logs shows the following (n.b. I tried this with a different ftp server too, so I really think that the problem is here):

      Output from the script when run from the COMMAND LINE

      Net::FTP: Net::FTP(2.61) Net::FTP: Exporter(5.562) Net::FTP: Net::Cmd(2.20) Net::FTP: IO::Socket::INET(1.25) Net::FTP: IO::Socket(1.26) Net::FTP: IO::Handle(1.21) Net::FTP=GLOB(0x81177a8)<<< 220 xxxxxxxx.xxx ULTRA FTP SERVER Net::FTP=GLOB(0x81177a8)>>> PASV Net::FTP=GLOB(0x81177a8)<<< 530 Please login with USER and PASS. Net::FTP=GLOB(0x81177a8)>>> user xxx Net::FTP=GLOB(0x81177a8)<<< 331 Password required for xxx. Net::FTP=GLOB(0x81177a8)>>> PASS .... Net::FTP=GLOB(0x81177a8)<<< 230 CyBo welcomes replicant xxx. Net::FTP=GLOB(0x81177a8)>>> PASV Net::FTP=GLOB(0x81177a8)<<< 227 Entering Passive Mode (XXX,XX,XXX,XXX, +XXX,XXX). Net::FTP=GLOB(0x81177a8)>>> NLST /tst Net::FTP=GLOB(0x81177a8)<<< 150 Opening ASCII mode data connection for + file list Net::FTP=GLOB(0x81177a8)<<< 226 Transfer complete. Content-Type: text/html /tst/graphics /tst/style /tst/images done Net::FTP=GLOB(0x81177a8)>>> QUIT Net::FTP=GLOB(0x81177a8)<<< 221 Goodbye.

      Error Log report when run from the WEB BROWSER

      Net::FTP: Net::FTP(2.61) Net::FTP: Exporter(5.562) Net::FTP: Net::Cmd(2.20) Net::FTP: IO::Socket::INET(1.25) Net::FTP: IO::Socket(1.26) Net::FTP: IO::Handle(1.21) Net::FTP=GLOB(0x8286474)<<< 220 xxxxx.xxx ULTRA FTP SERVER Net::FTP=GLOB(0x8286474)>>> PASV Net::FTP=GLOB(0x8286474)<<< 530 Please login with USER and PASS. Net::FTP=GLOB(0x8286474)>>> user xxx Net::FTP=GLOB(0x8286474)<<< 331 Password required for xxx. Net::FTP=GLOB(0x8286474)>>> PASS .... Net::FTP=GLOB(0x8286474)<<< 230 CyBo welcomes replicant xxx. Net::FTP=GLOB(0x8286474)>>> PORT XXX,XX,XXX,XXX,XXX,XXX Net::FTP=GLOB(0x8286474)<<< 200 PORT command successful. Net::FTP=GLOB(0x8286474)>>> NLST /tst Net::FTP=GLOB(0x8286474)<<< 425 Can't build data connection: Connectio +n refused could not do ls at ftp.pl line 9. Net::FTP=GLOB(0x8286474)>>> QUIT Net::FTP=GLOB(0x8286474)<<< 221 Goodbye.
      So... it seems that what's happening is that when run from the browser the script doesn't go into passive mode (whatever that means)... but I thought it already was in passive mode. I'm completely out of my depth here, and I would be most grateful for a steer on how to get this to work.

      § George Sherston

        The following extract from the wsftp help might clarify things a little?

        Passive Transfers (definition) Normally, when you connect to an FTP site, the site establishes the da +ta connection to your PC (the client). However, if the site allows pa +ssive transfers, you can have your PC establish the data connection. We recommend that you use passive mode for most transfers. Note that p +assive mode may be required for users who are behind some types of ro +uter-based firewalls or behind a gateway requiring passive transfers.

        My only thought is that maybe one way is bypassing a proxy/firewall and the other isn't? HTH.

SOULUTION: Net::FTP via CGI mystery
by George_Sherston (Vicar) on Jul 13, 2002 at 14:46 UTC
    Molto thanks to tjh in the CB for the solution to this problem (and shame on me for not reading the manual finely enough) - the answer is to create the new ftp object with Passive mode turned on,
    my $ftp = Net::FTP->new("ftp.xxxxxxxxxx.xxx", Passive => 1, Debug => 1 +);
    So that's fixed. Although I'm still puzzled why one needs to do something different in order to call the script in a different way...

    § George Sherston
      No prob. I believe FTP clients open 2 ports, 1 out and 1 in (control and data, IIRC).

      Your shell login is likely running on a box that will allow the return port to be opened. The web server box (the one running your script), is likely behind a firewall that won't allow the return (data) port to open. You however are succeeding in logging in via the already open control port.

      If I missed badly others should correct me :), but it sounds good eh?

      Update: Beatnik followed up by pointing out to me more specifically that port (20), the traditional active FTP data port on the ftp server, may be blocked, while port (21), the traditional ftp control channel, may not be.

Re: Net::FTP via CGI mystery
by amphiplex (Monk) on Jul 13, 2002 at 09:42 UTC
    Hi !

    The script worked for me. Are you sure you haven't started it from the commandline and then "from the browser" ?
    It would then die because there are no more files.

    ---- kurt
Re: Net::FTP via CGI mystery
by rah (Monk) on Jul 13, 2002 at 20:37 UTC
    Glad you got it to work. I don't believe you said if your shell account was on the same box as the webserver. If your shell account is on a different box, active ftp may be denied to the webserver box. This could be easily done with an access list on a router, no firewall required. If your login is on the same box, then a more complex explaination is required, and a firewall is more likely involved. In that case, it might be that your login is allowed to do active ftp, but the login your CGI runs as (www, apache, nobody, etc.) is denied by the firewall. Given my limited knowledge of network security I can't understand how this could be accomplished. The actual tcp connections are anonymous. Unless perhaps the firewall is on the same box.

    Passive ftp is typically allowed, because it is viewed as less of a security risk. In passive ftp the client open both connections. A common assumption in network security is that if the inside box is initiaing the connections it must be OK.

    For an excellent description of active vs. passive ftp try this.

      It's the same file on the same box - which was what made it so puzzling. I think tjh is right about it being an environment issue, but it's not at all my area of expertise (in fact it's not even my area of knowing-anything-about-it-at-all, so far...). Thanks (and to other sibling monks) for your help and advice on this.

      § George Sherston
Re: Net::FTP via CGI mystery
by FuzInc (Initiate) on Sep 04, 2013 at 15:32 UTC

    I'm having the same problem except it connects to the remote server, and prints at the "Connected to $host" line, but then stops completely just before or during the 'get' command. This is when run from the browser. It works perfectly from the command line. I don't understand!

    I'm a bit of a newbie, and so I'm not sure on how to generate debug information in the browser window. Help!? I've tried all the suggestions I've seen on here so far (I think) with no joy - including the passive settings, and I'm not sure what to do next!

    Basically I'm trying to automate the movement of a file from one server (FTP) to another (SFTP) on our system. I know th path to file and I'm not using cwd() as it works fine in command line...would cwd() help?

    Any help would be hugely appreciated! I haven't even gotten to any possible problems with Net::SFTP::Foreign!

    Here's the code:

    #!/usr/bin/perl $|=1; print "Content-Type: text/plain\n\n"; use warnings; use strict; use Net::SFTP::Foreign; use Net::FTP; my $host = "xxxxxxxxxxxxxxxxxx"; my $user = "yyyyyyyyyyy"; my $password = "zzzzzzzz"; my $securehost = "aaaaaaaaaaaaaaaaa"; my $secureuser = "bbbbbbbbbbbbbb"; my $securepassword = "cccccccccccccc"; my $timeout = "120"; my ($sec,$min,$hr,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(tim +e); $mon++; $year += 1900; my $handle = $year . $mon . $mday; my $ftp = Net::FTP->new($host, Debug=>3, Timeout=>120, Passive=>1); $ftp->error and die "unable to connect to remote host: " . $ftp->error +; $ftp->login($user,$password); print "Connected to $host\n"; $ftp->get("/remote/path/to/file/file.txt", "/local/path/to/file/file_$handle.txt") or die "file transfer failed: " . $ftp->error; print "file.txt transferred locally and filename changed to file_$hand +le\n"; $ftp->quit(); my $sftp = Net::SFTP::Foreign->new($securehost, user=>$secureuser, pas +sword=>$securepassword, timeout=>$timeout); $sftp->error and die "unable to connect to remote host: " . $sftp->err +or; print "Connected to $securehost\n"; $sftp->put("/local/path/to/file/file_$handle.txt", "/remote/path/to/fi +le/file_$handle.txt") or die "file transfer failed: " . $sftp->error; print "file_$handle uploaded to $securehost at $hr:$min."; $sftp->disconnect(); exit;