FTP & MD5

Ronnie has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to write a script that will get a file from a PC and place it on a local server. Using Net::FTP this works fine......however a bright spark in audit has decided that the file on the PC should be MD5'ed before transfer and the MD5 validated on the server after retrieval. Fine! Just one problem, now that I don't know the full name of the file to be transferred I need to be able to use a wildcard. This I cannot get to work. Is this possible? The remote PC has Perl installed if that helps?

 
#!/usr/bin/perl -w 
#
use Net::FTP ;                                       # Used because se
+ctran requires a responder on the PC
#
#
###############################################################
#                    FTP Variables                            #
###############################################################
#
my $file_in       = "TRISCAN.txt.*" ;
my $password      = 'summit' ;
my $username      = 'summitelse' ;
my $host          = 'PCxxxx' ;
my $ftp           = undef ;
my $msg           = undef ;
#
####################################################################
#                     P R O C E S S I N G                          #
####################################################################
#
   print "\n\tNet::FTP Test begins\n" ;
#
   if ($ftp = Net::FTP->new($host, Debug => 0, Timeout => 10)) {
      print "\n\tFTP Connection established with $host\n" ;
   }
   else
   {
      $msg = "\n\tFTP Connection failed with $host\n" ;
      print $msg ;
      exit 0 ;
   }
#
################################################################
#          Login with the username/password details            #
################################################################
#
   if ($ftp->login($username,$password)) {
      print "\n\t$username login accepted\n" ;
   }
   else
   {
      $msg = "Username $username or password invalid!" ;
      print "\n\t$msg\n" ;
      exit 0 ;
   }
#
###############################################################
#                Try and get the file                         #
###############################################################
#
   if ($ftp->get("$file_in")) {
      print "\n\tYippee we goat the bleedin' file!\n" ;
   }
   else
   {
      $msg = "Oh Frag we didnae get the blighter!" ;
      print "\n\t$msg\n" ;
      exit 0 ;
   }
#
##############################################################
#               Try and quit the FTP                         #
##############################################################
#
   $ftp->quit ;
#
######################################################################
+#####
# Check that the transfer went okay. If the file doesn't exist the    
+    #
# transfer definitely failed.                                         
+    #
######################################################################
+#####
#
   if (! -e $file_in) {
     $msg = "No $file_in exists after FTP has completed" ;
     exit 0 ;
   }
#
#############################################################
#                      Toodle pip                           #
#############################################################
#
print "\n\tNet::FTP Test Ends\n" ;
[download]

Any suggestions?

READMORE tags added by Arunbear

Comment on FTP & MD5 Download Code

Replies are listed 'Best First'.
Re: FTP & MD5 by thor (Priest) on Jun 21, 2005 at 13:42 UTC
Your "bright spark" isn't very bright at all. FTP access to a server doesn't imply the ability to obtain an MD5 sum of files on said server. Moreover, in the general case, it is not a requirement that the files have the same bytes on the local and remote side. Transferring in ASCII mode will convert line endings to the local notion; if the two systems have a different idea of what constitutes an end of line ("\r\n" vs "\n" vs "\r"), ftp will convert them on-the-fly. So, the two files needn't even hash to the same value. However, you said that you're transferring from one PC to another, so that might help. The only way I could see pulling this off is to have pre-computed md5 files lying around for each file that you'd want to transfer. You'd grab the file you want, it's corresponding hash file, hash the file when you're done transferring it, and compare the computed value with the hash file that you got from the server. HTH, thor `Feel the white light, the light within Be your own disciple, fan the sparks of will For all of us waiting, your kingdom will come`	[reply]
Re^2: FTP & MD5 by Fletch (Bishop) on Jun 21, 2005 at 13:46 UTC
Not to mention that if you're not doing any sort of authentication of the conversation with the other end there's nothing to stop someone able to muck with the data in transit from mucking with the MD5 as well. -- We're looking for people in ATL	[reply]
Re^2: FTP & MD5 by shiza (Hermit) on Jun 22, 2005 at 00:13 UTC
Transferring in BINARY should eliminate size differences, but, that still doesn't solve the problem.	[reply]
Re: FTP & MD5 by bofh_of_oz (Hermit) on Jun 21, 2005 at 13:42 UTC
Note: your script shouldn't work as is, because $ftp->get needs a single file name as argument, while you're giving a wildcard. That means you need to find out the exact name of the file to transfer, before the transfer. One way to do so is: `my @files=$ftp->ls or $error=1; $ftp->quit if $error; foreach(@files) { if (/^yourRegex/i) #Filter file names { print "$_\n"; unshift @match, $_; } } defined $match[0] ? $fname = $match[0] : $ftp->quit;` [download] So you list the files, and grab the filenames... Then you download the files... Then you run MD5 on them. On the server side, you can run MD5 on the files on a schedule, or "upon arrival"; then you transfer MD5 file as well and match it to your local result - voila! Hope this helps... -------------------------------- An idea is not responsible for the people who believe in it...	[reply] [d/l]
Re: FTP & MD5 by anonymized user 468275 (Curate) on Jun 21, 2005 at 14:07 UTC
...leaving public key encryption as the only solution I can't eliminate as being too insecure - but maybe others have bright ideas?	[reply]
Re^2: FTP & MD5 by bofh_of_oz (Hermit) on Jun 21, 2005 at 14:12 UTC
Yeah, SFTP would work nice and it is supported by Perl... I wouldn't suggested using FTP and SSL though as I've spent a few weeks trying to figure out how do to that in Perl and so far it doesn't want to work properly... -------------------------------- An idea is not responsible for the people who believe in it...	[reply]
Re^3: FTP & MD5 by kral (Monk) on Jun 22, 2005 at 07:49 UTC
I wouldn't suggested using FTP and SSL though as I've spent a few weeks trying to figure out how do to that in Perl and so far it doesn't want to work properly... Some months ago I had the same problem, but I was not able to get Net::SFTP work. So I wrote my own module. Few days ago I released a module called Net::FTPSSL, which use IO::Socket::SSL and Net::SSLeay::Handle for connections. :D Its methods are similar to Net::FTP, so you can easily convert a script from Net::FTP to Net::FTPSSL. If anyone wants give it a try, I'll be very happy! :D Cheers, Update:I didn't see Eyck post! :D ---------- kral (I apologise for my english!)	[reply]
Re^4: FTP & MD5 by Eyck (Priest) on Jun 22, 2005 at 09:50 UTC
Re^5: FTP & MD5 by kral (Monk) on Jun 22, 2005 at 11:58 UTC
Re^4: FTP & MD5 by bofh_of_oz (Hermit) on Jun 22, 2005 at 13:25 UTC
Re^5: FTP & MD5 by kral (Monk) on Jun 22, 2005 at 14:30 UTC
Re^2: FTP & MD5 by Eyck (Priest) on Jun 22, 2005 at 06:45 UTC
There's no problem with transferring files securely using FTP, look at Net::Lite::FTP and Net::FTPSSL, both support TLS-protected FTP servers (proftpd, wzdftp, pureftpd and few others support this)	[reply]
Re^3: FTP & MD5 by bofh_of_oz (Hermit) on Jun 22, 2005 at 13:41 UTC
I tried your module for transfer, and it gives me a couple of errors... Would you be able to look at those? Here's the code: `my $tlsftp=Net::Lite::FTP->new(); $tlsftp->open($ftphost, 2030); $tlsftp->user("$ftpuser"); $tlsftp->pass("$ftppass"); $tlsftp->cwd("$dir"); my @files=$tlsftp->nlst("$fname"); print "@files\n";` [download] Here's the output: OPEN.Received: 220 ftpssl.dstoutput.ca X2 WS_FTP Server 5.0.4 (3934600 +204) SRV Response: 220 ftpssl.dstoutput.ca X2 WS_FTP Server 5.0.4 (39346002 +04) RECV: 220 ftpssl.dstoutput.ca X2 WS_FTP Server 5.0.4 (3934600204) OPEN..Received: 220 ftpssl.dstoutput.ca X2 WS_FTP Server 5.0.4 (393460 +0204) Received: 234 SSL enabled and waiting for negotiation Prototype mismatch: sub Net::SSLeay::randomize (;$$) vs none at (eval +5) line 1. Sending: PBSZ 0 Prototype mismatch: sub Net::SSLeay::ssl_read_until ($;$$) vs none at +(eval 6) l ine 1. Damn! undefined response Sending: PROT P Damn! undefined response [download] I suspect my SSLeay, but lack knowledge to successfully troubleshoot/fix the problem... More info on this: The script is compiled on WindowsXP, then I run perl2exe on it, then run the .EXE on Windows NT box. perl2exe is supposed to include all the necessary libraries tho... -------------------------------- An idea is not responsible for the people who believe in it...	[reply] [d/l] [select]
Re^4: FTP & MD5 by Eyck (Priest) on Jun 23, 2005 at 07:11 UTC