lev36 has asked for the wisdom of the Perl Monks concerning the following question:

O Most Wise Brethren & Sistern, I am a Perl newbie faced with the following task:

I have a file that lists, on each line, a path and filename of a file to be amended on our intranet server. These are all plain text files.

I want to write a script to do the following:

  1. ftp each file in the list to my working folder
  2. save a backup of each file (with a .bak extension) to my working folder
  3. do a substitution (e.g. s|foo|bar|g) to each file
  4. save each updated file in the working folder
  5. and ftp the updated files back to the server

I'm poring through my O'Reilly books right now, but any suggestions would be most appreciated!

I realize that the order listed above might not be the most optimal, and and open to whatever ideas would work best. The main thing is simply to update these files on the server, while saving backups of them just in case, with the constraint that the script won't be on the same server as the original files.

Replies are listed 'Best First'.
Re: FTP and update files from a list
by InfiniteSilence (Curate) on Mar 27, 2006 at 23:05 UTC
    Let's make a deal (because this smells like homework): Show us you can do 2-4 on your own, we'll show you how to FTP some files to/from your machine.

    Celebrate Intellectual Diversity

      It feels like homework, but it's work work.

      Anyway, I figured I would open the list of files and read them into an array. And please don't laugh, but I really am new to this language! I'm sure I'm missing some subtleties in what follows:

      open FILE_LIST, "list.txt"; while(<FILE_LIST>) { #read each line into the array here -- gotta look up the syntax } close FILE_LIST;

      Then, I figured I'd iterate through the array and FTP the files to the working directory, which I don't yet know how to do.

      Next, I'd save off the .bak copies, doing something like this:

      foreach $filename (@files2change) { my $backup = $filename . '.bak'; rename($filename, $backup) open OLD_FILE, $backup; open NEW_FILE, ">>$filename; while(<OLD_FILE>) { print NEW_FILE s/foo/bar/g; } close OLD_FILE; close NEW_FILE; }

      And finally, FTP the updated files back.

        Here's a quick example how you can use ftp to transfer your updated txt files:
        use strict; use warnings; use Net::FTP; my ($ftp); my $host = "xxx.xxx.xxx.xxx"; my $directory = "destination"; my $localDir = "C:\textfiles"; print "Connecting to $host ..."; if ($ftp=Net::FTP->new($host,Timeout=>240)) { print "Successful\n"; print "Logging in ..."; if ($ftp->login("username","password")) { print "Successful\n"; print "Changing Directory to $directory ..."; if ($ftp->cwd($directory)) { print "Successful\n"; print "Transfering Files\n"; &FileSend($localDir); $ftp->quit; } else { print "Failed\n"; } } else { print "Failed\n"; } } else { print "Failed\n"; } ################################################## # Sub Routines ################################################## sub FileSend() { my $dir = shift; if (opendir DIR, $dir) { while (my $file = readdir DIR) { #print "$file\n"; if ($file =~ /txt$/) #finds plain text files { print "Sending htm files $file ... "; if ($ftp->put($file)) { print "Successful\n"; #unlink ($file); # delete file after ftp } else { print "Failed\n"; } } else { next; } } } else { print "Error opening handle\n"; } }
        Cheers

        Scrat
        Well, this line is really wrong:
        print NEW_FILE s/foo/bar/g;
        The "s///" operator will return true or false (1 or 0), and that is what the "print" function will get. Your output file will have a "0" or a "1" for each line of the input file, instead of the actual data.

        Also, you're likely to run this script more than once (since you probably won't get everything right the first time), so you probably want the edited versions of files to be in a different directory (keep the source files and source directory unchanged), and you surely don't want to open the output files with  ">>$filename".

        Since perl gets used for this sort of thing a lot, there are a lot of short-cuts to make it easier. You could do the edit like this:

        #!/usr/bin/perl use strict; # suppose original files are in directory "source", and # we want to put edited versions into directory "edited": my @files = <source/*>; # that's called a "file glob" for my $ifile ( @files ) { my $ofile = $ifile; $ofile =~ s/source/edited/; open( I, "<", $ifile ); open( O, ">", $ofile ) or die "$ofile: $!"; while (<I>) { s/foo/bar/g; print O; } close I; close O; }
        As for the FTP part, if the remote machine is running an anonymous-ftp server, and the files in question are available that way, then Net::FTP is fine. But if you need to use a particular (non-anonymous) user account and a password, you really should use Net::SFTP.

        I would recommand that you keep the (S)FTP stuff in a separate script from the editing stuff. (It's actually more likely that you don't need a perl script at all to do the file transfers between machines, but if you want to make up a perl script to "reinvent" the existing (s)ftp programs, go ahead.)

        The point is that you can write a re-usable file-transfer script that will be handy for lots of occasions (assuming the standard tools are less handy), and you can even write a re-usable editing script for making changes to a list of files, which will be handy for lots of occasions (it is possible to provide a list of regex substitutions as an input). Both of these things are relatively easy.

        But if a script is going to do both file transfers and editing, that one script will be larger and more complicated and will take longer to write; and making it re-usable in any practical sense is going to be a lot more difficult.

      It's mainly the FTP part of the script I don't have a clue about. I'll post my meagre lines of script in the morning to (hopefully) prove my worthines...

Re: FTP and update files from a list
by lev36 (Sexton) on Mar 28, 2006 at 16:04 UTC
    Thanks, all, that gives me some great ideas to run with. I'll look into Net::SFTP, since this can't be done with an anonymous FTP session. And thanks for pointing out the lines that are 'just plain wrong' (as I knew there would be) - that's one of the reasons I'm here Seeking Wisdom!