in reply to Re: Problem using Net::FTP
in thread Problem using Net::FTP

thanks jarich
for the great and very competent support,
but I wrote my problems not so clearly.

the info file looks :
----------------------
AB00001 N120001
BB00002 N200023
----------------------
with info file I should get many times a day
also 2 files: AB00001 and BB00002
all files are TEXT files.
left above is the name of the first file and at the bottom left hand corner the name of the second file.

script should check if the files are complete and names on info file matches to the names of this 2 files.
only if complete, rename first file to N120001 transfer,
wait for 20 minutes, rename the second to N200023 and transfer to the different directory then the first one.
the name of the info file never changed, but names of these 2 files and names on the info file change all the time.
on info file I don't get any paths to change directory on the ftp server.
these paths are fixed:
already renamed first file N120001 should be transfered to:
$ftp->cwd("FTP/A1");
and second file N200023 to:
$ftp->cwd("FTP/B2");
this is, I think my biggest problem at the moment.

when the transfer completed should create a new subdirectory with date and time, something like: 200402241200
and put (backup) all files there, otherwise files will be overwritten each time.

this script will be scheduled every 5 minutes from cron job and waits for the files.

there are very important things:
the second file cannot be transfered without the first one
the second file must be transfered after 20 minutes after the first transfer.
these files cannot be transfered twice and should be there.

and I would like use your script, that you already posted.
if you have time to help by this script, I 'll be very happy
I know, it's a lot of changes, but I want to explain,
what should do this script on the end.

kind regards
cc

Replies are listed 'Best First'.
Re: Re: Re: Problem using Net::FTP
by jarich (Curate) on Mar 09, 2004 at 00:38 UTC
    the info file looks :
    ----------------------
    AB00001 N120001
    BB00002 N200023
    ----------------------
    However, because the code looks like:
    while (<FILE> ) {
    you will process as many files as the info file has lines. So, if the info file were to be replaced with something like this:
    AB00001 N120001 BB00002 N200023 CB00003 N300011 DB00004 N400022
    then your code will attempt to ftp and rename each of those 4 files.

    If you want to limit the number of files processed to always be two and only two then you don't want to use a while loop. Perhaps a for loop would be better:

    # replace the while(<FILE>) { with the following two lines for(my $i = 1; $i <= 2; $i++) { $_ = <FILE>;

    script should check if the files are complete and names on info file matches to the names of this 2 files.
    I have no idea what this means.

    first file N120001 should be transfered to: $ftp->cwd("FTP/A1"); and second file N200023 to: $ftp->cwd("FTP/B2");
    If you can't change what gets put into the info file then my suggestion the other day won't work as well. If you can change it then that would probably make better sense. This way the directory names to ftp things to aren't hard coded in your script and therefore may be easier to change in future.

    Assuming you can't make the changes to the info script then you might want to change your script to look more like:

    # ftp directories my @ftp_locations = ("FTP/A1", "FTP/B2"); # open the file safely or complain it wouldn't open open(FILE, "<", "info") or die "Failed to open info file: $!"; while (<FILE> ) { s/\W*$//; # remove trailing whitespace next if (!$_); # skip empty lines # check that we get our match. If not, # complain and move on. Want to see two # filenames. unless(/^([\w.-]) \s+ ([\w.-])$/x) { print STDERR "$_ is not a valid line"; next; } my ($old, $new) = ($1, $2); rename $old, $new; # Now that we have our filenames, grab an # ftp directory from the list unless(@ftp_locations) { die "Not enough specified ftp_locations for ". "given number of files!"; } my $destination = shift @ftp_locations; # ftp transfer my $server = "X.X.X.X"; my $ftp = Net::FTP->new ($server, Timeout => 9000, Debug => 3) +; $ftp or die "$server: cannot connect: $@"; # If you don't use ~/.netrc $ftp->login ('anonymous', 'mail@address') or die "$_: cannot logon: " . $ftp->message; # change remote directory for the first file $ftp->cwd($destination); # Send file to ftp server $ftp->put($new) or die "$server: cannot put $new: " . $ftp->message; # Sleep for 20 minutes before processing next file. sleep (20 * 60) }
    Note that this still allows you to deal with more than 2 files in the info file, but only if you add additional ftp locations into the array up the top.

    when the transfer completed should create a new subdirectory with date and time, something like: 200402241200 and put (backup) all files there, otherwise files will be overwritten each time.
    How about you give this bit a go and get back to us with how you do at it?

    and I would like use your script, that you already posted.
    Ah, but of course. Certainly, go ahead and use the code I've provided you with. Good luck with the rest.

    Hope this helps,

    jarich

      hi

      thank you very much, it helps a lot.
      I'm also very happy that you explain so clearly.
      You are excellent teacher.
      I have only some little problems and don't know how to solve it.
      the first problem is this line:
      unless(/^([\w.-]) \s+ ([\w.-])$/x) {
      and script doesn't want to process, although file are correct.
      I get this message:
      "AB0000011 LA120040212 is not a valid lineBB0000012 LB20040212 is not a valid lineErrors:"
      info file looks:
      AB0000011 LA120040212 BB0000012 LB20040212
      and I have these 2 files: AB0000011 and BB0000012
      I think, I did not explain the problem so well before.
      it should read the filenames in, compare them to what is in the info file and if they do not match don't process.
      for example: should NOT process with above info file and these files: AB0000015 / BB0000016
      the second problem is, that after transfer doesn't move these 2 files to the backup directory
      this line has an error:
      system("mv $new /var/save/$subfolder_name");
      my third problem is, it sends the mail or moved the files without ftp transfer

      #!/usr/bin/perl -w use strict; use warnings; use File::Copy; use Net::FTP; use Net::Netrc; my $linux = "....."; my $recipient = "....."; my $server = "....."; my $user = "....."; my $password = "....."; # ftp directories my $directory1 = "/ftp/B1"; my $directory2 = "/ftp/B2"; chdir "/var/files" or die "/var/files: $!\n"; -f "/var/files/info" or die "No info NO TRANSFER !\n"; my @ftp_locations = ($directory_1, $directory_2); # open the file safely or complain it wouldn't open open(FILE, "<", "info") or die "Failed to open info file: $!"; for(my $i = 1; $i <= 2; $i++) { $_ = <FILE>; s/\W*$//; # remove trailing whitespace next if (!$_); # skip empty lines # check that we get our match. If not, # complain and move on. Want to see two # filenames. unless(/^([\w.-]) \s+ ([\w.-])$/x) { print STDERR "$_ is not a valid line"; next; } my ($old, $new) = ($1, $2); rename $old, $new; # Now that we have our filenames, grab an # ftp directory from the list unless(@ftp_locations) { die "Not enough specified ftp_locations for ". "given number of files!"; } my $destination = shift @ftp_locations; # ftp transfer my $ftp = Net::FTP->new ($server, Timeout => 9000, Debug => 3) +; $ftp or die "$server: cannot connect: $@"; # If you don't use ~/.netrc $ftp->login ($user,$password) or die "$_: cannot logon: " . $ftp->message; # change remote directory for the first file $ftp->cwd($destination); # Send file to ftp server $ftp->put($new) or die "$server: cannot put $new: " . $ftp->message; #Quit FTP When finished $ftp->quit; # Sleep for 20 minutes before processing next file. sleep (20 * 60) } # send the mail, only when transfer completed open(MAIL, "|/usr/sbin/sendmail -t") || die "Cant send mail. Reason: $ +!"; print MAIL "from:$linux\n"; print MAIL "to:$recipient\n"; print MAIL "subject: file transfer was successfully !\n"; print MAIL "file transfer was successfully ! \n\n"; print MAIL "Time: ", scalar localtime, "\n"; close(MAIL); # when transfer completed, create backup subdirectory and move all fil +e there my @dt = localtime; my $subfolder_name = ((((1900 + $dt[5]) * 100 + 1 + $dt[4]) * 100 + $d +t[3]) * 100 + $dt[2]) * 100 + $dt[1]; mkdir "/var/save/$subfolder_name" or die "$subfolder_name: $!"; system("mv /var/files/info /var/save/$subfolder_name"); system("mv $new /var/save/$subfolder_name"); exit;

      janitored by ybiC: Balanced <code> tags around codeblock

        the first problem is this line:
        unless(/^([\w.-]) \s+ ([\w.-])$/x) {
        and script doesn't want to process, although file are correct.
        Heh. Yup, that one's my fault. Should be:
        unless(/^([\w.-]+) \s+ ([\w.-]+)$/x) {
        notice the extra pluses.

        By the way, you can make your code a whole lot nicer to read by using <code> tags. For example:

        <code> unless(/^([\w.-]) \s+ ([\w.-])$/x) { </code > (without this extra space)
        Code tags mean that you don't have to put <br> tags after ever line. It makes it a tonne easier to read too.

        it should read the filenames in, compare them to what is in the info file and if they do not match don't process. for example: should NOT process with above info file and these files: AB0000015 / BB0000016
        In order to do this you'll need to see if the files in the info file exist. Try something like this:
        unless(/^([\w.-]+) \s+ ([\w.-]+)$/x) { print STDERR "$_ is not a valid line"; next; } my ($old, $new) = ($1, $2); unless( -e $old ) { print STDERR "$old does not exist!\n"; next; } rename $old, $new;
        Note that this won't stop you from processing other files in the info file. If you want to test that first each and every file mentioned in the info file exists and then transfer it you're going to have to completely change your code structure.

        This new code structure would probably have you read in each line in from the file, check that everything's good and push the filenames onto an array. Then, when you know that all the files exist and match what you expect, you'd rename them and transfer them. Of course you could get a similar result by changing those "next"s to "die"s as well.

        the second problem is, that after transfer doesn't move these 2 files to the backup directory this line has an error:
        system("mv $new /var/save/$subfolder_name");
        Oh dear. You're using system. Make sure that you change your shebang line up the top of your script to look like this:
        #!/usr/bin/perl -wT
        notice the capital T. If you don't know about taint checking do a few searches for it. It's very straight forward and should save you lots of grief later.

        So, what's wrong with your line there? Easy. $new doesn't exist outside of your for loop. So, you can either move this line up into your for loop, or your can come up with a different way to remember $new.

        It might be a good idea to create your backup directory before you start processing your files, then move each successfully transferred file into that directory as you go. Then your final act will be to send an email and copy the info file in.

        You might be pleased to know that there's a Perl module that makes moving files a little tidier though:

        #!/usr/bin/perl -wT use strict; use warnings; use File::Copy; ... unless(move("$new", "/var/save/$subfolder_name")) { print STDERR "Oops! Couldn't move the file: $!"; }
        Note that File::Copy still uses system underneath so taint checking is still required.

        my third problem is, it sends the mail or moved the files without ftp transfer
        Yes, indeed it does.

        By the time you get to the sending email/moving the files part of your script there is no indication of whether you transferred your files or not. You do know that at this point (since the script is still running) that nothing disasterous happened, but you don't know if you skipped cases you didn't like (by using next) or if you transferred your files, or if 1 file was transfered and the other wasn't.

        Since I don't know how important it is to you that you successfully transfer both files etc, I can't tell you the exact way to fix things here, but one way to tell if you've transfered the file is to move it after successful transfer (as I suggested above) and then check to see if there's anything in the backup directory at this point.

        Truth to tell, though, I think you probably want to rewrite the logic behind your script altogether to be more like this:

        Create backup directory Open file for each line in info file next if blank get filename and rename from file push filename and rename onto arrays end for if size of arrays is too small (smaller than @ftp_locations) complain that I don't have enough files exit; end if for each item in file array rename file ftp file move file to backup directory end for send message saying everything worked.

        Good luck,

        jarich