in reply to Re: Re: Re: Re: Problem using Net::FTP
in thread Problem using Net::FTP

the first problem is this line: 
unless(/^([\w.-]) \s+ ([\w.-])$/x) {

[download]
and script doesn't want to process, although file are correct.
Heh. Yup, that one's my fault. Should be: 
unless(/^([\w.-]+) \s+ ([\w.-]+)$/x) {

[download]
notice the extra pluses.

By the way, you can make your code a whole lot nicer to read by using <code> tags. For example: 

<code>
unless(/^([\w.-]) \s+ ([\w.-])$/x) {
</code >    (without this extra space)

[download]
Code tags mean that you don't have to put <br> tags after ever line. It makes it a tonne easier to read too.

it should read the filenames in, compare them to what is in the info file and if they do not match don't process. for example: should NOT process with above info file and these files: AB0000015 / BB0000016
In order to do this you'll need to see if the files in the info file exist. Try something like this: 
        unless(/^([\w.-]+) \s+ ([\w.-]+)$/x) {
                print STDERR "$_ is not a valid line";
                next;
        } 

        my ($old, $new) = ($1, $2);
        unless( -e $old ) {
                print STDERR "$old does not exist!\n";
                next;
        }
        rename $old, $new;

[download]
Note that this won't stop you from processing other files in the info file. If you want to test that first each and every file mentioned in the info file exists and then transfer it you're going to have to completely change your code structure.

This new code structure would probably have you read in each line in from the file, check that everything's good and push the filenames onto an array. Then, when you know that all the files exist and match what you expect, you'd rename them and transfer them. Of course you could get a similar result by changing those "next"s to "die"s as well.

the second problem is, that after transfer doesn't move these 2 files to the backup directory this line has an error: 
system("mv $new /var/save/$subfolder_name");

[download]
Oh dear. You're using system. Make sure that you change your shebang line up the top of your script to look like this: 
#!/usr/bin/perl -wT

[download]
notice the capital T. If you don't know about taint checking do a few searches for it. It's very straight forward and should save you lots of grief later.

So, what's wrong with your line there? Easy. $new doesn't exist outside of your for loop. So, you can either move this line up into your for loop, or your can come up with a different way to remember $new.

It might be a good idea to create your backup directory before you start processing your files, then move each successfully transferred file into that directory as you go. Then your final act will be to send an email and copy the info file in.

You might be pleased to know that there's a Perl module that makes moving files a little tidier though: 

#!/usr/bin/perl -wT
use strict;
use warnings;
use File::Copy;

...

unless(move("$new", "/var/save/$subfolder_name")) {
       print STDERR "Oops!  Couldn't move the file: $!";
}

[download]
Note that File::Copy still uses system underneath so taint checking is still required.

my third problem is, it sends the mail or moved the files without ftp transfer
Yes, indeed it does.

By the time you get to the sending email/moving the files part of your script there is no indication of whether you transferred your files or not. You do know that at this point (since the script is still running) that nothing disasterous happened, but you don't know if you skipped cases you didn't like (by using next) or if you transferred your files, or if 1 file was transfered and the other wasn't.

Since I don't know how important it is to you that you successfully transfer both files etc, I can't tell you the exact way to fix things here, but one way to tell if you've transfered the file is to move it after successful transfer (as I suggested above) and then check to see if there's anything in the backup directory at this point.

Truth to tell, though, I think you probably want to rewrite the logic behind your script altogether to be more like this: 

Create backup directory

Open file

for each line in info file
     next if blank
     get filename and rename from file 
     push filename and rename onto arrays
end for

if size of arrays is too small (smaller than @ftp_locations)
     complain that I don't have enough files
     exit;
end if

for each item in file array
     rename file
     ftp file
     move file to backup directory
end for

send message saying everything worked.

[download]

Good luck,

jarich

Replies are listed 'Best First'.
Re: Re: Re: Re: Re: Re: Problem using Net::FTP
by cc (Beadle) on Mar 14, 2004 at 02:11 UTC
    hi jarich

    thanks again !
    your code is all the time perfectly.

    I think, for me it's better to create backup directory
    later,because this script will be scheduled
    every 5 minutes and after I will have a lot of backup directories.

    something like:
    Open file
    for each line in info file
    next if blank
    get filename and rename from file
    push filename and rename onto arrays
    end for
    if size of arrays is too small (smaller than @ftp_locations)
    complain that I don't have enough files
    exit;
    end if
    for each item in file array
    rename file
    Create backup directory
    change the path
    ftp file
    move file to backup directory
    end for
    send message saying everything worked.

    I tried something, but it doesn't work correctly:
    ..............
chdir "/var/files" or die "/var/files: $!\n";
-f "/var/files/info" or die "No info NO TRANSFER !\n";

my @ftp_locations = ($directory_1, $directory_2);
       
# open the file safely or complain it wouldn't open
open(FILE, "<", "info") or die "Failed to open info file: $!";

for(my $i = 1; $i <= 2; $i++) {
        $_ = <FILE>;

        s/\W*$//;            # remove trailing whitespace
        next if (!$_);       # skip empty lines
       
              # check that we get our match.  If not,
              # complain and move on.  Want to see two
              # filenames.
       unless(/^([\w.-]+) \s+ ([\w.-]+)$/x) {
                print STDERR "$_ is not a valid line";
                next;
        } 

        my ($old, $new) = ($1, $2);
        unless( -e $old ) {
                print STDERR "$old does not exist!\n";
                next;
        }
        rename $old, $new; 


              # Now that we have our filenames, grab an
              # ftp directory from the list
        unless(@ftp_locations) {
              die "Not enough specified ftp_locations for ".
                  "given number of files!";
        }

        my $destination = shift @ftp_locations;

# create backup subfolder 
my @dt = localtime;
my $subfolder_name = ((((1900 + $dt[5]) * 100 + 1 + $dt[4]) * 100 + $d
+t[3]) * 100 + $dt[2]) * 100 + $dt[1];
mkdir "/var/files/out/$subfolder_name" or die "$subfolder_name: $!";

unless(move("$new", "/var/files/out/$subfolder_name")) {
       print STDERR "Oops!  Couldn't move the file: $!";
}

chdir "/var/files/out/$subfolder_name" or die "/var/files/out/$subfold
+er_name: $!\n";
        
# ftp transfer
        my $ftp = Net::FTP->new ($server, Timeout => 9000, Debug => 3)
+;
        $ftp or die "$server: cannot connect: $@";
        # If you don't use ~/.netrc
        $ftp->login ($user,$password)
            or die "$_: cannot logon: " . $ftp->message;

        # change remote directory for the first file
        $ftp->cwd($destination);

        # Send file to ftp server
        $ftp->put($new) or
        die "$server: cannot put $new: " . $ftp->message;
        #Quit FTP When finished
        $ftp->quit;

        # Sleep for 20 minutes before processing next file.
        sleep (20 * 60)
}

    [download]
    the first file will be renamed, send and backuped correctly,
    but it won't rename or send the second file.
    do you know what's wrong, if you have a little time ?
    greetings
    cc
[reply]
[d/l]
      The problem is that you haven't actually read what I've suggested. In the pseudocode, that you've written (cut'n'pasted), you parrot back to me the idea of using two separate loops and of pushing the filename onto an array but you've kept your single loop structure.

      Of course it's not going to work.

      The specific problem of what is wrong is (because you're doing all of this in a single loop) when you chdir into your backup directory. Because you never chdir out again, you never find the second file and so your program halts there.

      You should be getting an error about that, but you haven't included that in your message. :(

      Anyway, the code that follows is probably what you're looking for. Compare it with the pseudocode I suggested to you. See how we have 2 loops and a check between them and all the things I suggested.

      Notice too, that I've changed your backup directory name. If you don't like my suggestion then feel free to change it back again. 

      #!/usr/bin/perl -wT

use strict;
use File::Copy;

...

chdir "/var/files" or die "/var/files: $!\n";
-f "/var/files/info" or die "No info NO TRANSFER !\n";

my @ftp_locations = ($directory_1, $directory_2);

# open the file safely or complain it wouldn't open
open(FILE, "<", "info") or die "Failed to open info file: $!";

# Read all the lines in from info
my @files;
for(my $i = 1; $i <= @ftp_locations; $i++) {
        $_ = <FILE>;

        s/\W*$//;            # remove trailing whitespace
        next if (!$_);       # skip empty lines

              # check that we get our match.  If not,
              # complain and move on.  Want to see two
              # filenames.
       unless(/^([\w.-]+) \s+ ([\w.-]+)$/x) {
                print STDERR "$_ is not a valid line";
                next;
        }

        # rename the files as per rename in file
        my ($old, $new) = ($1, $2);
        unless( -e $old ) {
                print STDERR "$old does not exist!\n";
                next;
        }
        rename $old, $new;

        push @files, $new;
}

# check that we have the number of files that we expect
unless(@files == @ftp_locations) {
      die "Not enough specified ftp_locations for ".
          "given number of files!";
}

# create backup subfolder
my @dt = localtime;
my $subfoldername = ($dt[5]+1900) . "-" . ($dt[4]+1) .
                    "-$dt[3]:$dt[2]:$dt[1]_$$";

mkdir "/var/files/out/$subfolder_name" or die "$subfolder_name: $!";

# FTP each file across, die on errors
foreach my $new (@files) {
       my $destination = shift @ftp_locations;

        # ftp transfer
        my $ftp = Net::FTP->new ($server, Timeout => 9000, Debug => 3)
+;

        $ftp or die "$server: cannot connect: $@";
        # If you don't use ~/.netrc
        $ftp->login ($user,$password)
            or die "$_: cannot logon: " . $ftp->message;

        # change remote directory for the first file
        $ftp->cwd($destination);

        # Send file to ftp server
        $ftp->put($new) or
        die "$server: cannot put $new: " . $ftp->message;
        #Quit FTP When finished
        $ftp->quit;

        # move file to backup directory
        unless(move("$new", "/var/files/out/$subfolder_name")) {
               print STDERR "Oops!  Couldn't move the file: $!";
        }
        # Sleep for 20 minutes before processing next file.
        sleep (20 * 60)
}

# If we get to this point then we've renamed, ftped and moved the file
+s
# so send an email saying so...

      [download]
      Good luck with this.

      jarich

[reply]
[d/l]
        hi jarich

        thank you very much for your time and HELP
        now it works perfectly !
        and very sorry, I didn't answer so quickly, because I was in vacation

        I have only a small problem:
        I try to log the whole process and put: 
        # log the whole process
BEGIN
{
use CGI::Carp qw(carpout);
my $errorlog = "/var/logs/log.txt";
open(LOG, ">$errorlog") or die("Unable to open $errorlog: $!\n");
print LOG "Errors:\n";
carpout(*LOG);
}

        [download]
        on the top of the script.

        It works and I get the log.
        On the end I try to move all files: 2 data files, info file and log.txt to the backup directory: 
        ............................................................
# move the file to the backup subfolder

unless(move("$new", "/var/files/out/$subfolder_name"))     
{
               print STDERR "Oops!  Couldn't move the file: $!";
}

system("mv /var/files/info /var/files/out/$subfolder_name");

system("mv /var/logs/log.txt /var/files/out/$subfolder_name");

............................................................

        [download]
        it works and all files are moved, but I get in the log following error messages :
        mv: cannot stat `/var/files/info': No such file or directory
        mv: cannot stat `/var/logs/log.txt': No such file or directory

        I don't know what's wrong and why I get these error messages.

        greetings
        cc
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom