To Check Last Upload Date on Amazon S3

babah has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I'm new to Perl.
I try to check Last Upload in perl on Amazon S3 but the script failed to print/recognize last successful upload date, here is the code:

#!/usr/bin/perl

use DateTime;
use Time::Local;

$s3_bucket = 'some/folder/';

$state_failure = 1;
@checkB = &checkBackup;
$checkBT = checkBackup(latest);
print "CHECKBT: $checkBT\n";
$dt = DateTime->now;
$ymdh = $dt->ymd . "-" . $dt->hour;
$ymdh = converttoEpoch($ymdh);
if ($ymdh - $checkB[0] > 24){
        print "ERROR\n";
        $state_failure = 1;
} else {
        print "SUCCESS";
        $state_failure = 0;
}

sub checkBackup {
        local ($req) = @_;
        @last_backups = ();
        @backupList = ();
        @last_backups = `s3cmd --access_key=XXX --secret_key=xxx ls s3
+://$s3_bucket \| awk \'\{print \$1\" \"\$2\}\'`;

        $i = 0;
        foreach $lineB (@last_backups) {
                chomp $lineB;
                if ($i > 0){
                        #print "Before Convert: $lineB\n";
                        $lineBC = converttoEpoch($lineB);
                        #print "After convert: $lineB\n";
                        push (@backupListB, $lineB);
                        push (@backupList, $lineBC);
                }
                $i++;
        }

        @sortedList = sort {$b <=> $a} @backupList;
        @sortedListB = sort {$b <=> $a} @backupListB;
        if ($req eq "latest") {
                return $sortedListB[0];
        }
        else {
                return @sortedList;
        }

}

sub converttoEpoch {
        local ($input) = @_;

        @all_num = $input =~ /(\d+)/g;

        $yearE = $all_num[0];
        $monthE = $all_num[1];
        $dayE = $all_num[2];
        $hourE = $all_num[3];
        $minuteE = 0;
        $secondE = 0;
        if (@all_num){

                $dt = DateTime->new( year   => $yearE, month  => $mont
+hE, day    => $dayE, hour   => $hourE, minute => 0,
                        second => 0, nanosecond => 000000000, time_zon
+e => 'local' );
                $epoch_time  = $dt->epoch;

                $epoch_time = $epoch_time / 3600;

                return $epoch_time;
        }
}

use Mail::Sender;

$sender = new Mail::Sender {
#        smtp => 'some.mail.provider',
        on_errors => 'code',
        debug_level => 1,
} or die "Can't create the Mail::Sender object: $Mail::Sender::Error\n
+";

if ($state_failure == 1){
        if ($sender->MailMsg(
                {
                        to => 'account@some.mail.provider',
                        subject => "NOTICE - No Additional Backup is c
+reated on Amazon S3",
                        msg => "There is no Backup activity more than 
+24 hour\nLast Backup Time: $checkBT\n",
                }
        ) < 0 ) { die "$Mail::Sender::Error\n"; }
}
[download]

Result: I got email and the script print the first upload date (the oldest) file in Amazon S3, not the latest date (newest date).
Please help, thank you very much.

Comment on To Check Last Upload Date on Amazon S3 Download Code

Replies are listed 'Best First'.
Re: To Check Last Upload Date on Amazon S3 by Eily (Monsignor) on Apr 20, 2017 at 09:14 UTC
Hi babah, welcome to perlmonks! To improve your chances of getting the answer you expect, you should reduce your code to a Short, Self-Contained, Correct Example. The mail-sending part seems to be working just fine, but it's not actually necessary to understand what the problem is, so you can remove it and replace it by a few prints. If you're confident that your call to s3cmd works, replace it by a file with a little sample data, so that you can share it with us, that way we can understand what happens rather than guess, and even run the code ourselves. print intermediary values. Especially the content of the two @backupList arrays, this can be done with Data::Dump like this: `use Data::Dump qw(pp); print pp @backupList` (my advice is to always dump your debug output rather than print it, because you'll get extra information, like the difference between undef and the empty string, and invisible characters). You can also do some work on the readability, and syntax of your code. First, you should use strict and warnings, they are cumbersome, but they let perl try its best at finding your mistake, and perl is really good at it. This means that you'll have to declare your variables with my, which has two effects: telling perl what the proper spelling is so that it can catch a typo, and where it can be used (it is limited to the current block). BTW, local is not a local variable, it means that the changes you apply are only temporary. You should also rename your variables to give them more recognizable names, `push (@backupListB, $lineB); push (@backupList, $lineBC);` pretty much looks like twice the same thing. Because your code isn't easy to read, and you didn't provide input data, I can't tell you what the problem is for sure, but `awk '{print $1" "$2}'` does look suspicious. Maybe I don't know awk enough, but as far as I know, `print $1, $2` should print the first two fields separated by a space, which I guess is what your code is trying to do. But actually, there's not much awk can do that perl can't, so you should drop the awk altogether and do all the work in perl, this will add another temporary value (the line before it is processed) to debug, or log when something wrong happens.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re: To Check Last Upload Date on Amazon S3
by Eily (Monsignor) on Apr 20, 2017 at 09:14 UTC

Hi babah, welcome to perlmonks!

To improve your chances of getting the answer you expect, you should reduce your code to a Short, Self-Contained, Correct Example.

The mail-sending part seems to be working just fine, but it's not actually necessary to understand what the problem is, so you can remove it and replace it by a few prints.
If you're confident that your call to s3cmd works, replace it by a file with a little sample data, so that you can share it with us, that way we can understand what happens rather than guess, and even run the code ourselves.
print intermediary values. Especially the content of the two @backupList arrays, this can be done with Data::Dump like this: use Data::Dump qw(pp); print pp @backupList (my advice is to always dump your debug output rather than print it, because you'll get extra information, like the difference between undef and the empty string, and invisible characters).

You can also do some work on the readability, and syntax of your code. First, you should use strict and warnings, they are cumbersome, but they let perl try its best at finding your mistake, and perl is really good at it.
This means that you'll have to declare your variables with my, which has two effects: telling perl what the proper spelling is so that it can catch a typo, and where it can be used (it is limited to the current block). BTW, local is not a local variable, it means that the changes you apply are only temporary.
You should also rename your variables to give them more recognizable names, push (@backupListB, $lineB); push (@backupList, $lineBC); pretty much looks like twice the same thing.

Because your code isn't easy to read, and you didn't provide input data, I can't tell you what the problem is for sure, but awk '{print $1" "$2}' does look suspicious. Maybe I don't know awk enough, but as far as I know, print $1, $2 should print the first two fields separated by a space, which I guess is what your code is trying to do. But actually, there's not much awk can do that perl can't, so you should drop the awk altogether and do all the work in perl, this will add another temporary value (the line before it is processed) to debug, or log when something wrong happens.

[reply]
[d/l]
[select]