Need to check CSV file and identify rows containing old end dates and end times

SlapShot has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I'm looking for help please in having a Perl script on a Linux server evaluate CSV output that it generates and mark/identify the rows where the end date (field 4) and end time (field 5) are older than the current local current date/time on the Linux server where the Perl script will be ran.

If it's too difficult to evaluate both the end date and end time together to compare against the current date/time, then as plan "B" at minimum targeting the the end date would be somewhat helpful.

#!/usr/bin/perl

$FILEPATH="/abc/def/logs/servers";
@x=`<application command to get server names> | grep  Name`;
foreach $srv(@x){
@servers=split("=",$srv);
$sname=@servers[1];
#print "$sname\n";
chomp $sname;
$sname =~ s/^\s+//;
if ( open($f1,"<$FILEPATH/$sname.log") ){
   @lines = reverse <$f1>;
$count=1;
foreach $line(@lines){
    @sdata=split(' ',$line);
    $count=$count+1;
        if ($count > 4 or  @sdata[6] =~ /ETA/){
          last;
        }
}

if ( @sdata[1] eq "outage_start"){
$servername=@sdata[0];
$outagesdate=@sdata[4];
$outagestime=@sdata[5];
$outageedate=@sdata[7];
$outageetime=@sdata[8];
print "$servername,$outagesdate,$outagestime,$outageedate,$outageetime
+\n";
}
close $f1;
}
else{
     warn "$sname missing\n";
}
}
[download]

sample CSV output generated by script is below, with real server names replaced. the csv output could contain zero to a couple hundred rows, server name would be unique to each row.

The lines 3, 5, 6, 7, 8 are examples of where the end date and end time are older than the current local Linux server date and time whenever the script is ran.

The CSV format of the date is always YYYY-MM-DD and the format of the time in the CSV ouptut is always HH:MM:00 - I thought it might be important to note that the CSV end time is always going to be 00 seconds in the CSV file in case the perl code for getting the current time to use for comparison also needs to be made to end in 00.

server,start date,start time,end date,end time

aaa.aaa.net,2015-01-07,06:45:00,2015-03-31,19:00:00

bbb.bbb.net,2014-06-27,09:25:00,2015-06-27,09:40:00

ccc.ccc.net,2014-12-01,23:15:00,2014-12-06,07:00:00

ddd.ddd.net,2015-01-31,23:15:00,2015-02-23,07:00:00

eee.eee.net,2015-01-30,23:15:00,2015-02-01,07:00:00

fff.fff.net,2014-11-24,12:15:00,2014-11-25,01:00:00

ggg.ggg.net,2014-10-27,09:25:00,2014-12-15,09:40:00

hhh.hhh.net,2015-01-05,23:15:00,2015-02-01,07:00:00

hhh.hhh.net,2015-01-24,18:15:00,2015-02-24,23:00:00

if it's easier to output the CSV data to an output file, then have another script go through the CSV data, and identify no longer current end date/times, that would also be fine. My goal is to identify somehow the rows containing the old date/times. Thanks

Comment on Need to check CSV file and identify rows containing old end dates and end times Download Code

Replies are listed 'Best First'.
Re: Need to check CSV file and identify rows containing old end dates and end times by blindluke (Hermit) on Feb 04, 2015 at 18:39 UTC
You already have the tough part done, assuming your script works. The only thing you need is to check if the date is located in the past. This can be done using one of the many available time modules - I recommend the DateTime module, as it could end up being your default go-to module for almost anything date/time related. Your dates can be easily parsed by DateTime::Format::HTTP. Read the docs, try improving your existing code, and ask here if you have any problems. - Luke	[reply]
Re^2: Need to check CSV file and identify rows containing old end dates and end times by Anonymous Monk on Feb 05, 2015 at 15:07 UTC
Thanks Luke, appreciate the suggestion, and I'll do that. In the meantime, I was hoping to see some code/script examples to help me solve this quicker. If someone has and scripts that can try and use the example CSV file already posted as input to the script, and show me how you'd use that as input to your own script to print out lines older than the current date (or date and time) that would really be appreciated. Point being, I can alway run the script that I show to produce the CSV file, but I need help (code) to then evaluate that CSV output to report on the rows where the end date (or end date and end time if do-able) is older than current local time script is ran. Thanks much.	[reply]
Re^3: Need to check CSV file and identify rows containing old end dates and end times by blindluke (Hermit) on Feb 05, 2015 at 21:17 UTC
Point being, I can alway run the script that I show to produce the CSV file, but I need help (code) to then evaluate that CSV output It's very hard not to read the above quote as "write me the code I need". This is not the purpose of the Monastery. We're here to help, not to write code for you. If you want to learn, and you hope to see a code example to get you started on your own solution - look at the following code: #!/usr/bin/perl use strict; use warnings; # the following line tells Perl not to warn us against comparing # strings to numbers - we want to use this comparison later, # knowing that Perl treats the string '2015-12-01' in numeric context # just the same as the number 2015 no warnings "numeric"; use Time::Piece; my $time = localtime; my $current_year = $time->year; while (my $line = <DATA>) { my ($server, $start_d, $start_t, $end_d, $end_t) = split /,/, $lin +e; if ( $end_d < $current_year ) { print "#OLD# $server,$start_d,$start_t,$end_d,$end_t"; } else { print "$server,$start_d,$start_t,$end_d,$end_t"; } } __DATA__ aaa.aaa.net,2015-01-07,06:45:00,2015-03-31,19:00:00 bbb.bbb.net,2014-06-27,09:25:00,2015-06-27,09:40:00 ccc.ccc.net,2014-12-01,23:15:00,2014-12-06,07:00:00 ddd.ddd.net,2015-01-31,23:15:00,2015-02-23,07:00:00 eee.eee.net,2015-01-30,23:15:00,2015-02-01,07:00:00 fff.fff.net,2014-11-24,12:15:00,2014-11-25,01:00:00 ggg.ggg.net,2014-10-27,09:25:00,2014-12-15,09:40:00 hhh.hhh.net,2015-01-05,23:15:00,2015-02-01,07:00:00 hhh.hhh.net,2015-01-24,18:15:00,2015-02-24,23:00:00 [download] The output: `aaa.aaa.net,2015-01-07,06:45:00,2015-03-31,19:00:00 bbb.bbb.net,2014-06-27,09:25:00,2015-06-27,09:40:00 #OLD# ccc.ccc.net,2014-12-01,23:15:00,2014-12-06,07:00:00 ddd.ddd.net,2015-01-31,23:15:00,2015-02-23,07:00:00 eee.eee.net,2015-01-30,23:15:00,2015-02-01,07:00:00 #OLD# fff.fff.net,2014-11-24,12:15:00,2014-11-25,01:00:00 #OLD# ggg.ggg.net,2014-10-27,09:25:00,2014-12-15,09:40:00 hhh.hhh.net,2015-01-05,23:15:00,2015-02-01,07:00:00 hhh.hhh.net,2015-01-24,18:15:00,2015-02-24,23:00:00` [download] This is a code that solves a problem similar to yours - it operates on your input and it marks all the lines in which the end year is not current as #OLD#. You should start by making it operate on a file, rather than the DATA section, then focus on comparing the whole date, and not just the year. Some things that will be useful to you, like the Time::Piece module, are already in place. If you are interested in the numeric comparison between '2015-03-31' and the number 2015, and how it works, read the Context tutorial, and you will find the detailed explanation in the 'More flavors of scalars' section. Good luck. - Luke	[reply] [d/l] [select]