Your data format looks like JSON, which you can parse with modules such as JSON.

As for the timestamps, you can use localtime or gmtime to extract the various date/time parts (year, month, day, hour, minute, second) from the timestamp.

G'day spikeinc,

Here's a technique for doing what you want.

I couldn't find any instances of 12:15 in the data you provided; I've used 15:13 as it did appear a few times. You may want localtime instead of the gmtime I've used here (you didn't specify your requirements in this regard). The JSON decoding was straightforward. Check the Time::Piece documentation to see how I've used it and for ways to format the output (if that's what you want).

#!/usr/bin/env perl -l

use strict;
use warnings;

use JSON;
use Time::Piece;

my $wanted_hour = 15;
my $wanted_minute = 13;
my $data = decode_json join '' => map { chomp; $_ } (<DATA>);

for (@{$data->{aaData}}) {
    print "@$_" if is_wanted_time(@$_[0,1]);
}

sub is_wanted_time {
    for (@_) {
        my $t = gmtime $_;
        return 1 if $t->hour == $wanted_hour && $t->min == $wanted_min
+ute;
    }

    return 0;
} 

__DATA__
{"DisplayRecords":"12","Records":"12","sColumns":"startTime,endTime,
remoteNode,srcIP,srcPort,destIP,destPort,egress,ingress","aaData":[["1
+375976271"
,"1375976430","LAN","D0:05:FE","172.20.30.2",1093,"172.20.28.2",1330,"
+1034,348",
"0,0"],["1375976246","1375976404","LAN","D0:05:FE","172.20.30.2",1092,
+"172.20.28
.2",1327,"1034,348","0,0"],["1375976191","1375976381","LAN","D0:05:FE"
+,"172.20.3
0.2",1091,"172.20.28.2",1324,"1034,348","0,0"],["1375976171","13759763
+14","LAN",
"D0:05:FE","172.20.30.2",1090,"172.20.28.2",1321,"1034,348","0,0"],["1
+375976144"
,"1375976302","LAN","D0:05:FE","172.20.30.2",1089,"172.20.28.2",1319,"
+1034,348",
"0,0"],["1375975155","1375976089","LAN","D0:05:FE","172.20.30.2",1085,
+"172.20.28
.2",1278,"1034,348","0,0"],["1375975194","1375975352","LAN","D0:05:FE"
+,"172.20.3
0.2",1086,"172.20.28.2",1281,"1034,348","0,0"],["1375974693","13759748
+35","LAN",
"D0:05:FE","172.20.30.2",1083,"172.20.28.2",1271,"1034,348","0,0"],["1
+375974668"
,"1375974827","LAN","D0:05:FE","172.20.30.2",1081,"172.20.28.2",1268,"
+1034,348",
"0,0"],["1375974691","1375974821","LAN","D0:05:FE","172.20.30.2",1082,
+"172.20.28
.2",1270,"0,0","0,0"],["1375881371","1375881561","LAN","D0:05:FE","172
+.20.30.2",
1055,"172.20.28.2",1122,"1034,348","0,0"],["1375881410","1375881540","
+LAN","E0:0
5:AE","172.20.30.2",1056,"172.20.28.2",1125,"0,0","0,0"]]}
[download]

Output:

$ pm_epoch_from_json.pl
1375974693 1375974835 LAN D0:05:FE 172.20.30.2 1083 172.20.28.2 1271 1
+034,348 0,0
1375974668 1375974827 LAN D0:05:FE 172.20.30.2 1081 172.20.28.2 1268 1
+034,348 0,0
1375974691 1375974821 LAN D0:05:FE 172.20.30.2 1082 172.20.28.2 1270 0
+,0 0,0
[download]

Thankyou Ken for saving my day!!

This is pretty much what I wanted. If I would like to print the output with localtime instead of epoch time (in the output) can I use this function :

 $epochTime=scalar(localtime($Stime));

Also instead of a specific time mentioned, if I want to filter restults for every one hour (say first 15:13, then 16:13... 17:13 and so on until the end of data) how could I do that...

I am so thankful to u as I am learning perl a lot, i m a very new beginner so making a lot of mistakes hence hoping that your help will make me proceed. thankyou so much again

"If I would like to print the output with localtime instead of epoch time (in the output) can I use this function : ..."

Instead of asking if you can use it, just use it. If it does what you want, you've learnt something; if it doesn't do what you want, you've still learnt something; and, if you make a mistake and get a warning or error message, you've learnt even more. It won't melt your keyboard, explode your mouse or cause your screen to burst into flames. :-)

"Also instead of a specific time mentioned, if I want to filter restults for every one hour (say first 15:13, then 16:13... 17:13 and so on until the end of data) how could I do that..."

That should be really obvious from the code I posted. I strongly recommend that, before reading anything else, you study perlintro -- a brief introduction and overview of Perl.

"... I am learning perl a lot, i m a very new beginner so making a lot of mistakes ..."

We all make mistakes; even those who've been writing Perl (or, indeed, any language) for decades still mistakes. Mistakes are good: you'll learn a lot from making mistakes. You'll learn nothing from just copying someone else's code (with all the mistakes already fixed).

As you're clearly very new to Perl, here's some additional documentation links that may further help with the original code I posted (and with any code you write in the future):

• For any functions: Perl functions A-Z.
• For any operators: perlop - Perl operators and precedence.
• For @_ and $_: perlvar - Perl predefined variables.
• For @$_: perlreftut - Mark's very short tutorial about references.
• For @$_[0,1]: perldata - Perl data types (Slices section).
• For all Perl documentation: perl - The Perl 5 language interpreter (all of the above documentation is linked from here; as is pretty much anything else you might need). I recommend you bookmark this link!
• For non-builtin Perl modules (such as JSON): CPAN - Comprehensive Perl Archive Network. I recommend you bookmark this link!

If, after writing some code, you encounter a hurdle you can't resolve yourself, post a question here using the guidelines in "How do I post a question effectively?".

-- Ken

Hi Ken,

Finally I made some changes and then also added code to read the json string from a file and parse it. However, my file is going to contain the same kind of json output appended to it continuously over time.

When I try to decode_json the entire file, it doesnt allow, as I believe decode_json looks for the braces to see if it finished. How can I parse multiline json from a file?

Heres the code I have now:

#!/usr/bin/env perl -l
use strict;
use warnings;
use Time::Local;
use JSON;
use Time::Piece;
use Math::Round;
use File::Read; 
use Text::Table;

my $DestIP="192.168.127.111";
my $jsonc = read_file ('C:\Documents and Settings\SonicWALL User\My Do
+cuments\Scripts\logs\json.txt');
my $data = decode_json join '' => map { chomp; $_ } ($jsonc);
#my $data = decode_json $jsonc;


my $tb = Text::Table->new(
        "Date", "Start Time", "End Time", "Time Taken" , "Egress: Conv
+eyed", "Egress : Sent" , "Ingress: Conveyed" , "Ingress: Sent");
   
 for (@{$data->{aaData}}) {
 my $difftime=(@$_[1] - @$_[0]); 
 my $destip=@$_[6];
 my @egress=split(',', @$_[8]);
 my @ingress=split(',', @$_[9]);
 my $econveyed=$egress[0]/1000000;
 my $esent=$egress[1]/1000000;
 my $iconveyed=$ingress[0]/1000;
 my $isent=$ingress[1]/1000;
    my $startTime=scalar(localtime(@$_[0]));
 my $endTime=scalar(localtime(@$_[1]));
 
 
 #Filter only for required traffic and eliminate any traffic less than
+ 1MB
 
 if ( $destip eq $DestIP && $egress[0]>1000000)
 {
 #print "@$_";
 
 my $date = $startTime->mdy; 
 my $stime1 = $startTime->hms;
 my $etime1 =$endTime->hms;
 $tb->load(
  [ $date, $stime1 , $etime1 ,convert_time($difftime) , nearest(.01,$e
+conveyed) , nearest(.01,$esent), $iconveyed , $isent ],
    );
    }
 
}
print $tb;
[download]

The file input would be something like:

{"DisplayRecords":"12","Records":"12","sColumns":"startTime,endTime,
remoteNode,srcIP,srcPort,destIP,destPort,egress,ingress","aaData":[["1
+375976271"
,"1375976430","LAN","D0:05:FE","172.20.30.2",1093,"172.20.28.2",1330,"
+1034,348"]]}
{"DisplayRecords":"12","Records":"12","sColumns":"startTime,endTime,
remoteNode,srcIP,srcPort,destIP,destPort,egress,ingress","aaData":[["1
+375976271"
,"1375976430","LAN","D0:05:FE","172.20.30.2",1093,"172.20.28.2",1330,"
+1034,348"]]}
.... and so on
[download]

I couldnt figure this part out. Thanks for your help so far.

"When I try to decode_json the entire file, it doesnt allow, as I believe decode_json looks for the braces to see if it finished. How can I parse multiline json from a file?"

With the sample input you posted, you can just read records as being delimited with "}\n" by setting the input record separator: $/ (see perlvar: Variables related to filehandles). You can then remove the embedded "\n" and "\n+" strings with s/\n[+]?//gm (see perlre if you're unfamiliar with that). Here's a modification of my original code that does this. [Note: you haven't supplied data that matches your original 12:15 or subsequent any:20 — I've made an additional change in order to get some output.]

#!/usr/bin/env perl -l

use strict;
use warnings;

use JSON;
use Time::Piece;

my $wanted_minute = 40;

{
    local $/ = "}\n";

    while (<DATA>) {
        s/\n[+]?//gm;

        my $data = decode_json $_;

        for (@{$data->{aaData}}) {
            print "@$_" if is_wanted_time(@$_[0,1]);
        }
    }
}

sub is_wanted_time {
    for (@_) {
        my $t = gmtime $_;
        return 1 if $t->min == $wanted_minute;
    }

    return 0;
}

__DATA__
{"DisplayRecords":"12","Records":"12","sColumns":"startTime,endTime,
remoteNode,srcIP,srcPort,destIP,destPort,egress,ingress","aaData":[["1
+375976271"
,"1375976430","LAN","D0:05:FE","172.20.30.2",1093,"172.20.28.2",1330,"
+1034,348"]]}
{"DisplayRecords":"12","Records":"12","sColumns":"startTime,endTime,
remoteNode,srcIP,srcPort,destIP,destPort,egress,ingress","aaData":[["1
+375976271"
,"1375976430","LAN","D0:05:FE","172.20.30.2",1093,"172.20.28.2",1330,"
+1034,348"]]}
[download]

Output:

$ pm_epoch_from_json_2.pl
1375976271 1375976430 LAN D0:05:FE 172.20.30.2 1093 172.20.28.2 1330 1
+034,348
1375976271 1375976430 LAN D0:05:FE 172.20.30.2 1093 172.20.28.2 1330 1
+034,348
[download]

"The file input would be something like: ..."

I doubt it!

That looks like you've just pasted it from the web page including the leading '+'s indicating text wrapping at 70 characters. Assuming that's right, you only need "s/\n//gm" for the regex.

-- Ken

Seems you want your HH:MM (localtime or GMT?) to match within the given range?

use strict;
use warnings;
use JSON;
use Time::Local;


my ($hh, $mm) = (15, 20);  # 15:20

while ( my $data = <DATA> ) {
  my $struct = from_json( $data );

  if ( my @res = filter_data_hh_mm( $struct, $hh, $mm ) ) {
    # ... do something
    print "Matches: $hh:$mm:  [ " .gmtime( $_->[0] ) . ".." . gmtime( 
+$_->[1] ) . " ]\n" for ( @res );
  }

}


sub filter_data_hh_mm {
  my ($struct, $hh, $mm) = @_;
  my @res;

  for my $line ( @{ $struct->{aaData} } ) {

    # match if begin is before begin-day HH:MM:59 and
    #       if end   is after    end-day HH:MM:00
    if ( $line->[0] <= epoch_at_hhmmss( $line->[0], $hh, $mm , 59)
         and
         epoch_at_hhmmss( $line->[1], $hh, $mm , 0) <= $line->[1]
       ) {
           push @res, $line;
    }
  }

  return @res; # matching lines or ()
}



sub epoch_at_hhmmss {
  my ( $epoch, $hh, $mm, $ss ) = @_;
  #-- use localtime(), timelocal() if $hh:$mm is in localtime
  #   or adjust $hh:$mm to UTC/GMT accordingly
  my  @epoch0 = gmtime( $epoch );
  @epoch0[0..2] = ($ss, $mm, $hh);
  return timegm( @epoch0 );
}

__DATA__
{"DisplayRecords":"12","Records":"12","sColumns":"startTime,endTime,re
+moteNode,srcIP,srcPort,destIP,destPort,egress,ingress","aaData":[["13
+75976271","1375976430","LAN","D0:05:FE","172.20.30.2",1093,"172.20.28
+.2",1330,"1034,348","0,0"],["1375976246","1375976404","LAN","D0:05:FE
+","172.20.30.2",1092,"172.20.28.2",1327,"1034,348","0,0"],["137597619
+1","1375976381","LAN","D0:05:FE","172.20.30.2",1091,"172.20.28.2",132
+4,"1034,348","0,0"],["1375976171","1375976314","LAN","D0:05:FE","172.
+20.30.2",1090,"172.20.28.2",1321,"1034,348","0,0"],["1375976144","137
+5976302","LAN","D0:05:FE","172.20.30.2",1089,"172.20.28.2",1319,"1034
+,348","0,0"],["1375975155","1375976089","LAN","D0:05:FE","172.20.30.2
+",1085,"172.20.28.2",1278,"1034,348","0,0"],["1375975194","1375975352
+","LAN","D0:05:FE","172.20.30.2",1086,"172.20.28.2",1281,"1034,348","
+0,0"],["1375974693","1375974835","LAN","D0:05:FE","172.20.30.2",1083,
+"172.20.28.2",1271,"1034,348","0,0"],["1375974668","1375974827","LAN"
+,"D0:05:FE","172.20.30.2",1081,"172.20.28.2",1268,"1034,348","0,0"],[
+"1375974691","1375974821","LAN","D0:05:FE","172.20.30.2",1082,"172.20
+.28.2",1270,"0,0","0,0"],["1375881371","1375881561","LAN","D0:05:FE",
+"172.20.30.2",1055,"172.20.28.2",1122,"1034,348","0,0"],["1375881410"
+,"1375881540","LAN","E0:05:AE","172.20.30.2",1056,"172.20.28.2",1125,
+"0,0","0,0"]]}
[download]

Result:

Matches: 15:20:  [ Thu Aug  8 15:19:15 2013..Thu Aug  8 15:34:49 2013 
+]
Matches: 15:20:  [ Thu Aug  8 15:19:54 2013..Thu Aug  8 15:22:32 2013 
+]
[download]

This is great too! am combining all your inputs and trying to make it work. thanks so much for all your help.

also Time::Piece provides an object with lots of methods make handling dates easier.