Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello, Perl Monks..

I've been struggling to find a suitable solution (or direction) in regards to a parsing task I'm working on. The dataset looks similar to the following (without the dashes - those are simply used here to encapsulate each entry as an example: 

--
rn: uid:<user>, <irrelevant-text>
id-info: <URL> | <ID> | <random-string>
creation-time: 1366069064
--
rn: uid:<user>, <irrelevant-text>
id-info: <URL> | <ID> | <random-string>
id-info: <URL> | <ID> | <random-string>
id-info: <URL> | <ID> | <random-string>
creation-time: 1366069064
--
rn: uid:<user>, <irrelevant-text>
id-info: <URL> | <ID> | <random-string>
id-info: <URL> | <ID> | <random-string>
     # random empty line in each entry with 'deletion-time'
deletion-time: 1367949064
creation-time: 1366069064
--

[download]
Now what I need to do is go through each entry and count the number of <ID>'s assigned to it, adding it to a hash I figure, like so: $myHash{totalForEntry}++, eventually ending up so I can print out a table like: IDS TOTAL 1 23 2 536 3 51 4 353 ..etc I'm not exactly sure how I can break down each section and count the total IDs assigned only within that entry. I had headed in the direction of 'paragraph mode' but what quickly threw me off of there was the empty line within each entry containing a 'deletion-time', which throws off the input separator. Here's some very messy code I had started with, lots of nonsense as I was experimenting here and there: 

#!/usr/bin/perl

use DateTime;
use strict;
use warnings;

my $LOGFH = 'testdata';
my $/ = '\n';
my $x=0;
open(LOGFH) or die("Couldn't open 'er!");
while(<LOGFH>) {
   #chomp;
   print $_, "\n";
   $x++;
   last if $x == 2;
}


#   #print $_, "\n";
#   if ($_ =~ m!\|(.*)\|!){
#    $id = $1 . "|";
#        #print $id;
   }
   elsif ($_ =~ m!creationDate:\s+(\d+)!){
        ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localt
+ime($1);
    $dt = DateTime->from_epoch( epoch => $1 );
    print $dt->year;
   }
   #last;
}

[download]

Monks, do you have any pointers for me in regards to which direction I should be taking this to solve the problem? I'm in no way looking for a handout - I like a challenge and have been thinking this over for the past day, but figured a little community input wouldn't hurt! :) Many thanks, VoidWander

Replies are listed 'Best First'.
Re: Parsing multi-line record with varying data
by NetWallah (Canon) on Aug 10, 2013 at 18:06 UTC
    Here is a revision of your code that correctly finds the chunks, and counts ID's.
    Since ID's are the same in your sample data - I leave hashing the found ID's as an exercise. 
    #!/usr/bin/perl

#use DateTime;
use strict;
use warnings;

my $LOGFILE = 'parsechunk.data';
local $/ = "\nrn:";
my $x=0;
open(my $LOGFH,"<",$LOGFILE) or die("Couldn't open '$LOGFILE:$!");
while(<$LOGFH>) {
   #chomp;
   print $_, "==RECORD END++\n";
   my $id_count = () = m/(id-info:)/og;
   print  " Found $id_count ID's in last chunk\n";
}
close $LOGFH;

    [download]
    Update: Added "close $LOGFH;" ... because I'm a nice guy and like to keep the OS happy, and my code clean/complete.

                 My goal ... to kill off the slow brain cells that are holding me back from synergizing my knowledge of vertically integrated mobile platforms in local cloud-based content management system datafication.

[reply]
[d/l]
Re: Parsing multi-line record with varying data
by Laurent_R (Canon) on Aug 10, 2013 at 23:04 UTC

    If you use something like "\nrn" as input record separator (assigning this string to the $/ variable), you will be able to work with each group of lines in paragraph mode. Then, you only need to count the number of occurrences of "id-info" in each record (one way to do it is to use the s/// function on the id-info pattern in scalar context, as it will return the number of substitutions made). I will not elaborate more for the time being since you are not showing real input data nor really detailing the output you need.

    Update 23:06 UTC: Oops, I did not see that's basically the solution proposed by NetWallah, whose code I had not tried to read before posting.

[reply]
Re: Parsing multi-line record with varying data
by Anonymous Monk on Aug 11, 2013 at 01:42 UTC
[reply]

Back to Seekers of Perl Wisdom