in reply to Re^2: How to manage a big file
in thread How to manage a pattern matching & counting with big data file

while thinking about your last addition,
consider this
open IN,'<1'; while (<IN>) { $a = 1 if /Endpoint/; $b = 1 if /-/ and $a; if($a and $b){ if(/^\s+(.+)\s\(/){ $h{$1}{'x'} = ++$x if ! exists $h{$1}; $r .= $h{$1}{'x'}."."; } } $a = 0, $b = 0, ++$r{$r}, $r = '' if /data arrival time/ and $ +b; } close IN; foreach (sort keys %r) { print "$_: $r{$_}\n"; }
algorithm:
find /Endpoint/
find /-/
assign every line pattern a unique code from 1 towards infinity
form the unique string number, count it as one occurence
find /data arrival time/
which will result in something like this:
1.2.2.3.3.4.4.5.6.7.8.9.10.10.11.11.12.12.: 8
which means I copied your example file eight times in my '1' file
update
regarding your patterns:
they are quite confusing
why did you pick up the middle line to form your pattern?
Please consider this snippet above, maybe it will suit your needs as well?
update 2
have you tried this soultion? has it worked? please tell.

Replies are listed 'Best First'.
Re^4: How to manage a big file
by Anonymous Monk on Apr 23, 2014 at 10:04 UTC
    FWIW, $a and $b are special, its better to use meaningful variable names not already reserved