in reply to Compare two log files line by line containing a keyword

Hello rahu_6697,

I'm not sure I've understood your question, but time will tell..

First of all I think you need some advice on syntax: in order of importance @arr & @arr2

perl -E "@a1=qw(1 2 3); @a2=qw(a b c d e); say for (@a1 & @a2)"

1

perl -E "@a1=qw(1 2 3); @a2=qw(a b c d e); say for (@a1 , @a2)"

1
2
3
a
b
c
d
e

[download]

Infact to concatenate two list in list context the comma is used not & see perlop

Then your way to open filehandle, while running, is oldish and error prone:

open(FILE, "<x.log");
close (FILE);

# has to be

open my $file_handle, '<', 'x.log' or die "unable to open 'x.log' $!";
..
close $file_handle;

[download]

Third, you are reassigning twice to @array2 = grep .. and, as long as you print to the destination file immediately, can also be what your intended, but is not clear and error prone in long distance: grep in list context will returns a list and you can use push to extend the array:


my @array;
...
#push 1

push @array, $_ for grep{..

#push 2

push @array, $_ for grep{..

[download]

Also please use  <code> ..code tags.. </code> also for your data input/output

PS After reading twice, please clearify the following:

> Now I want to compare the data present after the keyword following colon(:) and in output file I have to print the mismatched lines from first data-set on comparing it with second data-set and number of lines in second data set that are not present in first data-set.

If I understand what you are asking for I'd go with a datastructure like a hash of arrays, which keys will be all keywords occurences and array element populed by the evntual presence in file 1 and file 2:

# pseudocode

my %report;

foreach my $line( $fh_one ){
 if... matching..
   extract the part interesting

   $report{  $interesting_part  }[0] = 1;  # first file populates the 
+element 0


foreach my $line( $fh_two ){
 if... matching..
   extract the part interesting

   $report{  $interesting_part  }[1] = 1;  # first file populates the 
+element 1 

   
# result like

%report = (

    interesting1 = [ undef, 1 ], # is present only in the second file
    interesting2 = [ 1, undef ], # is present only in the first file
    interesting3 = [ 1, 1 ],     # is present in both files
);

[download]

L*

There are no rules, there are no thumbs..
Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

Replies are listed 'Best First'.
Re^2: Compare two log files line by line containing a keyword
by rahu_6697 (Novice) on Feb 19, 2019 at 09:45 UTC
    I think I failed to explain my problem. The task is I have two log files (x.log and y.log) I have to search through them line by line and have to compare only those lines which contains a string ("mti_clk_chk") in between.

    lets take an example

    If my data set is like this.

    LOG 1 (x.log) contains

    INFO @1102266 PHResourceLayer_Z4: mti_clk_chk:################ start of test ################ ; T=1102266

    INFO @1102334 PHResourceLayer_Z4: mti_clk_chk:Checking the period of MTI, MTI10 clk from SV; T=1102334


    LOG 2 (y.log) contains

    UVM_INFO @1092507 reporter Z4_COREA: mti_clk_chk: ################ start of test ################ ; T=1092507

    UVM_INFO @1092563 reporter Z4_COREA: mti_clk_chk: Checking the period of MTI, MTI10 clk from SV; T=1092563


    Then for first line I have to check "################ start of test ################ ; T=1102266" and "################ start of test ################ ; T=1092507" as value of T are not same so it should give these details in output file stating the details are not matching.

    Similiarly for line 2 we have to match "Checking the period of MTI, MTI10 clk from SV; T=1102334" and "Checking the period of MTI, MTI10 clk from SV; T=1092563". Here also values of T are not matching so pass it to output file.

    (For each line in the logs, every details after "mti_clk_chk:" needs to be checked )

[reply]

      Hello rahu_6697,

      Based on your updated description something like that should work:

      #!/usr/bin/perl
use strict;
use warnings;

use IO::All;
use Data::Dumper;
use List::Compare;

my @x = io('x.log')->chomp->slurp;
my @x_start = grep /mti_clk_chk/, @x && grep /start of test/, @x;
my @x_period = grep /mti_clk_chk/, @x && grep /Checking the period/, @
+x;
print Dumper \@x_start;
# print Dumper \@x_period;

my @y = io('y.log')->chomp->slurp;
my @y_start = grep /mti_clk_chk/, @x && grep /start of test/, @y;
my @y_period = grep /mti_clk_chk/, @x && grep /Checking the period/, @
+y;
print Dumper \@y_start;
# print Dumper \@y_period;

my $lcStart = List::Compare->new('-u', \@x_start, \@y_start);
my @LorRonlyStart = $lcStart->get_symmetric_difference;
print Dumper \@LorRonlyStart;

io('start.log')->appendln(@LorRonlyStart);

my $lcPeriod = List::Compare->new('-u', \@x_period, \@y_period);
my @LorRonlyPeriod = $lcPeriod->get_symmetric_difference;
print Dumper \@LorRonlyPeriod;

io('period.log')->appendln(@LorRonlyPeriod);

      [download]

      But I am not really sure if this is what you want as from my point of view you are comparing wrong data. I mean the lines that you provide us as sample there are not the same not just by T=<TIME> but also as syntax. For example you are comparing:

      INFO @1102266 PHResourceLayer_Z4: mti_clk_chk:################ start o
+f test ################ ; T=1102266
UVM_INFO @1092507 reporter Z4_COREA: mti_clk_chk: ################ sta
+rt of test ################ ; T=1092507

      [download]

      The lines they do not have the same "@1102266" and "@1092507" so how you are going to be sure you are comparing the correct lines?

      On the sample of code that I provided you above I am grepping lines with two parameters (I assume this should be enough). The problem appears when you are going to compare the arrays. Based on your description the T=<TIME> can be different but from my point of view this is not enough, you might need to add more parameters to be sure you are comparing the correct lines.

      If so update your answer so we can try to help you further.

      Hope this helps, BR

      Seeking for Perl wisdom...on the process of learning...not there...yet!
[reply]
[d/l]
[select]
      Your second example suggests that the log files are the same except for "details". Perhaps you want to examine the first line of each file. If they contain your "keyword", report the difference between them, otherwise do nothing. Repeat for the second line of each file, then the third line of each file, etc. Even if this simplification is true, you still have not explained what kind of differences to expect or how to format them.
      Bill
[reply]

      Do the strings '################ start of test ################' and 'Checking the period of MTI, MTI10 clk from SV' appear only once in the files ?

      poj
[reply]
[d/l]
[select]
        yes in both the files every line is coming just once, I need to compare those lines only
[reply]

In Section Seekers of Perl Wisdom