I have two log files and I want to parse certain block of data from both, then compare them and print all unmatched lines in an output file. The required block of data has a fixed starting and ending with particular string. I also want to omit certain lines which have a particular keyword.

In first log file

For first block of data, starting pattern is "################ start of test ################ "

ending pattern is "Testcase completed Successfully"

Remove lines having keyword "PHResourceLayer"

Here is the first log file 

INFO @1101994 PHResourceLayer_Z7b: DUMMY:Hi this is CORE2  ; T=1101994
INFO @1102266 PHResourceLayer_Z4: mti_clk_chk:################ start o
+f test ################ ; T=1092507
INFO @1102334 PHResourceLayer_Z4: mti_clk_chk:Checking the period of M
+TI, MTI10 clk from SV; T=1102334
INFO @1102369 PHResourceLayer_Z4: CPRINT:FLAG_SET:Setting flag 1; T=11
+02369
INFO @1102372 mti_clk_chk:  Checking period of MTI CLk; T=1102372
INFO @1102377 mti_clk_chk: Period value of MTI Clock: 3.125000 ns; T=1
+102377
INFO @1102377 mti_clk_chk: MTI Clock is being generated correctly ; T=
+1102377
INFO @1102377 mti_clk_chk:  Checking period of MTI10 CLk; T=1102377
INFO @1102409 PHResourceLayer_Z4: CPRINT:FLAG WAIT:Waiting flag 2 stat
+us is 0; T=1102409
INFO @1102418 mti_clk_chk: Period value of MTI10 Clock: 31.250000 ns; 
+T=1102418
INFO @1102418 mti_clk_chk: MTI10 Clock is being generated correctly ; 
+T=1102418
INFO @1102418 PHResourceLayer_Z4: FLAG_CLEAR: Clearing flag 1; T=11024
+18
INFO @1102418 PHResourceLayer_Z4: FLAG_SET: Setting flag 2; T=1102418
INFO @1102418 PHResourceLayer_Z4: FLAG_SET: Check flag 1; T=1102418
INFO @1148661 mti_clk_chk: C-Code exit execution. code=<aa>; T=1148661
INFO @1148661 mti_clk_chk: ************************  SV END***********
+********* ; T=1148661
INFO @1148661 testbench.soc_monitors_inst: DMA multi loop check start;
+ T=1148661
INFO @1148661 testbench.soc_monitors_inst.ASSERT_IPD_REQ_PERIPH_DMA_CH
+_MUX_0: Assertion ASSERT_IPD_REQ_PERIPH_DMA_CH_MUX_0 PASSED; T=114866
+1
INFO @1148661 testbench.soc_monitors_inst.ASSERT_IPD_REQ_PERIPH_DMA_CH
+_MUX_1: Assertion ASSERT_IPD_REQ_PERIPH_DMA_CH_MUX_1 PASSED; T=114866
+1
INFO @1148661 mti_lane2_bw_mon: total bytes = 0, at time = 1148661; T=
+1148661
INFO @1148661 mti_lane2_bw_mon: time window = 0, at time = 1148661; T=
+1148661
INFO @1148661 mti_lane2_bw_mon: BW in Mbps = 0, at time = 1148661; T=1
+148661
INFO @1148662 END_TESTCASE: Warns= 0; T=1148662
INFO @1148662 END_TESTCASE: Errors= 0; T=1148662
INFO @1148662 END_TESTCASE: Fatals= 0; T=1148662
INFO @1148662 END_TESTCASE:    TESTCASE PASS    ; T=1148662
INFO @1148662 END_TESTCASE: Testcase completed Successfully; T=1148662

[download]

After parsing from log file sample Block 1 for comparison is : 

INFO @1102372 mti_clk_chk:  Checking period of MTI CLk; T=1102372
INFO @1102377 mti_clk_chk: Period value of MTI Clock: 3.125000 ns; T=1
+102377
INFO @1102377 mti_clk_chk: MTI Clock is being generated correctly ; T=
+1102377
INFO @1102377 mti_clk_chk:  Checking period of MTI10 CLk; T=1102377
INFO @1102418 mti_clk_chk: Period value of MTI10 Clock: 31.250000 ns; 
+T=1102418
INFO @1102418 mti_clk_chk: MTI10 Clock is being generated correctly ; 
+T=1102418
INFO @1148661 mti_clk_chk: C-Code exit execution. code=<aa>; T=1148661
INFO @1148661 mti_clk_chk: ************************  SV END***********
+********* ; T=1148661
INFO @1148661 testbench.soc_monitors_inst: DMA multi loop check start;
+ T=1148661
INFO @1148661 testbench.soc_monitors_inst.ASSERT_IPD_REQ_PERIPH_DMA_CH
+_MUX_0: Assertion ASSERT_IPD_REQ_PERIPH_DMA_CH_MUX_0 PASSED; T=114866
+1
INFO @1148661 testbench.soc_monitors_inst.ASSERT_IPD_REQ_PERIPH_DMA_CH
+_MUX_1: Assertion ASSERT_IPD_REQ_PERIPH_DMA_CH_MUX_1 PASSED; T=114866
+1
INFO @1148661 mti_lane2_bw_mon: total bytes = 0, at time = 1148661; T=
+1148661
INFO @1148661 mti_lane2_bw_mon: time window = 0, at time = 1148661; T=
+1148661
INFO @1148661 mti_lane2_bw_mon: BW in Mbps = 0, at time = 1148661; T=1
+148661
INFO @1148662 END_TESTCASE: Warns= 0; T=1148662
INFO @1148662 END_TESTCASE: Errors= 0; T=1148662
INFO @1148662 END_TESTCASE: Fatals= 0; T=1148662
INFO @1148662 END_TESTCASE:    TESTCASE PASS    ; T=1148662
INFO @1148662 END_TESTCASE: Testcase completed Successfully; T=1148662

[download]

For second block of data, starting pattern is "################ start of test ################ "

ending pattern is "Testcase completed Successfully"

Remove lines having keyword "CORE"

Here is the second log file 

UVM_INFO @1091277 reporter [Z7_COREB]: WR8: A=fc03c002 W=08 R=08; T=10
+91277
UVM_INFO @1091528 reporter [Z7_COREB]: pre_main: Inserting random dela
+y through asm nops, random loop count is  00000009; T=1091528
UVM_INFO @1092507 reporter [Z4_COREA]: mti_clk_chk: ################ s
+tart of test ################ ; T=1092507
UVM_INFO @1092563 reporter [Z4_COREA]: mti_clk_chk: Checking the perio
+d of MTI, MTI10 clk from SV; T=1092563
UVM_INFO @1092598 reporter [Z4_COREA]: C FLAG_SET: Setting flag 1; T=1
+092598
UVM_INFO @1092598 reporter [Z4_COREA]: V FLAG_WAIT: Received flag 1; T
+=1092598
UVM_INFO @1092598 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]:  Checking period of MTI CLk; T=1092598
UVM_INFO @1092605 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipd
+ss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(147) uvm_test
+_top.default_env.default_sequencer[100]@@mti_clk_chk [mti_clk_chk:INF
+O]: Period value of MTI Clock: 3.125000 ns; T=1092605
UVM_INFO @1092605 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]: MTI Clock is being generated correctly ; T=1092605
UVM_INFO @1092605 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]:  Checking period of MTI10 CLk; T=1092605
UVM_INFO @1092634 reporter [Z4_COREA]: C FLAG_WAIT: Checking flag 2, s
+tatus is 0; T=1092634
UVM_INFO @1092655 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipd
+ss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(165) uvm_test
+_top.default_env.default_sequencer[100]@@mti_clk_chk [mti_clk_chk:INF
+O]: Period value of MTI10 Clock: 31.250000 ns; T=1092655
UVM_INFO @1092655 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]: MTI10 Clock is being generated correctly ; T=1092655
UVM_INFO @1092655 reporter [Z4_COREA]: V FLAG_CLEAR: Clearing flag 1; 
+T=1092655
UVM_INFO @1092655 reporter [Z4_COREA]: V FLAG_SET: Setting flag 2; T=1
+092655
UVM_INFO @1092655 reporter [Z4_COREA]: V FLAG_WAIT: Checking flag 3, s
+tatus is 0; T=1092655
UVM_INFO @1092655 reporter [Z4_COREA]: V FLAG_WAIT: Received flag 2; T
+=1092655
UVM_INFO @1092655 reporter [Z4_COREA]: V FLAG_WAIT: Checking flag 2, s
+tatus is 1; T=1092655
UVM_INFO @1092738 reporter [Z4_COREA]: C FLAG_WAIT: Checking flag 2, s
+tatus is 1; T=1092738
UVM_INFO @1092850 reporter [Z4_COREA]: mti_clk_chk:  All clock period 
+Checking done; T=1092850
UVM_INFO @1092886 reporter [Z4_COREA]: C FLAG_SET: Setting flag 3; T=1
+092886
UVM_INFO @1092886 reporter [Z4_COREA]: V FLAG_WAIT: Received flag 3; T
+=1092886
UVM_INFO @1092886 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipd
+ss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(186) uvm_test
+_top.default_env.default_sequencer[100]@@mti_clk_chk [mti_clk_chk:INF
+O]: ************************  SV END******************** ; T=1092886
UVM_INFO @1092886 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]: END OF execute; T=1092886
UVM_INFO @1092886 reporter [Z4_COREA]: V FLAG_WAIT: Received flag 3; T
+=1092886
UVM_INFO @1092886 reporter [Z4_COREA]: V FLAG_WAIT: Checking flag 3, s
+tatus is 1; T=1092886
UVM_INFO @1092906 reporter [Z7_COREB]: DUMMY: Hi this is CORE2  ; T=10
+92906
UVM_INFO @1093261 reporter [Z4_COREA]: W8: A=0100fe00 W=80; T=1093261
[testbench.tb_aux_vip.GET_RANDOM_GEN.genblk1[2].TB_AUX_GET_RANDOM_32] 
+    1
[testbench.top.rru2_top.pmc.A_IP_VREF_CLN16FFC]    11
[testbench.top_level_module.\mti_clk_chk::main ]     7
[testbench.unnamed$$_22]     1
[testbench.unnamed$$_23]     4
[testbench.unnamed$$_26]     3
[uvm_test_top]     2
[uvm_test_top.uvm_fuse_load_manager_inst]   215
UVM_INFO @1130797 END_SIM: Testcases completed Total= 1; T=1130797
UVM_INFO @1130797 END_SIM: Testcases completed Successfully= 1; T=1130
+797
UVM_INFO @1130797 END_SIM: Testcases with Warnings= 0; T=1130797

[download]

After parsing from log file sample Block 2 for comparison is: 

 
UVM_INFO @1092598 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]:  Checking period of MTI CLk; T=1092598
UVM_INFO @1092605 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipd
+ss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(147) uvm_test
+_top.default_env.default_sequencer[100]@@mti_clk_chk [mti_clk_chk:INF
+O]: Period value of MTI Clock: 3.125000 ns; T=1092605
UVM_INFO @1092605 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]: MTI Clock is being generated correctly ; T=1092605
UVM_INFO @1092605 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]:  Checking period of MTI10 CLk; T=1092605
UVM_INFO @1092655 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipd
+ss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(165) uvm_test
+_top.default_env.default_sequencer[100]@@mti_clk_chk [mti_clk_chk:INF
+O]: Period value of MTI10 Clock: 31.250000 ns; T=1092655
UVM_INFO @1092655 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]: MTI10 Clock is being generated correctly ; T=1092655
UVM_INFO @1092886 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipd
+ss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(186) uvm_test
+_top.default_env.default_sequencer[100]@@mti_clk_chk [mti_clk_chk:INF
+O]: ************************  SV END******************** ; T=1092886
UVM_INFO @1092886 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]: END OF execute; T=1092886
[testbench.tb_aux_vip.GET_RANDOM_GEN.genblk1[2].TB_AUX_GET_RANDOM_32] 
+    1
[testbench.top.rru2_top.pmc.A_IP_VREF_CLN16FFC]    11
[testbench.top_level_module.\mti_clk_chk::main ]     7
[testbench.unnamed$$_22]     1
[testbench.unnamed$$_23]     4
[testbench.unnamed$$_26]     3
[uvm_test_top]     2
[uvm_test_top.uvm_fuse_load_manager_inst]   215
UVM_INFO @1130797 END_SIM: Testcases completed Total= 1; T=1130797
UVM_INFO @1130797 END_SIM: Testcases completed Successfully= 1; T=1130
+797

[download]

Now we have to check these extracted two blocks of data for each line. For block 1, compare it with block 2 and print all mismatched lines in output file and similiarly for block 2 also. (Concatenate the results(mismatched lines) from both logs in same output file

This would be the output text file 

Data from LOG1:

INFO @1102372 mti_clk_chk:  Checking period of MTI CLk; T=1102372
INFO @1102377 mti_clk_chk: Period value of MTI Clock: 3.125000 ns; T=1
+102377
INFO @1102377 mti_clk_chk: MTI Clock is being generated correctly ; T=
+1102377
INFO @1102377 mti_clk_chk:  Checking period of MTI10 CLk; T=1102377
INFO @1102418 mti_clk_chk: Period value of MTI10 Clock: 31.250000 ns; 
+T=1102418
INFO @1102418 mti_clk_chk: MTI10 Clock is being generated correctly ; 
+T=1102418
INFO @1148661 mti_clk_chk: C-Code exit execution. code=<aa>; T=1148661
INFO @1148661 mti_clk_chk: ************************  SV END***********
+********* ; T=1148661
INFO @1148661 testbench.soc_monitors_inst: DMA multi loop check start;
+ T=1148661
INFO @1148661 testbench.soc_monitors_inst.ASSERT_IPD_REQ_PERIPH_DMA_CH
+_MUX_0: Assertion ASSERT_IPD_REQ_PERIPH_DMA_CH_MUX_0 PASSED; T=114866
+1
INFO @1148661 testbench.soc_monitors_inst.ASSERT_IPD_REQ_PERIPH_DMA_CH
+_MUX_1: Assertion ASSERT_IPD_REQ_PERIPH_DMA_CH_MUX_1 PASSED; T=114866
+1
INFO @1148661 mti_lane2_bw_mon: total bytes = 0, at time = 1148661; T=
+1148661
INFO @1148661 mti_lane2_bw_mon: time window = 0, at time = 1148661; T=
+1148661
INFO @1148661 mti_lane2_bw_mon: BW in Mbps = 0, at time = 1148661; T=1
+148661
INFO @1148662 END_TESTCASE: Warns= 0; T=1148662
INFO @1148662 END_TESTCASE: Errors= 0; T=1148662
INFO @1148662 END_TESTCASE: Fatals= 0; T=1148662
INFO @1148662 END_TESTCASE:    TESTCASE PASS    ; T=1148662
INFO @1148662 END_TESTCASE: Testcase completed Successfully; T=1148662

Data from LOG 2:

UVM_INFO @1092598 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]:  Checking period of MTI CLk; T=1092598
UVM_INFO @1092605 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipd
+ss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(147) uvm_test
+_top.default_env.default_sequencer[100]@@mti_clk_chk [mti_clk_chk:INF
+O]: Period value of MTI Clock: 3.125000 ns; T=1092605
UVM_INFO @1092605 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]: MTI Clock is being generated correctly ; T=1092605
UVM_INFO @1092605 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]:  Checking period of MTI10 CLk; T=1092605
UVM_INFO @1092655 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipd
+ss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(165) uvm_test
+_top.default_env.default_sequencer[100]@@mti_clk_chk [mti_clk_chk:INF
+O]: Period value of MTI10 Clock: 31.250000 ns; T=1092655
UVM_INFO @1092655 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]: MTI10 Clock is being generated correctly ; T=1092655
UVM_INFO @1092886 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipd
+ss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(186) uvm_test
+_top.default_env.default_sequencer[100]@@mti_clk_chk [mti_clk_chk:INF
+O]: ************************  SV END******************** ; T=1092886
UVM_INFO @1092886 reporter [testbench.top_level_module.\mti_clk_chk::m
+ain ]: END OF execute; T=1092886
[testbench.tb_aux_vip.GET_RANDOM_GEN.genblk1[2].TB_AUX_GET_RANDOM_32] 
+    1
[testbench.top.rru2_top.pmc.A_IP_VREF_CLN16FFC]    11
[testbench.top_level_module.\mti_clk_chk::main ]     7
[testbench.unnamed$$_22]     1
[testbench.unnamed$$_23]     4
[testbench.unnamed$$_26]     3
[uvm_test_top]     2
[uvm_test_top.uvm_fuse_load_manager_inst]   215
UVM_INFO @1130797 END_SIM: Testcases completed Total= 1; T=1130797
UVM_INFO @1130797 END_SIM: Testcases completed Successfully= 1; T=1130
+797

[download]

In reply to Fetching data blocks in between starting and ending keywords and then compare line by line by rahu_6697

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.