in reply to Re: Perl script speed
in thread Perl script speed

Input file format is something like this: 

VERSION {1.0}
PTDEF {instance} {pin} {cell} {edge} {clock_edge} {clock} {phase}
BANNER
  {Module} {xyz}
  {Timing} {EARLY}
  {Slew Propagation} {WORST}
  {Operating Condition} {W_125_0.99_0.99}
  {PVT Mode} {min}
  {Tree Type} {balanced}
  {Process} {3.0000}
  {Voltage} {0.9900}
  {Temperature} {125.0000}
  {time unit} {1.0000 ps}
  {capacitance unit} {1.0000 fF}
  {resistance unit} {1.0000 kOhm}
  {TOOL} {v13.22-s020_1 ((64bit) 11/20/2013 12:28 (Linux 2.6))}
  {DATE} {May 16, 2014}
END_BANNER
PATH 1
  VIEW  func_STRONG_LT_MINC
  CHECK_TYPE {Removal Check}
  REF {xsd_gen/xdiv_sd/DLSB/reg2_reg} {clk}
  ENDPT {xsd_gen/xdiv_sd/DLSB/reg2_reg} {clrz} {dffcqs_f1_fs_dh} {^} {
+leading} {ADPLLS_CLKOUT} {ADPLLS_CLKOUT(C)(P)(func_STRONG_LT_MINC)*}
  BEGINPT {xsd_gen/div_code_u_reg_reg_1_} {q} {sdffpqs_f2_dh_xsvt} {^}
+ {leading} {DIVSD} {DIVSD(D)(P)(func_STRONG_LT_MINC)*}
  REQ_CLC
    {} {Other End Arrival Time} {307.3350}
    {+} {Removal} {61.1000}
    {+} {Phase Shift} {0.0000}
    {-} {CPPR Adjustment} {11.9700}
    {-} {Cycle Adjustment} {0.0000}
    {+} {Uncertainty} {20.0000}
    {=} {Required Time} {376.4650}
  END_REQ_CLC
  SLK_CLC
    {} {Arrival Time} {580.6149}
    {} {Slack Time} {204.1499}
  END_SLK_CLC
  SLK 204.1499
  ARR_CLC
    {} {Clock Rise Edge} {0.0000}
    {+} {Drive Adjustment} {1.3300}
    {=} {Beginpoint Arrival Time} {1.3300}
  END_ARR_CLC
  LAUNCH_CLK_PATH
    COLUMNS {instance} {fpin} {fedge} {tpin} {tedge} {net} {cell} {del
+ay} {incr_delay} {slew} {load} {arrival} {required} {stolen} {fanout}
+ {pin_location} {adjustment}
    PORT {} {adplls_clk} {^} {} {} {adplls_clk} {} {} {} {6.6000} {2.0
+255} {1.3300} {-202.8199} {} {1} {(0.00, 0.00) } {} 
    NET {} {} {} {} {} {adplls_clk} {} {0.4750} {0.0000} {6.6000} {2.0
+255} {1.8050} {-202.3449} {} {} {} {} 
    INST {dft_mux_adplls1_clk/tiboxv_clk_mx2_mux_0} {a} {^} {y} {^} {}
+ {ctmux2_f4} {40.4700} {0.0000} {17.9000} {} {42.2750} {-161.8749} {}
+ {6} {(16.47, 7.88) } {(16.11, 7.58)} {} 
    NET {} {} {} {} {} {mx_fref_adplls_clk} {} {1.1400} {0.0000} {18.1
+000} {17.3511} {43.4150} {-160.7349} {} {} {} {} 
    INST {dft_core_clock_leaker/func_clk_in_gate_dt/tiboxv_clk_icg_icg
+_0} {clkin} {^} {clkout} {^} {} {icg_f4} {26.4100} {0.0000} {7.8000} 
+{} {69.8250} {-134.3249} {} {1} {(66.33, 12.83) } {(65.43, 12.53)} {}
+ 
    NET {} {} {} {} {} {dft_core_clock_leaker/func_clk_in_gate} {} {0.
+0950} {0.0000} {7.8000} {2.4991} {69.9200} {-134.2299} {} {} {} {} 
    INST {dft_core_clock_leaker/func_clk_mux_dt/tiboxv_clk_mx2_mux_0} 
+{a} {^} {y} {^} {} {ctmux2_f4} {43.7950} {0.0000} {21.9000} {} {113.7
+150} {-90.4349} {} {9} {(64.22, 13.72) } {(64.58, 14.03)} {} 
    NET {} {} {} {} {} {cleaker_clk} {} {1.4250} {0.0000} {22.1000} {2
+1.7218} {115.1400} {-89.0099} {} {} {} {} 
    INST {icg_clk_983mhz/tiboxv_clk_icg_icg_0} {clkin} {^} {clkout} {^
+} {} {icg_f4} {85.5000} {0.0000} {103.6000} {} {200.6400} {-3.5099} {
+} {137} {(68.58, 8.78) } {(67.68, 9.08)} {} 
    NET {} {} {} {} {} {clk_2ghz} {} {9.8800} {0.0000} {105.1000} {174
+.7581} {210.5200} {6.3701} {} {} {} {} 
    INST {xsd_gen/xdiv_sd/U0_clkin_gate0/tiboxh_clk_or2_or_0} {a} {^} 
+{y} {^} {} {ctor2_b4} {44.9350} {0.0000} {13.3000} {} {255.4550} {51.
+3051} {} {1} {(48.20, 31.88) } {(47.12, 31.73)} {} 
    NET {} {} {} {} {} {xsd_gen/xdiv_sd/clkin_0_} {} {0.2850} {0.0000}
+ {13.3000} {2.6745} {255.7400} {51.5901} {} {} {} {} 
    INST {xsd_gen/xdiv_sd/U0_clkin0_buf/tiboxh_clk_buf_buf_0} {a} {^} 
+{y} {^} {} {ctb_f4} {21.9450} {0.0000} {9.7000} {} {277.6850} {73.535
+1} {} {5} {(47.07, 30.53) } {(48.33, 30.53)} {} 
    NET {} {} {} {} {} {xsd_gen/xdiv_sd/clkout_gdc_gated} {} {0.3800} 
+{0.0000} {9.7000} {10.3982} {278.0650} {73.9151} {} {} {} {} 
    INST {xsd_gen/xdiv_sd/DLSB/reg4_reg} {clk} {^} {q} {^} {} {dffcqs_
+f1_fs_dh} {94.0500} {0.0000} {51.7000} {} {372.1150} {167.9651} {} {4
+} {(52.61, 40.12) } {(53.19, 41.27)} {DIVSD Adj. = 0.0000} 
    NET {} {} {} {} {} {xsd_gen/xdiv_sd/clkout_gdc} {} {3.8950} {0.000
+0} {51.8000} {11.1143} {376.0100} {171.8600} {} {} {} {} 
    INST {xsd_gen/xdiv_sd/clkout_mux2/tiboxh_clk_mx2_mux_0} {b} {^} {y
+} {^} {} {ctmux2_f4} {64.1250} {0.0000} {34.2000} {} {440.1350} {235.
+9850} {} {8} {(54.36, 46.27) } {(53.28, 45.98)} {} 
    NET {} {} {} {} {} {clkgen_obs_clk} {} {4.1800} {0.0000} {34.5000}
+ {47.5230} {444.3150} {240.1650} {} {} {} {} 
  END_LAUNCH_CLK_PATH 
  DATA_PATH 
    COLUMNS {instance} {fpin} {fedge} {tpin} {tedge} {net} {cell} {del
+ay} {incr_delay} {slew} {load} {arrival} {required} {stolen} {fanout}
+ {pin_location} {adjustment} 
    INST {xsd_gen/div_code_u_reg_reg_1_} {clk} {^} {q} {^} {} {sdffpqs
+_f2_dh_xsvt} {39.3000} {0.0000} {16.0000} {} {483.6150} {279.4650} {}
+ {4} {(59.22, 41.18) } {(60.66, 41.18)} {} 
    NET {} {} {} {} {} {xsd_gen/div_code_u_reg_1_} {} {0.6000} {0.0000
+} {16.0000} {4.8318} {484.2150} {280.0650} {} {} {} {} 
    INST {xsd_gen/xdiv_sd/g397} {a} {^} {y} {v} {} {inv_f1_xsvt} {16.6
+000} {0.0000} {12.5000} {} {500.8150} {296.6650} {} {3} {(56.97, 39.2
+3) } {(57.15, 39.08)} {} 
    NET {} {} {} {} {} {xsd_gen/xdiv_sd/n_16} {} {0.5000} {0.0000} {12
+.5000} {3.5621} {501.3150} {297.1650} {} {} {} {} 
    INST {xsd_gen/xdiv_sd/g384} {b} {v} {y} {^} {} {nand3_f0p33_xsvt} 
+{17.0000} {0.0000} {10.6000} {} {518.3149} {314.1650} {} {1} {(56.72,
+ 37.73) } {(56.84, 37.43)} {} 
    NET {} {} {} {} {} {xsd_gen/xdiv_sd/n_7} {} {0.1000} {0.0000} {10.
+6000} {0.8820} {518.4149} {314.2650} {} {} {} {} 
    INST {xsd_gen/xdiv_sd/g375} {c2} {^} {y} {^} {} {aoa112_f1_xsvt} {
+62.2000} {0.0000} {12.2000} {} {580.6149} {376.4650} {} {1} {(54.86, 
+37.73) } {(53.95, 37.58)} {} 
    NET {} {} {} {} {} {xsd_gen/xdiv_sd/n_25} {} {0.0000} {0.0000} {12
+.2000} {1.2445} {580.6149} {376.4650} {} {} {} {} 
  END_DATA_PATH
  OTHER_ARR_CLC
    {} {Clock Rise Edge} {0.0000}
    {+} {Drive Adjustment} {1.4700}
    {=} {Beginpoint Arrival Time} {1.4700}
  END_OTHER_ARR_CLC
  CAP_CLK_PATH
    COLUMNS {instance} {fpin} {fedge} {tpin} {tedge} {net} {cell} {del
+ay} {incr_delay} {slew} {load} {arrival} {required} {stolen} {fanout}
+ {pin_location} {adjustment}
    PORT {} {adplls_clk} {^} {} {} {adplls_clk} {} {} {} {6.6000} {2.1
+483} {1.4700} {205.6199} {} {1} {(0.00, 0.00) } {} 
    NET {} {} {} {} {} {adplls_clk} {} {0.5250} {0.0000} {6.6000} {2.1
+483} {1.9950} {206.1449} {} {} {} {} 
    INST {dft_mux_adplls1_clk/tiboxv_clk_mx2_mux_0} {a} {^} {y} {^} {}
+ {ctmux2_f4} {44.7300} {0.0000} {17.9000} {} {46.7250} {250.8749} {} 
+{6} {(16.47, 7.88) } {(16.11, 7.58)} {} 
    NET {} {} {} {} {} {mx_fref_adplls_clk} {} {1.2600} {0.0000} {18.1
+000} {17.5437} {47.9850} {252.1349} {} {} {} {} 
    INST {dft_core_clock_leaker/func_clk_in_gate_dt/tiboxv_clk_icg_icg
+_0} {clkin} {^} {clkout} {^} {} {icg_f4} {29.1900} {0.0000} {7.8000} 
+{} {77.1750} {281.3249} {} {1} {(66.33, 12.83) } {(65.43, 12.53)} {} 
    NET {} {} {} {} {} {dft_core_clock_leaker/func_clk_in_gate} {} {0.
+1050} {0.0000} {7.8000} {2.6219} {77.2800} {281.4299} {} {} {} {} 
    INST {dft_core_clock_leaker/func_clk_mux_dt/tiboxv_clk_mx2_mux_0} 
+{a} {^} {y} {^} {} {ctmux2_f4} {48.4050} {0.0000} {21.9000} {} {125.6
+850} {329.8349} {} {9} {(64.22, 13.72) } {(64.58, 14.03)} {} 
    NET {} {} {} {} {} {cleaker_clk} {} {1.5750} {0.0000} {22.1000} {2
+2.3134} {127.2600} {331.4099} {} {} {} {} 
    INST {icg_clk_983mhz/tiboxv_clk_icg_icg_0} {clkin} {^} {clkout} {^
+} {} {icg_f4} {94.5000} {0.0000} {103.7000} {} {221.7600} {425.9099} 
+{} {137} {(68.58, 8.78) } {(67.68, 9.08)} {} 
    NET {} {} {} {} {} {clk_2ghz} {} {10.9200} {0.0000} {105.1000} {18
+2.0893} {232.6800} {436.8299} {} {} {} {} 
    INST {xsd_gen/xdiv_sd/U0_clkin_gate0/tiboxh_clk_or2_or_0} {a} {^} 
+{y} {^} {} {ctor2_b4} {49.6650} {0.0000} {13.3000} {} {282.3450} {486
+.4949} {} {1} {(48.20, 31.88) } {(47.12, 31.73)} {} 
    NET {} {} {} {} {} {xsd_gen/xdiv_sd/clkin_0_} {} {0.3150} {0.0000}
+ {13.3000} {2.8831} {282.6600} {486.8099} {} {} {} {} 
    INST {xsd_gen/xdiv_sd/U0_clkin0_buf/tiboxh_clk_buf_buf_0} {a} {^} 
+{y} {^} {} {ctb_f4} {24.2550} {0.0000} {9.7000} {} {306.9150} {511.06
+49} {} {5} {(47.07, 30.53) } {(48.33, 30.53)} {} 
    NET {} {} {} {} {} {xsd_gen/xdiv_sd/clkout_gdc_gated} {} {0.4200} 
+{0.0000} {9.7000} {10.7594} {307.3350} {511.4849} {} {} {} {} 
  END_CAP_CLK_PATH
END_PATH 1

[download]

Replies are listed 'Best First'.
Re^3: Perl script speed
by LanX (Saint) on May 30, 2014 at 14:13 UTC
    You'd be wise to completely refactor the code and to separate the logic for parsing and processing.

    ATM this is a maintenance nightmare (sorry)

    parser

    This "format" seems to have a strict grammar:

    • lines have <indentation> <capitalized KEYWORD > <datafields>
    • the <datafields> are either space separated or {grouped by curlies}
    • if lines are nested the level is finished by END_keyword

    So write a _generic_ parser (and better implicitly check for correctness) by

    • parsing each line with a regex,
    • keeping record of nesting levels
    • putting parsed data into a hash of hashes

    If this parser works correctly you can start measuring the _speed_ for your huge file.

    Now you are able to tell if the DB operations are the bottleneck (most likely they are)

    processing

    You should try to bundle many DB-operations, cause there is always a overhead.

    If the complete HoH after parsing the whole file is to large, you must define "rules" which hook into the parsing.

    This can be done by checking after each line for suitable callbacks to evaluate your temporary data so far.

    Define a callback-hash $action{KEYWORD} holding subs to be called if a keyword is processed.

    HTH! :)

    Cheers Rolf

    ( addicted to the Perl Programming Language)

[reply]
[d/l]
[select]
Re^3: Perl script speed
by 2teez (Vicar) on May 30, 2014 at 06:32 UTC

    After this I might then have to ask how would the output you are trying to get will look like?
    Please could you see this How do I post a question effectively?

    If you tell me, I'll forget.
    If you show me, I'll remember.
    if you involve me, I'll understand.
    --- Author unknown to me
[reply]

In Section Seekers of Perl Wisdom