I like graff's benchmark idea. I'd like to point out something that hopefully will happen.... if you run the benchmark the second time, it may speed up!!

I don't know what OS the OP has or how big total file size is, but there is a level of caching that the OS does that Perl doesn't even see.

As an example, I have one app that works with a dataset of 700 text files. The user can use a search window to find various things in this dataset. The first search takes about 7 seconds. The second search is so fast that it is barely perceptible to the user (>10x performance). The reason why things get faster is that my machine has lots of memory that the OS can use for file cache. The OS knows that these files haven't been modified and there is very little disk access going on. I am running on WinXP which isn't exactly famous for performance or "smarts".

The number of things that can affect I/O performance are legion and I have very little information to go on in the OP's case. Anyway in this particular app, I quit optimizing because on user query #8, they have already forgotten that query #1 took a long longer!

In reply to Re^2: How to improve speed of reading big files by Marshall
in thread How to improve speed of reading big files by korlaz

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.