in reply to Re: How to quickly parse a huge web log file?
in thread How to quickly parse a huge web log file?

That's a great point++.

To that end, if the file is sorted, setting $/ = \'the date'; and reading the first 'record' would move the comparison code from Perl into C, and locate the first record much more quickly. Potentially more quickly that coding a binary chop in Perl?

After reading that first record, you just back up the seek position to the start of the line, reset $/ to "\n", and read on using normal line-by-line semantics until the end of the lines for that day and stop.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."