in reply to Re: How to quickly parse a huge web log file?
in thread How to quickly parse a huge web log file?
That's a great point++.
To that end, if the file is sorted, setting $/ = \'the date'; and reading the first 'record' would move the comparison code from Perl into C, and locate the first record much more quickly. Potentially more quickly that coding a binary chop in Perl?
After reading that first record, you just back up the seek position to the start of the line, reset $/ to "\n", and read on using normal line-by-line semantics until the end of the lines for that day and stop.
|
|---|