With that rate of logging, you are likely to need locks the logger respects to modify the file in place. Given that, a binary search strategy will speed your search for the cutoff time. RecursivelyIteratively estimate the location of the first record you want to keep.

Another approach would be to set up a cron job renaming the log file out of the way at set times. You may have a system utility called logrotate which specializes in that.

Update: Oops :-)

After Compline,
Zaxo