I'd suggest using a binary search on @files to determine when your level change occurred, rather than reading backwards through time by 5 minute increments.

Alternatively, you could start by jumping backwards in time by 1 week intervals, then when you find the level change you could move forward in 1 day intervals, then backwards in 1 hour intervals, then forward in 5 minute intervals.

You could combine a date filter with either approach. Since you know the bot died around Christmas, it seems silly to test the approximately 8000 files between today and Jan 1, 2007 (or those created prior to about Dec 15, 2006). If you narrow your search window to a week or two, I'd be surprised if any method took very long. :-)

In reply to Re: Optimising a search of several thousand files by bobf
in thread Optimising a search of several thousand files by McDarren

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.