Good questions.

Why not do it at midnight? Freshness. Note the part where I say "late enough that the results are less than 45 minutes old 15 times, so the cache is really fresh."

Many of the sources I read update during the night. Think of a page of online newspaper links, typically updated around 3 am in whatever time zone the newspaper is located. But I'm also mixing in blog feeds, updated on a less predictable schedule. So the goal is to cache as soon as possible before a likely visit.

Simply caching -- with a conventional ttl scheme like you describe -- is, as I explained in my post, not going to cut it. Note I am not dealing with images or other static content that could live happily in such a cache -- or links to other pages, some of which maybe static -- only the text of the Web pages, most of which, again, change every single day.

I appreciate your reply.