I concur that the main element of your solution is as noted above:

• On each invocation, unconditionally append to a file all the stuff you will want to send on the next E-Mail update.
• At the end of each invocation, determine if E-Mail needs to be sent (timer expired, special event, Mayan Calendar turns a new page stone, etc.).
• Empty out the E-mail holder file upon sending

This may present a serious design problem. Let's say Event A happens, so your script runs and sends out an email about it, then sets a flag somewhere to say "Don't send any more emails for one hour." Five minutes later, Event B happens. Now people won't get notified about Event B until at least 55 minutes later when the next email is allowed.

That may not be acceptable in a "near-real time detection system"; and if it is acceptable, then it should be acceptable to run the script hourly. Either way, you're going to have notifications up to 1 hour old.

Good thought. The email notification only needs to be rate limited for each event:
• Event A happens, send email.
• Event B happens 5 minutes later, send email on Event B.
• Five minutes later there are 10 more Event A's, do not send email out for these repeated events.
• Event C happens 30 minutes, send email.
• Etc.

In that case, I'd probably have a config file that saves each event type along with a timestamp of the last email sent for that type. That could be done with any module that can save key/value pairs in a file. Then, in pseudo-code:

when there is an event, get the type (A)
if there is a timestamp saved for A
  and if the timestamp is less than 1 hour old
    do nothing
otherwise send an email about A
  and update the timestamp for A with the current time
[download]

Do it like that, and you can run your script as often as you like without getting extra emails.

Aaron B.
Available for small or large Perl jobs and *nix system administration; see my home node.