The OP doesn't make clear, but I could imagine something where most of the time you get no events, but if you get one you want the email sent right away (hence the check every few minutes) but sometimes things go really haywire and it turns itself into a spammer when the first event would have been sufficient to get the on-call person to fix things (which presumably takes less than an hour in most cases...). If you're the person on the receiving end of the emails, then it probably becomes important to learn enough Perl to fix the spam problem. Pure speculation, however.

That is exactly right. There are times when there are no emails and the first event of the hour is the only email needed. But the times when it gets very noisy, the email "spam" is a problem. Last night as an example we received 23000+ emails, hence the need for thresholding.

For something like that I might even add a feature where the on-call person who does the fix runs a little script that sets a flag to indicate the last time a human responded with a fix, and if a new event comes in after that time, but in less than an hour, it sends the email because either the fix didn't fix it or a new event has showed up to set things off and you want someone to respond right away.

If the process runs every couple of minutes how can you receive over 23000 emails in a single night? It must be sending one email per event. If you sum up all the events each time the script runs and send one email every couple of minutes the most emails you'll get overnight is 15 hours/night * 30 emails/hour = 450 emails/night.

That is a good thought but this is a near-real time detection system and has to be run every couple of minutes. It is just the email that has to be throttled to every hour for bursty periods.

I concur that the main element of your solution is as noted above:

• On each invocation, unconditionally append to a file all the stuff you will want to send on the next E-Mail update.
• At the end of each invocation, determine if E-Mail needs to be sent (timer expired, special event, Mayan Calendar turns a new page stone, etc.).
• Empty out the E-mail holder file upon sending

This may present a serious design problem. Let's say Event A happens, so your script runs and sends out an email about it, then sets a flag somewhere to say "Don't send any more emails for one hour." Five minutes later, Event B happens. Now people won't get notified about Event B until at least 55 minutes later when the next email is allowed.

That may not be acceptable in a "near-real time detection system"; and if it is acceptable, then it should be acceptable to run the script hourly. Either way, you're going to have notifications up to 1 hour old.

Aaron B.
Available for small or large Perl jobs and *nix system administration; see my home node.

Good thought. The email notification only needs to be rate limited for each event:
• Event A happens, send email.
• Event B happens 5 minutes later, send email on Event B.
• Five minutes later there are 10 more Event A's, do not send email out for these repeated events.
• Event C happens 30 minutes, send email.
• Etc.

What is it doing other than sending email? If you want 1 email per hr but something else more frequently: 1) run the notifier on a cron job once per hour and 2) separately run the other more frequently (possibly changing the interval on the fly if load ramps up).