About four years ago while at a certain acronymed telephone company, we implemented a system from scratch using a mixture of Sybase, Perl, and a Servlet front end. At the time, I thought it was really cool. Then I realized how many products were out there that did this better.

The awkwardness of collating all of this data, constructing packets, and writing a daemon to collect these packets led to some complexity that could be avoided by using more mature products.

However, it did have some good things going for it. It used one-way communication using UDP from the monitored servers back to the "reporting" server. No TCP, No database connections, from the production servers to the monitoring station. Perhaps it was limited in functionality, but it was stable and used very low resources.

If I had to reimplement a system from scratch again, I'd probably still use a UDP-like system, but would consider serializing data (YAML possibly) and putting the monitoring process on a crontab rather than running it as a daemon.

Anyhow, long story short, it was a pretty successful project, but we reinvented a lot of wheels and it wasn't the most extensible system when we got through with it. Picking up an off-the-shelf product would have probably been a better idea. At the time, though, I doubt many (free) solutions were ready for prime-time, especially on the obscure platforms we supported.