Fair point about the volume of the data, though I am hoping that logging to a database will help solve that.

At the moment I am logging quite verbosely, as the verbose logs are needed when working out why something crashed, but I am also keeping the log files for about two months, for statistics and reporting. (Which does not need the vebose messages)

My plan with logging to a database, is that I can log verbosely, and then purge all the verbose messages after a couple of days (unless something crashes), to leave behind a much smaller volume of non verbose messages, that will be kept for the standard 2 month window.

This would rely on using a database that will efficently compact and re-use space from deleted records. I have allready had bad experences in that area with MySQL. Postgres or Couch look to be promising.