My first suggestion is to use somethign like log4perl.
Look at your code and look at the log levels.
- If you die, you should probably fatal before hand.
- If you warn, you should prolly replace that with an error or warn.
- If someone is doing something potentially bad, definitely use warn.
- If you do something that's very important, or against the user's expectations, you can use info.
- If it's "important" debug wise, use debug.
After knowing what to log, it's a matter of preference from tere on out. Anything that isn't debugging information can usually be logged somewhere permanently, as they shouldn't happen too often. But then again, it depends on hwo you use it. Log4perl provides nice facilities to shoot log messages anywhere. And if you have a real time conifguration option, you can then tweak your app as its running, in case something goes awry.
good luck.