You could always store the sensitive stuff encrypted with GnuPG. Don't keep a copy of your private key anywhere on the server and it should work well. (providing you keep your key safe and encrypt transmissions with SSL.)
Also burn a copy of your private key and put it in a safe deposit box.