Two words: Log everything.

Since you can't reproduce the error and see what's happening, you have to keep looking at all times in order to be there when the bug strikes; you need a ridiculously detailed debug log file.

You can probably afford to delete it at regular intervals when things go right, because it's bound to grow fast. gzipping it works as well. A cron entry might be good for this.

You don't want to/can't delete an opened file, so open, print, close at each log entry to prevent perl from locking it.

Just make sure you can log this much without affecting efficiency before you deploy :)

/J

PS. Log files are a Good Thing. Consider using a general logging strategy to keep track of what the program does.