Perhaps the Sticky bit on the perl binary (or even the library files - not the directories) might help. It is, however, OS specific.

Update: This would not be my first choice, however, as I also believe that the OS should be able to do a better job, in most cases, of determining what to leave in memory. Setting this could actually be detrimental to overall performance of the machine. Instead, try installing on a ramdisk (Solaris /tmp is an example of this) or using something similar to FastCGI. See also persistant perl. Both of these suggestions have been mentioned elsewhere in the thread.