Are you using mod-perl? if your lib is properly written, and loaded (and compiled) during the startup of apache, (in startup.pl for example), it would probably be fast enough, and lives only in the main memspace of apache, all it's children can use it, and don't recomile it / and or use holli's suggestion.

For other items, you don't use too often, write them in their own module, and call them using 'require notSoOftenUsedModule;' to compile it as it's needed.