The previous monks already gave good advices about using the Perl profiler and Benchmark, so I won't repeat them. After deciding it the program bottleneck is really the configuration file, then you may want to try using Storable and mod_perl/fastCGI to load only once the configuration file into memory. If your configuration file is not big, I can tell you by experience that using Storable will make it slower than parsing everything (but storing it into memory may help). So you should test it and see what is better.

Of course, maybe using such a old Perl version may make this impossible, which I'm not so sure. Despite how difficult is for you to update your Perl version, I think is worth to do it: probably you would gain some more performance using it.