Other than testing and development, I wouldn't mind spending 10 seconds extra if that would mean some crucial parts would run 50% faster.

You first need to do some major profiling to find out exactly what you need/want to speed up.

No sense trying to optimize away things in perl when your slowdowns are coming from reading from disk or DB or something else along these lines, or re-instantiating objects in loops etc. Profile, then fix the code (cache objects or frequently used data, reduce calls to disk etc).