You really need to instrument your handlers and get a good idea of what's happening during a request. Get the mod_perl Developer's Cookbook (ISBN 0672322404), which has several recipies on using Devel::Dprof and Time::HiRes for gathering statistics. There's also sugguestions on tuning the number of httpd processes and what not.