Hi Kiat,

we run many installation of a web-based logistics application where there are lots of quick cgi requests with simple-to-complex database access and an average lifetime of 1 or 2 seconds, with peaks of 6/7 seconds, n. of hits (pages or transactions, not counting images, ...) per day is around 25,000. Statistically, we have lower concurrent requests than you mention (10/15 against 100).

We are currently fine using straight CGI with PostgreSQL db backend, though at different stages we seriously considered using mod_perl. Many months ago I did some experiments with Apache::Registry and Apache::PerlRun but the "migration" is not painless.

The most frequent performance bottlenecks in order of importance are:

1. bad written or sub-optimal database queries
2. missing or incorrect database indexes
3. low hard disks throughput and/or hardware misconfigured and/or not very well tuned for performance
4. serious inefficiencies of perl code in the script

We also have a low-level framework that is highly optimized and uses a very aggressive scheme of caching because the structure is relatively complex: every database table has a linked perl class that is entirely built at runtime. Also, every field in the table is handled internally by linked field class, so you end up with thousands of objects.

The whole process of class "building" at runtime was done for every cgi request and then wasted away, so after heavy profiling (with Devel::Dprof), we decided to store the "building result" as special .pm files, that we can stat, create if they don't exist (this happens only the first time there is a database structure change) and simply eval them, saving 95% of the work that is not database related.

Sorry if I didn't explain very well... The main point here is that if you want to go with cgi, you must plan very aggressive caching.

Another thing to do try changing httpd server. I found out that simply replacing apache 1.3.x with thttpd, you have a little immediate performance boost on number of static and dynamic requests per second.

Of course, YMMV.