Hi Kiat,

we run many installation of a web-based logistics application where there are lots of quick cgi requests with simple-to-complex database access and an average lifetime of 1 or 2 seconds, with peaks of 6/7 seconds, n. of hits (pages or transactions, not counting images, ...) per day is around 25,000. Statistically, we have lower concurrent requests than you mention (10/15 against 100).

We are currently fine using straight CGI with PostgreSQL db backend, though at different stages we seriously considered using mod_perl. Many months ago I did some experiments with Apache::Registry and Apache::PerlRun but the "migration" is not painless.

The most frequent performance bottlenecks in order of importance are:
  1. bad written or sub-optimal database queries
  2. missing or incorrect database indexes
  3. low hard disks throughput and/or hardware misconfigured and/or not very well tuned for performance
  4. serious inefficiencies of perl code in the script

We also have a low-level framework that is highly optimized and uses a very aggressive scheme of caching because the structure is relatively complex: every database table has a linked perl class that is entirely built at runtime. Also, every field in the table is handled internally by linked field class, so you end up with thousands of objects.

The whole process of class "building" at runtime was done for every cgi request and then wasted away, so after heavy profiling (with Devel::Dprof), we decided to store the "building result" as special .pm files, that we can stat, create if they don't exist (this happens only the first time there is a database structure change) and simply eval them, saving 95% of the work that is not database related.

Sorry if I didn't explain very well... The main point here is that if you want to go with cgi, you must plan very aggressive caching.

Another thing to do try changing httpd server. I found out that simply replacing apache 1.3.x with thttpd, you have a little immediate performance boost on number of static and dynamic requests per second.

Of course, YMMV.

In reply to Re: Perl cgi without mod_perl, your experience by cosimo
in thread Perl cgi without mod_perl, your experience by kiat

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.