Hi monks, I have a quite terrible problem. I've coded a nice application, the basics are working great, only need to add a few things and it will be done after a great while of programming. I've made it easy for myself and broke up the application in a few ways. I got include files for only 1 or 2 scripts using the same routines and/or values and packages for the scripts which need more functionality and/or the possibility to transport its engine and data towards these scripts.

I got 7 packages; whereof I mostly use 2 or maximally 3, an authentication library, a template library and a file handling library. The rest is more oriented to podcasting etc, which is not used in the main script but which performs also poorly.

I am using ApacheBench as tool to measure the speed of my script and I (should) feel ashamed to post this in public but here goes ;) (this all under Apache 1.3 latest release without mod_perl)

The code is written quite "optimized" to the standards which I know of; but those standards might be completely moot. I cannot post the sources of this application because of diverse reasons (whereof one is the size of it entirely).

How can I find out the culprit? How can I mark points for checking the "current" speed with time? how much time does time take? how should I work on this without rewriting the entire thing? is there any "official" way of finding such problems in code?

**UPDATE**

I've found out a "no packagename" makes no difference at all in a normal running cgi script. Which I wonder is: (1) why would I use this call? (2) would such slow down my scripts? (3) why isn't it unloading? Am I already loading all packages or only from the moment I do "use packagename" ?

A normal request to "printenv" which only contains a few lines (the classic) 

#!/usr/bin/perl

print "Content-type: text/html\n\n";
while (($key, $val) = each %ENV) {
        print "$key = $val<BR>\n";
}

[download]
Document Path:          /printenv.cgi
Document Length:        903 bytes

Concurrency Level:      20
Time taken for tests:   6.514 seconds
Complete requests:      1000
Failed requests:        0
Broken pipe errors:     0
Total transferred:      1045086 bytes
HTML transferred:       904806 bytes
Requests per second:    153.52 #/sec (mean)
Time per request:       130.28 ms (mean)
Time per request:       6.51 ms (mean, across all concurrent requests)
Transfer rate:          160.44 Kbytes/sec received
That's my basic; even if I divide this number by any sane number I do not even come close to what I come with the scripts I have written; so here comes (I do not accept tomatoes as reply :D) 
Document Path:          /podcaster.cgi
Document Length:        2755 bytes

Concurrency Level:      20
Time taken for tests:   486.796 seconds
Complete requests:      1000
Failed requests:        0
Broken pipe errors:     0
Total transferred:      2901000 bytes
HTML transferred:       2755000 bytes
Requests per second:    2.05 #/sec (mean)
Time per request:       9735.92 ms (mean)
Time per request:       486.80 ms (mean, across all concurrent requests)
Transfer rate:          5.96 Kbytes/sec received
This is .. uhm .. 2 requests per second, mean they say; indeed I find it rather cruel...

I can get crueler than that; now you will probably noticing some hallucinations and/or vomiting issues together with a higher heartrate and risk on instant death because of shock; maybe I should warn you to not read further ... 

Document Path:          /photocaster.cgi
Document Length:        41596 bytes

Concurrency Level:      20
Time taken for tests:   119.915 seconds
Complete requests:      200
Failed requests:        0
Broken pipe errors:     0
Total transferred:      8348546 bytes
HTML transferred:       8319200 bytes
Requests per second:    1.67 #/sec (mean)
Time per request:       11991.50 ms (mean)
Time per request:       599.58 ms (mean, across all concurrent requests)
Transfer rate:          69.62 Kbytes/sec received
shocked? .. no? as last but not least, I present to you the uber-worst-statistic you have probably seen in your entire life. 
Document Path:          /main.cgi
Document Length:        12586 bytes

Concurrency Level:      20
Time taken for tests:   235.334 seconds
Complete requests:      200
Failed requests:        0
Broken pipe errors:     0
Total transferred:      2545200 bytes
HTML transferred:       2517200 bytes
Requests per second:    0.85 #/sec (mean)
Time per request:       23533.40 ms (mean)
Time per request:       1176.67 ms (mean, across all concurrent requests)
Transfer rate:          10.82 Kbytes/sec received

Connnection Times (ms)
              min  mean+/-sd median   max
Connect:        0    60  420.6      0  2998
Processing: 18598 22893 1714.3  22948 26722
Waiting:    18597 22893 1714.4  22948 26722
Total:      18598 22953 1768.2  22951 26722

Percentage of the requests served within a certain time (ms)
  50%  22951
  66%  23791
  75%  24201
  80%  24498
  90%  25267
  95%  25864
  98%  26201
  99%  26359
 100%  26722 (last request)
So, does anyone has any advice on this? Where should I start? I can't let speed to be a reason to let me abandon this project...

In reply to Why can code be so slow? by freakingwildchild

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.