Timing concerns on PerlEX/Mod

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Timing concerns on PerlEX/Mod_Perl by BrowserUk (Patriarch) on Jul 25, 2008 at 19:23 UTC
Try: syncing the clocks on your server and client. Then have your script accept a url parameter (say:`?reqtime=12:34:56`. Add this to your request via LWP. Have the script relay this back as a part of the content, and have it add a second (local) timestamp as a part of the content. Have the client-side script add a third timestamp once the request is completed. That should allow you to identify whether the delays are on the outbound leg, during generation, or the return part of the circuit. It won't tell you what caused it, but if you know when they are occuring, it will narrow the possibilities somewhat. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l]
Re^2: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 25, 2008 at 22:46 UTC
Okay here is detailed results: Code Time=0.0715248584747314 seconds Start Code Execution=1217024869.1958 End Code Execution=1217024869.2 LWP Start Time=1217024848.42043 LWP End Time=1217024871.99951 LWP Total Time=23.5790810585022 LWP Start->Code Start=20.77537 seconds Code End->LWP End=2.79951 seconds Head Time=20.0828440189362 seconds Total LWP Time For Google: 1.80555105209351 seconds Margin Of Error For LWP<->Code results: around 2 seconds so by the look of this...it took 20 seconds just to start up the code...since the code though is in PerlEX/Mod_Perl it shouldn't take this long..so either server is taking logn time to start up...or ModPerl and PerlEX are loosing their cache/taking long time to access the cache? Anyone have any ideas on what to do now?	[reply]
Re^3: Timing concerns on PerlEX/Mod_Perl by BrowserUk (Patriarch) on Jul 25, 2008 at 23:10 UTC
Anyone have any ideas on what to do now? I know naff all about PerlEz or Mod_perl, but I would think the next thing to do is determine if it is all scripts on that server, or just this particular script. Ie. Write a simple script that does nothing except produce the timing information as above. If that also displays the same effects, it is a 'server problem'. Contact the hoster and ask for a different setup? If not, then you're down to trying to isolate what it is in that script that causes it. The usual process of commenting everything but the timing code out and then putting it back a bit at a time until the problem manifects itself again seems appropriate. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^4: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 26, 2008 at 03:54 UTC
Re^5: Timing concerns on PerlEX/Mod_Perl by BrowserUk (Patriarch) on Jul 26, 2008 at 06:25 UTC
Some notes below your chosen depth have not been shown here
Re^3: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 26, 2008 at 03:36 UTC
or ModPerl and PerlEX are loosing their cache/taking long time to access the cache? Or you're not using ModPerl/PerlEX at all, and its just plain CGI (or the really really slow one is running as CGI)	[reply]
Re^4: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 26, 2008 at 04:22 UTC
Re: Timing concerns on PerlEX/Mod_Perl by MidLifeXis (Monsignor) on Jul 25, 2008 at 18:31 UTC
How much output are you generating from these scripts? Have you been able to profile the areas where your application is spending its time? What are the logs saying? What type of input is being sent to each? Try putting a `use Time::HiRes; warn "Begin: " . Time::HiRes::time;` [download] at the beginning, and an analogous `warn "End: " . Time::HiRes::time;` [download] at the end of the called portions of your script (if there are blocks that are called each time before and after execution, those would be ideal locations - not knowing what they are, if they exist, I cannot say) will at least give you a feel of how much the perl side is contributing to the delay. There are quite a few points where things other than the Perl script contributes to the delay. Look at the time spent in Apache/IIS, the OS TCP/IP stack, and the client side of things as well. All things work together for the delay of those who write code. --MidLifeXis	[reply] [d/l] [select]
Re^2: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 25, 2008 at 18:51 UTC
Yes I did that already. My script takes 0.3 seconds average beginning to end. Which is fine, but when I LWP remotely and do a check. I get times from 2.5 seconds to 200 seconds(but in both cases script is still 0.3 seconds from beginning to end), also my ping seems same as google or around, so not a latency issue. (google.com LWPs for around 2.5 seconds). So if my code is taking only 0.3 seconds, I assumed it was IIS, so I swapped to Apache/ModPerl and am getting the same issue. I only have one script, which then loads preloaded html files and outputs them.It had a main namespace which has the output, and then a common name space where lal the common functions are stored and called. The output I'd say isn't that big, less then 30kb i'd say. I am not seeing anything strange in error logs	[reply]
Re^2: Timing concerns on PerlEX/Mod_Perl by shmem (Chancellor) on Jul 27, 2008 at 00:49 UTC
All things work together for the delay of those who write code. Great. I stick that next to the map is posted at nearly every major street corner. --shmem _($_=" "x(1<<5)."?\n".q·/)Oo. G°\ / /\_¯/(q / ---------------------------- \__(m.====·.(_("always off the crowd"))."· ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}	[reply] [d/l]
Re: Timing concerns on PerlEX/Mod_Perl by mr_mischief (Monsignor) on Jul 25, 2008 at 19:53 UTC
I'm no expert on these environments, but I do know that a browser will tend to hold the connection until the amount of data mentioned in the content-length header is received if it can. You might check to see if there's some reason for the browser (LWP in this case) to be expecting more data than it's getting, or for the web server to think there's more left to send to the browser.	[reply]
Re: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 25, 2008 at 21:12 UTC
I have added the time-taken to the logs in IIS and it seems it also jumps from 300 - 30,000 milliseconds per request. and there is no difference between the 300 and the 30,000 one other the one takes longer..logs looks something like 5 requests 300, then one 30,000....then 5 more 300. It is not for same IPs either that take the 30,000 and eventually I get like 100, 30,000 requests and then it drops back down :/ so any idea what could be causing this issue?	[reply]
Re^2: Timing concerns on PerlEX/Mod_Perl by mr_mischief (Monsignor) on Jul 26, 2008 at 00:23 UTC
What are the pictures like for the page faults, I/O faults, and resident status of your web server when this happens? Are you sure the web server isn't getting the response from your program quickly but then having to page back in from a thrashing disk?	[reply]
Re^3: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 26, 2008 at 02:13 UTC
Actually if I run 2 instances of the site...one could be slow while other one would be fast(the fast one is one no one accesses but uses the same source)...so if that would be the case, both would be slow? no? and yes I tried running multiple instances and load balancing but that didn't help either.	[reply]
Re^4: Timing concerns on PerlEX/Mod_Perl by MidLifeXis (Monsignor) on Jul 26, 2008 at 14:58 UTC
Re^5: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 26, 2008 at 21:06 UTC
Some notes below your chosen depth have not been shown here
Re^3: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 26, 2008 at 02:06 UTC
As far as i've seen there isn't really any corruption on the disk, or you mean maxed out disks read/write capacity speed and then causing slowdowns?	[reply]
Re^2: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 25, 2008 at 21:30 UTC
here is a sample. Page Beginning to End in perl: 0.506701946258545 seconds Page in IIS Time Taken: 26.063 seconds	[reply]
Re: Timing concerns on PerlEX/Mod_Perl by Burak (Chaplain) on Jul 27, 2008 at 18:07 UTC
Are you using a single print() under PerlEx? If not, try to use a single one (i.e.: buffer all output in a variable and send in one shot). I remember a buffering issue related to PerlIS in 5.8.x back in 2004/5 and it can only be mitigated with a single call to print. Multiple calls were creating some sort of error. And for the apache issue. I think if you did not compile Apache & mod_perl from source, we can't be sure if there is a problem in Apache setup or your code.	[reply]
Re^2: Timing concerns on PerlEX/Mod_Perl by Anonymous Monk on Jul 27, 2008 at 18:18 UTC
I only use single print in general because it is more efficient. As for apache/modperl, yes it was setup correctly since I used the PPM method of setting it up as suggested by modperl for activestate installations. not to mention the apche one works fine with a few users but has issues with more..same as PerlEX is having.	[reply]