printing loops on the fly

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

There is a problem I've run into many times before and now I'm trying to see if there is a solution to it. I have a chat log parser which takes about 100 different html pages, parses them and prints them. This is inside of a large foreach loop under CGI. It usually takes a good 5 minutes before it prints anything and when it does, it prints about half of it..then lags..then prints the rest.

Is there a way to make it print each new line INSTANTLY when it's down loading? Otherwise it looks like the script freezes.

Comment on printing loops on the fly

Replies are listed 'Best First'.
Re: printing loops on the fly by jonnyfolk (Vicar) on Mar 05, 2004 at 17:05 UTC
Suffering from Buffering would probably be an interesting read for you about now...	[reply]
Re: printing loops on the fly by Willard B. Trophy (Hermit) on Mar 05, 2004 at 16:45 UTC
Have you got `$\|=1;` in your program? This unbuffers STDOUT. Please see perlvar for more details. -- bowling trophy thieves, die!	[reply] [d/l]
Re: printing loops on the fly by ambrus (Abbot) on Mar 05, 2004 at 17:47 UTC
If the page has tables, that might be the problem. Most browses don't display tables until the table is finished. Make sure the page does not have table tag that contains lot of data. Update: Also make sure that the list in parenthesis after the foreach does not take long to calculate. If it does, transform it to some iterating form. For example, change `for $line (<FILE>) {...}` to `while ($line=<FILE>) {...}`; etc.	[reply] [d/l] [select]
Re: printing loops on the fly by stvn (Monsignor) on Mar 05, 2004 at 19:04 UTC
ambrus has a point here. Your CGI may not be the problem as much as your browser. It is common for large HTML tables/pages to take a very long time to render no matter how beefy your computer is. I would open up your ActivityMonitor/TaskManager/terminal-running-top and watch the CPU usage on your browser, and do the same on your server CGI process. Watch them both, and see if that gives you any clues as to where the time is being consumed. -stvn	[reply]