mt2k has asked for the wisdom of the Perl Monks concerning the following question:

Say I want to create a web script that continues going and going (just like the evil Energizer Bunny). I want it to keep running, so that I can continue to read in a database file and printing out contents of that file, in one stream of output. Part of the script might look somthing like this:

#!/usr/bin/perl

$| = 1;
print qq|...|; #PRINT HEADERS, ETC.

while (1) {

#CODE THAT READS
#IN DATABASE STUFF
#AND OUTPUTS IT TO STDOUT

sleep 5;
}

[download]
Will the OS quit the script eventually. If it does time out, is there a way to stop that from happening?

Replies are listed 'Best First'.
Re: Running Perl Continuously
by dws (Chancellor) on Dec 15, 2001 at 00:05 UTC
    Say I want to create a web script that continues going and going ... I want it to keep running, so that I can continue to read in a database file and printing out contents of that file, ...

    First, buy a big UPS. :)

    The problem you are liable to run into is connectivity to the client browser. There are all sorts of reasons that the socket can break, including death of the browser, and these you can do nothing about, except be prepared to gracefully handle the error on your end.

    It's also possible to emit an HTML page that refreshes itself every several seconds, re-invoking the CGI each time. This would leave you with less server-side error handling, though you might need to do something to cache a database handle. Tricks for doing that are available through mod_perl.

[reply]
Re: Running Perl Continuously
by dws (Chancellor) on Dec 15, 2001 at 08:42 UTC
    Erudil has an article in Linux Journal that mentions using NPH (non-parsed headers) to push 100's of megabytes of data to browsers over multi-week experiments. Look about 3/4ths of the way down the article.

[reply]
Re: Running Perl Continuously
by jerrygarciuh (Curate) on Dec 15, 2001 at 00:24 UTC
    Sounds like the sort of thing usually done with an xml socket pusing the updated info from the db to the browser like stock ticker scripts etc.
    hth
    jg
    _____________________________________________________
    If it gets a little bit out of hand sometimes, don't let it fool you into thinkin' you don't care.TvZ
[reply]
[d/l]
Re: Running Perl Continuously
by mt2k (Hermit) on Dec 15, 2001 at 01:12 UTC
    My point to not wanting to refresh the HTML page is to keep from having a new process started every 5 seconds by every user. Those add up fast. However, I also understand that keeping one process going (and opening/closing the database every few seconds) for every user adds up to a lot of system resources.

    Another reason for wanting to do this is so that the user doesn't get any flashing on their screen from the script reloading all the time. Smooth, continuous output looks pretty cool.

    Any ideas or comments on this?

[reply]
      Yeah, I would shy away from trying to server massive amounts of data thru your webserver - the size of the responses will bog your server down.

      Instead - if it's really a -massive- dataset - and you don't need to be serving it up all that fast, you just need to make sure you serve it -all- up when it's requested - how about using email?

      It's pretty reliable and can transmit a ton of data without causing any wierd hiccups. It's also fairly easy to write email parsers that can handle the data import too ...

      My 2 pesos.

      -mr.dunstan
[reply]
Re: Running Perl Continuously
by atcroft (Abbot) on Dec 15, 2001 at 08:31 UTC

    In a way, this sounds a lot like some of the early chat-room CGIs that were available 4 to 6 years ago. You might look to those for some ideas.

    Unless you're uploading the entire database file and it is extremely huge, there are likely better ways for doing something of that nature, such as uploading a dump of the database and having it processed after it finishes. This would also be advantageous because you could test the datafile's validity before starting the import, rather than depending on the connection not to break while loading.

    You mentioned having it run continuously-would that not be a case for using mod_perl, where the code is internally compiled once, and the perl executable is run with the webservice? (I may be mistaken, for I have not worked with mod_perl yet. Comments, anyone?)

    As for the single stream of output (I assume you mean not reloading pages), you might wish to look at the Content-type of "multipart/mixed" or "multipart/x-mixed-replace" for the output from your script.

    Good luck in your search for a solution, and I (and I am sure others) would enjoy seeing what you find as a solution, if you decide to post it.

[reply]
Re: Running Perl Continuously
by strat (Canon) on Dec 17, 2001 at 16:57 UTC
    Many webservers use a timeout for CGI-Scripts of about 2 minutes. So, maybe the webserver will kill your Script after about that time...

    Best regards,
    perl -e "print a|r,p|d=>b|p=>chr 3**2 .7=>t and t"

[reply]
[d/l]
Re: Running Perl Continuously
by 2501 (Pilgrim) on Dec 15, 2001 at 00:01 UTC
    Could you explain in more depth why it would need to input and output continously to a web script?.
    All I can think of at this moment is a script which refreshes once a minute or so.
[reply]

Back to Seekers of Perl Wisdom