the best way to implement persistent data across CGI scripts

d_i_r_t_y has asked for the wisdom of the Perl Monks concerning the following question:

yes, that old nutshell. though in my searches, there appears to be surprisingly few pearls in all the swine which actually discuss in some kind of detail (a la the cookbook) the relative merits of the various solutions.

the problem is: i'd like to have a persistent data structure or set of RDMS data which persists across multiple cgi script sessions.

the best methods that i can envisage for doing this are:

using disk-based DB files -- though might there be a problem with concurrency? one could flock, but then will this limit performance with many concurrent users?
using temporary DB tables -- but does this impact performance too greatly?? how does one ensure that temporary tables get deleted in due course? and how might one go about protecting oneself from a malicious user who just hammered the server with broad queries?
http cookies -- this is probably unacceptable for reasons of security -- i would ideally like to cache a whole sql WHERE clause for instance.
hidden CGI params -- same as above, but more complicated
a slightly more elaborate solution involving some king of daemon which caches data per host.

i'm sure a million monks have broached and solved all these issues dozens of times... can anyone offer me some wisdom? my particular site uses MySQL and the database involved is only fairly small.

advice, comparisons, and links all greatly appreciated.

j.a.dirty.p.h

Comment on the best way to implement persistent data across CGI scripts

Replies are listed 'Best First'.
Re: the best way to implement persistent data across CGI scripts by ncw (Friar) on Sep 20, 2000 at 10:17 UTC
There is no need to re-invent the wheel here - there is a module called Apache::Session which implements most of the above methods. It is an OO module with pluggable backends, and it can store things in files Apache::Session::File databases Apache::Session::MySQL and many more! A very good reference on this is the mod_perl book Writing Apache Modules with Perl and C. This goes on in depth about all the above methods you suggest. It also talks about how to make sure the user keeps giving you back the session number - in a cookie, in a form or as part of the URL eg http://z.com/87665456789/page.cgi(easy to do with mod_perl hard otherwise) merlyn implemented a novel approach to session storage by spawning a new webserver for each session on a new port. This has the advantage that you can store lots of data in variable form in memory. It needs careful management though. Look for it in his web column here.	[reply]
RE: the best way to implement persistent data across CGI scripts by geektron (Curate) on Sep 20, 2000 at 10:21 UTC
you missed one method: Storable.pm. (if you're using OO-data). you can freeze an object, pass it as a hidden parameter, and thaw the 'stringified' object on the next invocation. more secure than some, because if the stringified object is tampered with, it more than likely won't thaw properly, and session data is lost.	[reply]
Re: the best way to implement persistent data across CGI scripts by nop (Hermit) on Sep 20, 2000 at 14:10 UTC
In a friendly environment (say, a small trusted intranet), hidden CGI params can hold small amounts of data. They aren't secure, and you can't put reams of data in them, but are often convenient for small projects. I'd generally vote for option 2, DB tables. Then the script just passes a session id (say a random 30 digit string, to prevent session-stealing by guessing ids) and the server can retrieve / update anything it needs related to that session. I wouldn't necessarily make the tables temporary (in the DB sense of the word) either: session data can be invaluable when analyzing how your site is being used. When table starts to grow too large, have a process roll defunct or ancient sessions out of the DB into inactive archival storage. Just my two cents.	[reply]