Re: Improving efficiency - repeated script hits at small interval

Ok, elaboration time.

This is also going to invovle session persistience type ideas. Basically whats going to happen, is client A first hits the script. The script generates an object/data/etc for him, by a series of sql queries, some processing, some files, etc etc. He has his data. That data is then stored in the session attached to him via a cookie. Then the script generates an output screen for him, made up of some data from his "object", from other peoples objects, some environment data, etc etc. This screen would be sent back to him. However, the client would be constantly (2-3 a minute) refreshing the script. Since the output data is dependent on "outside" factors (i.e. not just data contained by the clients session object dealy), the output data that is sent back might be changed a little bit, (a few numbers), or might not be changed at all. As i mentioned above though, the little changes are critical, so whatever system i use must be able to keep sending back the new data. The problem (that i see from here, i havent tested/benched this at all) is that every time the client refreshes, it would have to generate the "output data" all over again, not to mention reinitialize the session data and so forth. Also note that this solution must be entirely "browser based", i.e. using nothing other then what a default browser has (and the server of course), which means basically html/javascript (yes, it will be required. deal.) is allowed, as well as whatever serverside perl / db magic you can conjure up.

Update:
I guess what this mostly boils down to, is there any way i can avoid having to constantly reinitialize 'session/objects' per user? It seems to me that every time the client hits the script, it would have to go through a (relatively) huge amount of work, reloading all the files, constructing the objects from mysql queries etc.

Comment on Re: Improving efficiency - repeated script hits at small interval

Replies are listed 'Best First'.
Re: Re: Improving efficiency - repeated script hits at small interval by perrin (Chancellor) on Jun 27, 2002 at 03:25 UTC
Is there any way you can have 100% up-to-date data without doing the work to look it up? Not really. If there was, Oracle would be out of business. The closest you can get is the invalidation technique that I discussed above: every time anyone changes any of the data that you generate your responses from, you have to clear any cached responses that depend on that data. This is only possible in some situations. I'd suggest you build it with no caching first, and then if it's slow you can look for where you'll get the most bang for your caching buck. You may find that your users will gladly trade a 5-minute delay on certain kinds of data for significantly better page response. Also, think twice about putting all of that data into "session" objects. Unless it is specific to a single user, it doesn't belong in a session. Things like query results are not usually user-specific, i.e. a search for all stocks that have gone up in the 24 hours should be cached under "stocks, age < 24", not some session ID. In most systems users frequently repeat each other's queries, so caching results in a user-independent way gives better hit rates in the cache.	[reply]

Replies are listed 'Best First'.

Re: Re: Improving efficiency - repeated script hits at small interval
by perrin (Chancellor) on Jun 27, 2002 at 03:25 UTC

I'd suggest you build it with no caching first, and then if it's slow you can look for where you'll get the most bang for your caching buck. You may find that your users will gladly trade a 5-minute delay on certain kinds of data for significantly better page response.

Also, think twice about putting all of that data into "session" objects. Unless it is specific to a single user, it doesn't belong in a session. Things like query results are not usually user-specific, i.e. a search for all stocks that have gone up in the 24 hours should be cached under "stocks, age < 24", not some session ID. In most systems users frequently repeat each other's queries, so caching results in a user-independent way gives better hit rates in the cache.

[reply]