Re^3: Feasability of quick generation of complex report from large db?

This sounds like a bit of heavy lifting for a database server -- but not an impossibility. What I'm not getting is how often you have to regenerate this big table. Every time a web user wants to look at it? Every time something in the underlying database changes?

I would imagine you could generate and then cache the 'latest version' of the table at the end of every day, assuming new data is added throughout the day.

Alex / talexb / Toronto

"Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

Comment on Re^3: Feasability of quick generation of complex report from large db?

Replies are listed 'Best First'.
Re^4: Feasability of quick generation of complex report from large db? by punch_card_don (Curate) on Feb 06, 2008 at 17:42 UTC
Hey TAlexb, Ya, everytime a user wants to see it is the request. The idea being circulated right now is that various combinations of product_classes and industry_groups will be dreamed up for a report. There are likely to be ~6,000 of these reports. Yes, someone is going to dream up ~6,000 combinations of 350 intersections. Then a user goes to the list of reports and selects which he wants to see today. And so, yes, I'm toying with the pre-generation option too, since once the data has been loaded, it never, ever changes. But at some point, you just know they're going to want to be able to generate every possible set and sub-set of intersections, so I think the pre-generation of static files would be a temporary delay of the inevitable pain.	[reply]

Replies are listed 'Best First'.

Re^4: Feasability of quick generation of complex report from large db?
by punch_card_don (Curate) on Feb 06, 2008 at 17:42 UTC

Ya, everytime a user wants to see it is the request. The idea being circulated right now is that various combinations of product_classes and industry_groups will be dreamed up for a report. There are likely to be ~6,000 of these reports. Yes, someone is going to dream up ~6,000 combinations of 350 intersections. Then a user goes to the list of reports and selects which he wants to see today.

And so, yes, I'm toying with the pre-generation option too, since once the data has been loaded, it never, ever changes. But at some point, you just know they're going to want to be able to generate every possible set and sub-set of intersections, so I think the pre-generation of static files would be a temporary delay of the inevitable pain.

[reply]