Re: Efficiently Walking Large Data Structures Stored in Multiple Tables

It sounds like your production app is slowing down due to the all the database accesses to find the next page, so what you want to do is minimize those.

You can think of progressing through the cirriculum as a sequential walk through the records, which makes me think of a linked list. So a simple solution is to add a column to your db that gives the location of the next page 'next_page_ID' (assuming that you have a pages table in which each page has a unique ID). Then finding the next page has no extra lookups or tricky hash walking--everything has been precomputed beforehand.

-Mark

Comment on Re: Efficiently Walking Large Data Structures Stored in Multiple Tables

Replies are listed 'Best First'.
Re: Re: Efficiently Walking Large Data Structures Stored in Multiple Tables by jerrygarciuh (Curate) on Feb 28, 2004 at 15:40 UTC
Mark, That is a great idea! I can make the curriculum admin wizard do the heavy lifting as the items are being added/reordered. Thanks! Off to work on it... jg _____________________________________________________ "The man who grasps principles can successfully select his own methods. The man who tries methods, ignoring principles, is sure to have trouble. ~ Ralph Waldo Emerson	[reply] [d/l]
Re^3: Efficiently Walking Large Data Structures Stored in Multiple Tables by adrianh (Chancellor) on Feb 28, 2004 at 17:28 UTC
If you're dealing with a hierarchy of information (OT?) Recursive sql queries? might be of interest.	[reply]
Re: Re^3: Efficiently Walking Large Data Structures Stored in Multiple Tables by aquarium (Curate) on Feb 29, 2004 at 21:51 UTC
whilst recursive queries are fun, as you can do a whole lot of lookups and join them together in one sql statement; they do take a long time to run, significantly degrading performance. The flatter you can make the structure of the database, especially the often used tables, the faster the retrieval -- unfortunatelly this also means you end up with extra effort in maintaining the tables, as they start having redundant data (not fully normalised).	[reply]
Re^5: Efficiently Walking Large Data Structures Stored in Multiple Tables by adrianh (Chancellor) on Feb 29, 2004 at 22:29 UTC
Re: Re^3: Efficiently Walking Large Data Structures Stored in Multiple Tables by dragonchild (Archbishop) on Feb 28, 2004 at 18:51 UTC
If you're dealing with a database that can handle tree structures natively, maybe. But, I read the articles on doing tree structures as sets and I was ... underwhelmed. If all you need is the next page and there is only ever one next page, then deal with it that way. If you need a true parent-child structure and a way of asking for all parents without wanting to explicitly code it into your SQL and have it handled correctly by the optimizer ... good luck without Oracle or PostgreSQL. Now, you could build the queries using some module (???), but the queries are very very very slow and very very very difficult to get right. Again, good luck. I wouldn't want to try ... ------ We are the carpenters and bricklayers of the Information Age. Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.	[reply]
Re: Re: Re^3: Efficiently Walking Large Data Structures Stored in Multiple Tables by tilly (Archbishop) on Feb 29, 2004 at 01:42 UTC
Re: Re: Re: Re^3: Efficiently Walking Large Data Structures Stored in Multiple Tables by dragonchild (Archbishop) on Feb 29, 2004 at 04:11 UTC
Some notes below your chosen depth have not been shown here
Re^5: Efficiently Walking Large Data Structures Stored in Multiple Tables by adrianh (Chancellor) on Feb 29, 2004 at 22:41 UTC
Re: Re^5: Efficiently Walking Large Data Structures Stored in Multiple Tables by dragonchild (Archbishop) on Mar 01, 2004 at 02:28 UTC
Some notes below your chosen depth have not been shown here


Welcome to the Monastery
	PerlMonks