recursively extracting links from yahoo, when to stop?

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Has anyone attempted to extract links / descriptions from yahoo? I was attempting this with LINK::Extractor and it worked just find, the only problem is that i need to do a recursive extraction and so categories have subcategories and some go deeper than others. The question is where do i find a boundry to stop at?

Does anyone have any ideas? thanks

Title edit by tye

Comment on recursively extracting links from yahoo, when to stop?

Replies are listed 'Best First'.
Re: extracting links from yahoo by chip (Curate) on May 15, 2003 at 14:32 UTC
Surely it's up to you to decide where you want to stop. We can help you with the how but not the why. -- Chip Salzenberg, Free-Floating Agent of Chaos	[reply]
Re: extracting links from yahoo by MrYoya (Monk) on May 15, 2003 at 22:47 UTC
There's several options. You can stop after recursing N number of levels N links are extracted N amount of time when you're extracting from a site that's not Yahoo. or anything else you can imagine. It's your call, really.	[reply]