Re: Building a Spidering Application

At the end I want to loop back and download the links in order to parse useful information. Please help! --Miriam

Comment on Re: Building a Spidering Application

Replies are listed 'Best First'.
Re^2: Building a Spidering Application by mailmeakhila (Sexton) on Jul 06, 2012 at 15:02 UTC
Why dont you add another $mech->get($url) in your foreach loop? I would also suggest using Web::Scraper.	[reply]
Re^3: Building a Spidering Application by MiriamH (Novice) on Jul 06, 2012 at 15:08 UTC
but how do I make the new $URL the subsequent HTML that is dowloaded?	[reply]
Re^4: Building a Spidering Application by choroba (Cardinal) on Jul 06, 2012 at 15:15 UTC
Call the `$link->url` method. Oh, you already do! So, use `$href`.	[reply] [d/l] [select]
Re^2: Building a Spidering Application by MiriamH (Novice) on Jul 06, 2012 at 15:10 UTC
I have a flow chart- I download the original webpage and print out the content as well as all the links. I then want to get( $URL) of each of the URL's that are linked to on the original webpage. I don't know how to write a code that will get each subsequent URL.	[reply]
Re^3: Building a Spidering Application by Corion (Patriarch) on Jul 06, 2012 at 15:18 UTC
Have you looked at the `->get` method of WWW::Mechanize? In fact, you already use it yourself when loading the initial page. Maybe consider using it also when you want to download another page. You can also look at the `->follow_link` method in the same documentation. But note that `->follow_link` will only work for following one link on a page.	[reply] [d/l] [select]