Getting data from web page

algonquin has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks, I have a question. My script is finally working (thanks to you all), I couldn't have come this far without your contributions. Anyways, what it's suppose to do is: it goes to web pages and gets the source code and looks for info (numbers) and does some math with them. The latest version is working if I save the source of a web page in a text file and parse it with my perl script. I supply the html address of the page and it gets and saves source in a file. Some links provide the addresses, but some links don 't contain the address for my destination. Instead, they initiate a java script or something. I get to the page when I click on it but my script needs to get there. How do I get around this. Any ideas??? Thanks

Comment on Getting data from web page

Replies are listed 'Best First'.
Re: Getting data from web page by kutsu (Priest) on Sep 23, 2004 at 16:53 UTC
The easiest way to follow links would proably be www::mechanize. Unfortunetly, www::mechanize will not follow javascript links (as stated in The Faq). You might be able to pull the javascript out of the source and search it for an address, which you could then follow. Update: URI::find might help, and there is a helpful tutorial right here on perlmonks. "Cogito cogito ergo cogito sum - I think that I think, therefore I think that I am." Ambrose Bierce	[reply]
Re: Getting data from web page by dba (Monk) on Sep 23, 2004 at 16:42 UTC
You need WWW::Mechanize Look for `follow_link`	[reply] [d/l]
Re: Getting data from web page by davorg (Chancellor) on Sep 23, 2004 at 16:31 UTC
Can you give a brief example of the kinds of problems that you are finding. -- <http://www.dave.org.uk> "The first rule of Perl club is you do not talk about Perl club." -- Chip Salzenberg	[reply]