parsing dynamic webpages

NvbrRain has asked for the wisdom of the Perl Monks concerning the following question:

hi, i'm new to the perl script and all, so i'm not quite sure if this is possible... i need to find a way to parse cgi output from a dynamic webpage. is there anyway to evaluate this webpage from within another perlscript page (i know there is an eval function...)? it is currently not known what exactly the command line for the program is (or if i have permission to it, in fact). all i know is the address to it. any thoughts?

Comment on parsing dynamic webpages

Replies are listed 'Best First'.
Re: parsing dynamic webpages by visnu (Sexton) on Jun 16, 2000 at 01:00 UTC
`use LWP::Simple; $_ = get "http://www.yahoo.com"; ## parse $_ however you like` [download] i'd suggest reading up on the LWP modules in CPAN if you want a more complicated request.. there should also be something there to help you with the parsing of the out too..	[reply] [d/l]
Re: parsing dynamic webpages by chromatic (Archbishop) on Jun 16, 2000 at 01:00 UTC
The LWP::Simple module does this quite nicely: `use LWP::Simple; my $content = get("http://perlmonks.org?node_id=18391");` [download] Voila, the text of your query.	[reply] [d/l]
Re: parsing dynamic webpages by btrott (Parson) on Jun 16, 2000 at 07:00 UTC
And if you're going to want to parse the document, take a look at HTML::Parser on CPAN. It's the best way to parse HTML and make sure that you get it right.	[reply]
RE: Re: parsing dynamic webpages by SuperCruncher (Pilgrim) on Jun 16, 2000 at 13:58 UTC
I definetly agree with this. If you attempt to parse HTML yourself, you will end up with all kinds of problems (e.g. angle brackets embedded in quotes will probably trip up your code etc.) Leave it to the experts!	[reply]