in reply to how to get dynamic html pages??

Not sure what you mean by that Dear Monk. The technology used to generate a web page bears no relevance to the ability of a browser or browser-alike to retrieve it.

The simplest example I can think of to accomplish this comes right from the man page of LWP::UserAgent:

require LWP::UserAgent; my $ua = LWP::UserAgent->new; $ua->timeout(10); $ua->env_proxy; my $response = $ua->get(’http://some.url.com/’); if ($response->is_success) { print $response->content; # or whatever } else { die $response->status_line; }


Peter L. Berghold -- Unix Professional
Peter -at- Berghold -dot- Net; AOL IM redcowdawg Yahoo IM: blue_cowdawg

Replies are listed 'Best First'.
Re^2: how to get dynamic html pages??
by monkster (Sexton) on Jan 25, 2008 at 08:34 UTC
    for example http://beta.nasdaq.com/ page got thru LWP::Useragent doesn ve all the data that u can see on screen.. think it uses lotta dynamic data.. i guess completely emulating a browser (like firefox) in executing the page may help.. can u pl propose some ways for that..
          can u pl propose some ways for that..

      Conceptually what you need to do is parse the HTML you have read in on a page, look for URIs pointing to other resources, fetch those, act on those, lather, scrub , rinse, repeat as necessary....


      Peter L. Berghold -- Unix Professional
      Peter -at- Berghold -dot- Net; AOL IM redcowdawg Yahoo IM: blue_cowdawg