If you want perl control, check out Mozilla::Mechanize::GUITester and/or Mozilla::Mechanize or WWW::Selenium. update2: Win32::IE::Mechanize

update: just to be clear: LWP does not support javascript at all, and as far as I know no-one is planning on adding javascript support. All options I linked drive conventional GUI browsers since that's a lot easier to build and support than creating your own compliant and effectively useful DOM/BOM with javascript support. I should know - I gave it a try once :-)

Joost,

Thanks. This all looks like the kind of things I need to add to a general toolbox for making webpages jump through hoops, mostly for automating testing, and computer aided manual testing. The specific task I wanted to solve is counting sponsored links (ads) on a web page. I cannot see them in LWP, only the Ajax that goes and gets them.

For some reason, I have not even tried the mechanize modules. I guess I thought I was getting more control from LWP. I just heard about Selenium last week for the first time, so I will take a look at that too.

In regard to your note about LWP, that confirms the sense I had about it. So I know now not to bark up that tree anymore.

Thanks

WWW::Mechanize is a subclass of LWP::UserAgent, so more or less anything you can do with LWP::UA you can do with WWW::Mechanize. Mechanize is just easier to use when automating click-trails (start at link such-and-such, click link named "bla bla", fill in form field "thingy", submit - etc)

The *::Mechanize::* modules linked in my post above mimic and extend WWW::Mechanize's behaviour but are probably not 100% API compatible - YMMV.

"What should it profit a man, if he should win a flame war, yet lose his cool?"