Creating browser-like mechs

Alien has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks !
How can a WWW::Mechanize object act more like a browser ? I'm speaking from a stats generator ( awstats etc ... )
point of view . Besides the user-agent header and the referrer , to what should the other headers be changed ?

Comment on Creating browser-like mechs

Replies are listed 'Best First'.
Re: Creating browser-like mechs by geekphilosopher (Friar) on Feb 22, 2007 at 20:21 UTC
If actually running a browser is an option, you can try Luke Closs' excellent WWW::Selenium (and Test::WWW::Selenium). Selenium is a great test automation tool - we use it to test all of our UIs.	[reply]
Re: Creating browser-like mechs by Joost (Canon) on Feb 22, 2007 at 20:22 UTC
JavaScript, probably, if you really want to look like a common browser. If it's only for stats I don't see the point, but try Win32::IE::Mechanize and Mozilla::Mechanize "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re: Creating browser-like mechs by perrin (Chancellor) on Feb 22, 2007 at 20:09 UTC
Why don't you look at the headers your browser sends and change mech until it sends the same ones?	[reply]