roxylandmusic has asked for the wisdom of the Perl Monks concerning the following question:

Hi, Sorry if this is a lame question, but I just can't get around a 'browser detection' javascript on a particular website with the LWP UserAgent's agent method. This is what I used and the site does work on the browser which sets the same agent. ( Safari and Firefox on Mac ).

User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv: +1.9.1.9) Gecko/20100315 Firefox/3.5.9 Content-Length: 71 Content-Type: application/x-www-form-urlencoded
Here is the section of javascript that stops me from viewing the page:
function broswerDetection(){ var appName = navigator.appName; var appVersion = navigator.appVersion; var userAgent = navigator.userAgent; var browserSupported = false; if(appName == "Netscape") { // the browser could be a Netscape o +r Firefox if(userAgent.indexOf("Firefox") == -1){ // the browser is Netscape, then we flag that the netscape + not supported browserSupported = false; } else if((userAgent.indexOf("Navigator") != -1) && (userAgent.i +ndexOf("Firefox") != -1) ){ // the broswer is Netscape v9.0 browserSupported = false; } else{ // this is actually a Firefox browser var firefoxIndex = userAgent.indexOf("Firefox") + 8; if(parseInt(userAgent.charAt(firefoxIndex)) >= 1){ browserSupported = true; } } } else if(appName == "Microsoft Internet Explorer"){ var version = 0.0; if(userAgent.indexOf("Opera") == -1){ // Microsoft Internet Explorer if(appVersion.indexOf("MSIE") != -1){ var temp = appVersion.split("MSIE"); version = parseFloat(temp[1]); } if(version >= 6.0){ browserSupported = true; } } else{ // Opera broswer, flag the broswer not supported browserSupported = false; } }
Could someone please help me understand why LWP can't get around this check ?

Replies are listed 'Best First'.
Re: LWP UserAgent's agent method and browser check
by Corion (Patriarch) on Nov 10, 2010 at 12:58 UTC

    LWP does not understand nor care about Javascript.

    If you want to replicate what the browser is doing, capture the network traffic with a network sniffer like Wireshark and compare what headers your Perl code sends with what headers your browser sends. Eliminate all differences.

Re: LWP UserAgent's agent method and browser check
by marto (Cardinal) on Nov 10, 2010 at 13:01 UTC

    LWP doesn't support JavaScript. If you're trying to automate a site which requires JavaScript see WWW::Mechanize::Firefox or do some network probing to find out what headers the browser sends and compare it to what your Perl code is sending.

      Thanks marto, corion. Unfortunately coding PERL to replicate headers sent by my brower would be too big a challenge for me with my present skills. So I'll try Mechanize for the time being and see if that suits this task. Thanks again.