LWP and Win32::Internet fail where MSIE sees all.

aplonis has asked for the wisdom of the Perl Monks concerning the following question:

Help me out with a most perplexing mystery, please. I have two Perl scripts, a longish one using LWP which is here...

gus_barometric_pressure.pl

...and a short (test only) script which looks like this...

use strict;
use warnings;
use Win32::Internet;

my $INET = new Win32::Internet();
my $file = $INET->FetchURL("http://www.yahoo.com");
print $file;
[download]

Those programs both work fine on one PC but not on another. Both PCs are in a company network and being run on my network login account. And both PCs have the same version of ActiveState Perl and the same modules. I get no Perl errors on either PC. But on one, and not on the other, both scripts fail to connect to their web pages, getting no data.

Now here is the really funny thing. Both PCs can see external web pages in MSIE, but only one can see the web in Firefox. The same PC which can't connect in Firefox is the one which also fails using LWP and Win32::Internet modules. And the only difference between their IP settings appears to be that the one which fails has different secondary WINS server. There does not appear to be any proxy, not that I can find, for MSIE.

How can this be? And more importantly, how can I work around it? What questions do I need to pose to our IT department so that they might enlighten me? Our local IT is but one person for a large plant, and not an especially inquisitive network guru. A network printer was left unusable for months because it was a low priority until I sorted it out myself. Our local IT person is kept busy full time just implementing things laid out by higher IT folks in another city out-of-state. Just asking there with no ideas to investigate would be a dead end. So I need to present my questions very concisely so that they will be passed up the line.

Thanks, kindly,

Gan Starling

ADDENDA:

In retrospect, I should prefer a Perl answer above all else. It was from having compared ipconfig /all for both machines that I determined, as mentioned in my initial post, that the only difference was the Secondary WINS IP. I also mentioned therein my use of the same network login for both.

MSIE works. So HTTP must work. So I guess my question might better have been, "How can the network distinguish between an HTTP request made by MSIE from one made to the same URL by LWP or by Win32::Internet?" Alternately, I might have better asked, "Is it possible to make them indistinguishable so that Perl can emulate MSIE as far as the network is concerned?"

Again, neither HTTP nor the URL is restricted as MSIE can see it. Only LWP and Win32::Internet are unable to see that URL. That can't possibly be a security issue, not with MSIE's reputation. So a more purely Perl-ish question might be, "How can I make LWP sufficiently emulate an MSIE HTTP request?" I have already tried using the MSIE user-agent ID string and that did not work.

Respectfully,

Gan Starling

Comment on LWP and Win32::Internet fail where MSIE sees all. Select or Download Code

Replies are listed 'Best First'.
Re: LWP and Win32::Internet fail where MSIE sees all. by glasswalk3r (Friar) on Feb 07, 2007 at 19:11 UTC
Looks like you need to search for errors in other spots besides Perl configuration. It may be a registry key or even a firewall running in the machine. As you have asked specifically for a Perlish answer, may you should try using a Perl module to implement a sniffer and check out how the connections are made (for example sniffer) or, even better, LWP::Debug. Remember, CPAN is your friend. Alceu Rodrigues de Freitas Junior --------------------------------- "You have enemies? Good. That means you've stood up for something, sometime in your life." - Sir Winston Churchill	[reply]
Re: LWP and Win32::Internet fail where MSIE sees all. by BrowserUk (Patriarch) on Feb 07, 2007 at 14:57 UTC
Try flushing the dns cache on the failing machine and see if that makes any difference? `c:\>ipconfig /flushdns Windows IP Configuration Successfully flushed the DNS Resolver Cache.` [download] Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l]
Re^2: LWP and Win32::Internet fail where MSIE sees all. by aplonis (Pilgrim) on Feb 07, 2007 at 15:15 UTC
It made no difference. I was able to do that and did get the "Successfully flushed..." response. But it made no difference in LWP or Win32::Internet failing where MSIE works. I am responding via the supspect PC.	[reply]
Re^3: LWP and Win32::Internet fail where MSIE sees all. by BrowserUk (Patriarch) on Feb 07, 2007 at 15:21 UTC
Does the failing machine fail in the same way for all urls, or only particular ones? Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^4: LWP and Win32::Internet fail where MSIE sees all. by aplonis (Pilgrim) on Feb 07, 2007 at 15:27 UTC
Re^5: LWP and Win32::Internet fail where MSIE sees all. by BrowserUk (Patriarch) on Feb 07, 2007 at 16:03 UTC
Some notes below your chosen depth have not been shown here
Re: LWP and Win32::Internet fail where MSIE sees all. by NiJo (Friar) on Feb 07, 2007 at 20:20 UTC
Did you try telnetting to port 80 from both machines an speaking html yourself? I expect it to fail on both machines, as a required proxy setting is not used. My guess is that MSIE relies on "automatic proxy configuration". Firefox seems to have acquired this feature just in the 2.0 releases. AFIR that feature uses DHCP to provide an URL where the browser can load and execute javascript to determine proxies. Your job is to ask, guess, hunt down or sniff that URL or the hard host name and port. Implementing that in Perl should be the easy part.	[reply]
Re: LWP and Win32::Internet fail where MSIE sees all. by spadacciniweb (Curate) on Feb 08, 2007 at 10:03 UTC
To emulate a browser HTTP request, you can see the whole HTTP-session between browser and server. So you can use a sniffer: wireshark ethereal (($_="Mzz ojjdloobnf jt uvy5502383")=~y~b-zg2-5c96-81~a-z0-9~s)=~s~~~s; print	[reply]