redirection using LWP::UserAgent

mrguy123 has asked for the wisdom of the Perl Monks concerning the following question:

Hi, first question ever, hope it's OK:
I'm using the LWP::UserAgent perl module in order to retrieve web pages. The UserAgent uses redirection when it is needed. The command I write is:

push @{$user_agent->requests_redirectable}, 'POST';

There is also a default value which I think works only for GET. This works nicely most of the time, but recently I have been starting retrieving web pages using EZPROXY, and except for one or two cases, I can't make it work. When I use manual redirection (very long and quite ugly), I manage to retrieve the web pages nearly all of the time. Any ideas how to make it work without manual redirection?
Thanks in advance,
Guy Naamati (mrguy123)

Comment on redirection using LWP::UserAgent Download Code

Replies are listed 'Best First'.
Re: redirection using LWP::UserAgent by perrin (Chancellor) on Apr 30, 2006 at 18:14 UTC
What does it do when it doesn't work? Does the same thing work without EZPROXY? What is EZPROXY?	[reply]
Re^2: redirection using LWP::UserAgent by mrguy123 (Hermit) on May 01, 2006 at 07:27 UTC
Thanks for your reply. EZPROXY is a proxy server that web transactions go through. It works by puting this prefix before a web transaction: http://10.1.235.60:8181/login?user=admin&pass=123456&url=http://www.something.com When I try to retrieve a web page through this proxy using the LWP::UserAgent, I get a page that wants my username and password (which I have put in the prefix). However, when I use manual redirection instead of the UserAgent redirection, I manage to retrieve the page. Manual redirection is a long and ugly process, which I would like to do without. Any Ideas? Thanks again Guy Naamati	[reply]
Re^3: redirection using LWP::UserAgent by perrin (Chancellor) on May 01, 2006 at 17:45 UTC
It sounds like a bug in EZPROXY, not handling cookies or redirects correctly. I'd suggest trying a different proxy, or else analyzing the difference in the sent and returned headers when using the proxy vs. not using it to find what EZPROXY is leaving out.	[reply]
Re^4: redirection using LWP::UserAgent by mrguy123 (Hermit) on May 02, 2006 at 06:01 UTC
Re^2: redirection using LWP::UserAgent by mrguy123 (Hermit) on May 01, 2006 at 07:30 UTC
I forgot to add, that the UserAgent redirection works when not using EZPROXY.	[reply]