in reply to LWP::UserAgent and correcting automatic redirection.

In the node Re: LWP redirect problem, produces malformed URL? I provided code to get in the middle of a redirect by overriding the redirect_ok() method. You should be able to adapt it.

Replies are listed 'Best First'.
Re^2: LWP::UserAgent and correcting automatic redirection.
by Brak (Novice) on Nov 12, 2009 at 20:26 UTC

    Thanks for the tip gmargo, I installed your bit of code and it does seem to do what it's intended to do. Even though your code would have helped in the case of the server returning multiple Location headers, which is different from my problem, it did get me a head-start into verifying that my problem is different from what I originally thought it was. I now believe that the server is returning the CORRECT Location headers, which includes 2x 302 forwarding jumps, but I'm not really sure where to go from here since it's reporting "400 URL must be absolute". Any thoughts on general problem solving techniques I can try, things to check, ways to profile and get some information back as to why their server may be responding that way?

      Since you don't post any traffic except the final error, it's difficult to say much about your particular problem. In various programs I've written, a similar problem comes up, where the URI must be extended from relative to absolute with a "base" host or uri derived from the request.

      I don't know what your debugging environment is, but as a "general problem solving technique" for LWP::UserAgent, one of the most valuable tools would probably be a proxy in the middle to record all traffic between the application and the server.

      There are two methods I frequently use for logging traffic - one is another form of an LWP::UserAgent derived class that logs everything sent to/from the server. Another is an external java tool called "paros" http://www.parosproxy.org. Once you can see all the two-way traffic, it may become more obvious when you need to edit the URI on the fly.