in reply to LWP::UserAgent non-feature?

I could be wrong, but my impression after having attempted to research this, is that the LWP::UserAgent->request method is being rather entirely unhelpful and unfriendly here

It looks like LWP::UserAgent is not the one splitting up the URL. It is *your* code that is doing that.

I'm guessing that you are under the impression that this is required of you for some reason. And yet, one of the very first lines of the documentation for one of the modules you use is: 

$request = HTTP::Request->new(GET => 'http://www.example.com/');

[download]

So, clearly, the module expects to get a full URL. Were you assuming that the module is too stupid to do any processing on that URL and would just send "GET http://www.example.com/ HTTP/1.1"?

Perhaps you got this impression from looking at what as_string() returns? But the documentation says: 

$r->as_string
$r->as_string( $eol )

[download]

Method returning a textual representation of the request.

It doesn't say "returns the string that will be sent to the HTTP server".

If you really need to see the code that constructs the actual HTTP request, then read the request() method in LWP::Protocol::http.

Change your code to stop doing naive splitting of the URL and use the modules as documented and you will probably get better results.

- tye        

Replies are listed 'Best First'.
Re^2: LWP::UserAgent non-feature? (docs)
by rfg (Initiate) on Jan 13, 2015 at 19:11 UTC
    Thanks for the replies folks. At this juncture, I should make a few brief points, and then just leave this "non-feature" be.

    1) After posting last nite, I traced the problem I was having with the server sending back "incorrect" results to my own programatic faux pas. My original test code (which I did not post) inadvertantly was lower casing the entire URL inappropriately, and that was the real problem. Once fixed, everything else worked beautifully. I am suitably humble that my own mistake was the primary cause of my posting here.

    2) Regardless of the above, the output generated by the as_string method of HTTP::Request is inherently prone to causing confusion, I think, given that the string generated by that method is not, apparently, what actually gets sent to the server in at least some cases.

    3) There is no compelling reason that I can see why the request method of LWP::UserAgent either cannot be or should not be smart enough and/or helpful enough to be able to obtain the server hostname from a Host: header which is present within the HTTP::Request object it is given. But it does appear to be the case that it does not do so, preferring instead to yield a simulated 400 error response from the actual server, which is itself arguably problematic, i.e. in that this faked server response might cause... and apparently has caused... some users to believe that this response is actually coming from the real server. (Wouldn't it perhaps be better to throw an exception, or some such other thing, rather than internally generating a confusing faked server response?)

[reply]
[d/l]
[select]
      Hmm, whenever LWP "fakes" a response, it identifies it as such "Client-Warning" header set to the value "Internal response".
[reply]

In Section Seekers of Perl Wisdom