When trying to act like a browser, you best send the same HTTP headers as the browser. Use something like Wireshark or the Mozilla Live HTTP Headers extension to see what your browser is sending, then send the same from Perl.

What you get back using LWP::Simple likely is JSON. This is also a good format with modules available to convert it to Perl data.

Thanks, Corion.

Using Mozilla Live HTTP Headers I was able to see the header information being sent. I used that information in combination with the suggestion by NetWallah, below, to send the proper header (Content-Type) information to the web service. I am now getting the same XML back as does my browser.




Regards,
pkupnorth

If you want XML, you need to ask for it - in your REQUEST's header :

  my $req = HTTP::Request->new('GET',
                               'Accept' => 'application/xml');
[download]

Thanks, NetWallah.

I had to slightly modify the syntax above - it should be just Accept => 'application/xml' - but the general idea worked. I am now getting the same XML back that my browser gets.

Regards,
pkupnorth

The modification should not have been necessary. The two are equivalent.

The "=>" operator auto-quotes the argument to its left, so the single-quotes I used are not strictly necessary, but it is a matter of style, and they should work fine.

Let me know what error you got when the quotes were in place.

        ...it is unhealthy to remain near things that are in the process of blowing up.     man page for WARP, by Larry Wall

update: meh, I completely read right past where you said you wanted to process in XML. I digress.../update

As Corion mentioned, this is JSON, and is exceptionally common for web API returns. Here's how you can import it into a Perl structure and get at it.

use warnings;
use strict;

use Data::Dumper;
use JSON::XS;

my $json;
{
    local $/;
    $json = <DATA>;
}

my $perl = decode_json $json;

for my $href (@$perl){
    print "vendor: $href->{Header}{VendorName}\n";
    for my $item (@{ $href->{Items} }){
        print "\track num: $item->{RackNumber}\n";
    }
}

__DATA__
...paste the JSON string here
[download]

output:

vendor: Some Business
        rack num: 3127824
        rack num: 3127825
vendor: Some Business
        rack num: 3127828
        rack num: 3128353
        rack num: 3128796
        rack num: 3128797
[download]

Now, a couple of notes. I'm going to rework it with your actual code, and explain what's happening. You don't need the special $/ block in your case, nor the __DATA__ section:

use warnings;
use strict;
use HTTP::Request::Common qw(GET);
use JSON::XS;
use LWP::UserAgent;
 
my $ua = LWP::UserAgent->new;
my $endpoint = 'http://blah';

my $req = HTTP::Request->new('GET');
$req->url($endpoint);

my $resp = $ua->request($req);
my $json = $resp->content;

my $perl = decode_json $json;

# it's an array of hashes of hashes (AoHoH). We need to
# access it like this:

for my $href (@$perl){
    # grabbed each hash reference from the top-level array
    # ...now, print the vendor's name

    print "vendor: $href->{Header}{VendorName}\n";

    for my $item (@{ $href->{Items} }){
        # each top level href has a Header href, and many
        # Item hrefs inside of an array ref. Here, we're
        # looping over each Item, and doing something

        print "\track num: $item->{RackNumber}\n";
    }
}
[download]

Thanks, Stevieb.

I appreciate that you took the time to layout the code and explain the suggestions. In light of the info on JSON, I may come back in the future and try it as an alternative processing method.

I used the Mozilla extension suggested by Corion and the request header modification suggested by NetWallah. I'm now getting back the response back in XML just as my browser does.

Regards,
pkupnorth

Why don't you use a REST client (assuming that this service is REST, at least it looks like). There are plenty of solutions on CPAN (REST::Client is something I already used in the past).

Investigate further how the service interface is suppose to work if possible, it will much easier for you to use it on the long way.

Thanks, glasswalk3r.

I appreciate the suggestion. I'll take a look at the REST::Client module and see if and how it fits in.

Regards,
pkupnorth