TommyAllan has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks, I'm using a simple robot to download a secure page. I've done this many times before with different pages without any problems, but this time it returns an ASP.NET error page. The script is: 
#!/bin/perl

use LWP::UserAgent;
use HTTP::Request::Common;
use Crypt::SSLeay;
use HTTP::Cookies;
use HTTP::Headers;

print "Content-type: text/html\n\n";
$ua = LWP::UserAgent->new;
$login = "https://secure.video.uefa.com/Selfcare/default.aspx?navGraph
+=Login";
$req = HTTP::Request->new(GET => $login);
$res = $ua->request($req);
print $res->as_string;

[download]
Any ideas?

Replies are listed 'Best First'.
Re: LWP::UserAgent gives ASP.NET error
by grinder (Bishop) on Oct 14, 2007 at 11:54 UTC

    Works for me. You do have many packages explicitly use'd that don't need to be, It will work just as well with:

    use LWP::UserAgent;
use HTTP::Request;

    [download]

    With just that, the page is downloaded successfully. That said, the contents do appear to be attempting to display some sort of error message via Javascript.

    It looks like the URL needs some addition GET parameters. You need navGraph=Login&foo=bar&klong=sproink and so on. Exactly what parameters are needed will require a little more detective work on your part.

    • another intruder with the mooring in the heart of the Perl

[reply]
[d/l]
Re: LWP::UserAgent gives ASP.NET error
by dsheroh (Monsignor) on Oct 14, 2007 at 14:55 UTC
    If you're getting an error, the first thing you should do is examine the reported error. Especially when you're getting the error from a remote system.

    Bringing up your URL manually in my browser, I am also getting an error page, "Message: Can not activate the view ErrorPopulate. Error executing child request for /SelfCare/Template.aspx." With the input you provide, /SelfCare/Template.aspx is failing. This means one of two things:

    1) /SelfCare/Template.aspx is broken
    -or-
    2) You're passing it the wrong input

    If it's #1, there's really nothing you can do about it except maybe complain to the uefa.com admins and tell them to fix it. If it's #2, then you need to poke at it a bit more with your browser to figure out the correct input (both URL and HTTP POST parameters) to send with your request.

    Either way, if you're receiving the same error page with your robot as I am with my browser, then your robot is working just fine.

[reply]
      Thanks for the responses.

      You're correct the $login I supplied gives the same error page with both browser and robot! I tested it this morning on my office computer. My home computer must have used a cookie which made the page work, as I didn't get the same response from the robot and the browser then.

      However now I've changed the $login to:
      $login = "https://secure.video.uefa.com/Selfcare/default.aspx?navGraph
+=Login&returnUrl=http://video.uefa.com/video/login.html&returnUrl2=ht
+tp://video.uefa.com/video/login.html&style=";

      [download]


      It still gives the same error as below: 
      HTTP/1.1 200 OK Connection: close
Date: Mon, 15 Oct 2007 08:47:07 GMT
Accept-Ranges: bytes
Server: Microsoft-IIS/6.0
Content-Length: 883
Content-Type: text/html
ETag: "0ce8cbe4b1dc71:df1"
Last-Modified: Mon, 11 Dec 2006 17:42:36 GMT
Client-Date: Mon, 15 Oct 2007 08:47:07 GMT
Client-Peer: 213.198.70.20:443
Client-SSL-Cert-Issuer: /C=BE/O=GlobalSign nv-sa/OU=ServerSign CA/CN=G
+lobalSign ServerSign CA
Client-SSL-Cert-Subject: /C=CH/ST=Vaud/L=Nyon/O=UEFA/OU=UEFA Media Tec
+hnologies SA/CN=secure.video.uefa.com
Client-SSL-Cipher: EDH-RSA-DES-CBC3-SHA
Client-SSL-Warning: Peer certificate not verified
Set-Cookie: AlteonP=2b2bfcaa2b2bfcf2a52fba71; path=/
Title: Error
X-Meta-GENERATOR: Microsoft Visual Studio .NET 7.1
X-Meta-Originator: Microsoft Visual Studio .NET 7.1
X-Meta-ProgId: VisualStudio.HTML
X-Powered-By: ASP.NET Error!
Message: undefined

      [download]
      The browser page now works okay, with cookies switched off and with same query string so that's not a problem. I just need to get the same page using the robot as the browser as I need to take out a few params so I can then automatically log in (most notably the massive VIEWSTATE).

      Thanks, Tommy
[reply]
[d/l]
[select]
Re: LWP::UserAgent gives ASP.NET error
by erroneousBollock (Curate) on Oct 14, 2007 at 12:02 UTC
    You probably don't need to manually include Crypt::SSLeay. Also, you don't seem to be creating or using a cookie-jar, so HTTP::Cookies goes unused. I guess both items are related to your real program.

    Otherwise the request looks OK. Could you post the ASP.net error? That'd probably be instructive.

    Update: Ah, right. grinder has given you a push in the right direction.

    -David

[reply]
Re: LWP::UserAgent gives ASP.NET error
by Gangabass (Vicar) on Oct 15, 2007 at 04:05 UTC

    Maybe target server dislike your GET method? Did you try POSTing? Also try exactly repeat all form fields that your browser sending when you pass authorization.

[reply]

Back to Seekers of Perl Wisdom