Monks:

I'm working on a project for work where I need to screen scrape an internal website. All of the internal sites use Single Signon (SSO).

This is the current code I've been working with: 
#!/usr/bin/perl -w

use strict;
use LWP::UserAgent;

my ($ua, $req, $res, $output);

$ua = new LWP::UserAgent(keep_alive=>1);
$ua->agent('Internet Explorer/6.0');
$ua->credentials('domain.com:80', '', "domain\\user", 'pass');

$req = HTTP::Request->new(POST => 'http://domain.com/cgi-bin/test.cgi'
+, HTTP::Headers->new('WWW-Authenticate' => 'NTLM'));
$req->content_type('application/x-www-form-urlencoded');
$req->content('testvar=testvalue');

$res = $ua->request($req);

$output = $res->content;

print "output:\n".$output;

[download]
This is the output I get when running this script: 
output:
<HTML>
<HEAD>
<TITLE></TITLE>
</HEAD>
<BODY onLoad="document.AUTOSUBMIT.submit();">
This page is used to hold your data while you are being authorized for
+ your request.<BR><BR>You will be forwarded to continue the authoriza
+tion process. If this does not happen automatically, please click the
+ Continue button below.

<FORM NAME="AUTOSUBMIT" METHOD="POST" ACTION="http://logon.domain.com/
+ntlm/creds.ntc?CHALLENGE=&SMAGENTNAME=$SM$7itOP8YnQhR1qQcOO6pp75JDJJF
+C96mLihRDilLmOSr8oAm7zddwfw%3d%3d&TARGET=$SM$http%3a%2f%2domain%2ecom
+%2fcgi-bin%2ftest%2ecgi">

<INPUT TYPE="HIDDEN" NAME="SMPostPreserve" VALUE="L2JDNDdmSXpOWVVOT2wy
+YUt4VjZpb1JkbkkxRVlZbUd4ZVdadkFoaGpQZWJVZHE4eW9IcGNJNEx3dUdmYi81Qw">
<INPUT TYPE="SUBMIT" VALUE="Continue">

</FORM>

</BODY>
</HTML>

[download]
I've tried a number of methods to handle the challenge/response, but to no avail. I've even tried using a stand-alone curl command, which has a --ntlm option.

Any input from the ranks is appreciated.

Thanks!

- Justin

In reply to LWP::UserAgent vs. NTLM Challenge/Response by jpk236

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.