Hi,

I have a webpage that I want to dump the source HTML for. The page appears to be rendered from extra execution of JavaScript functions, so I cannot use WWW::Mechanize to get the content because it requires further JavaScript execution. So, I've tried

Win32::IE::Mechanize

But I am still getting source that has unexecuted JavaScript in it ( for instance, I get one page that has a 'window.location( ... )' function in the text returned by my IE::Mechanize code, but it itself is not executed.

However, IE done interactively renders this as I expect it to.

My question is: how can I get this to finish rendering the page and get the output of that source? Interactively, once it renders and I ask for the source, it has the content I want.

PS -- Just so you know, I'm using IE rather than Firefox/Chrome/Whatever because the pages I am parsing are from an internal (3rd party) app that requires NTLM authentication, and my IE is configured to pass that along quite well ( but I see I can specify my creds in IE::Mechanize as well).

Any help is appreciated,
Matt

Nodes I've looked at: http://perlmonks.org/?node_id=807938

Code I've used:
use Win32::IE::Mechanize;



my $url = 'https://myrurl/mypage/';

my $ie = Win32::IE::Mechanize->new( visible => 0 );
$ie->credentials('myrurl:80', '', "TLR\\unnnnnnn", 'xxxxxxxx');
$ie->get( $url );

print $ie->content, "\n";

$ie->close;

[download]

In reply to Getting InternetExplorer Source after JavaScript Executes by mpettis

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.