Your script finds the element you seem to be looking for once I fix the bad Xpath query in line 21:

//dataAddress2[id="city"]

[download]

would be searching for an HTML tag dataAddress2, which does not exist on that page (nor anywhere else).

As you are searching for an element with an id attribute anyway, and id attributes are (supposed to be) unique across the page, using the following XPath expression extracts the element for me (provided I've unblocked the crappy Javascript on all those pages in Noscript):

//*[@id="city"]

[download]

For finding what elements I've captured, I like to print ->{innerHTML}:

print "..." . $mech->xpath('//*[@id="city"]', one => 1)->{innerHTML};

[download]

It seems that the Javascript gets triggered after some time without another event and the element just gets filled in instead of actually appearing, so you might need to wait in a loop to watch the element content change from   to the content you actually want.

In reply to Re: Scraping Rendered Text that is not in Source Code by Corion
in thread Scraping Rendered Text that is not in Source Code by bobross419

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.