Hello Monks - I have been trying to setup a crawler to check all my info on the web. I want it to get it daily. So far, it works like it's suppose to. Except once license site which has a page loading from a query from their database. After much digging I see SOAP everywhere. I am posting my code below. Any ideas?

Thanks in advance. 

     use WWW::Mechanize;

        $browser = WWW::Mechanize->new(autocheck=>1);
     
         $URL = "my.license site.gov";

        $browser->get("$URL",":content_file" => "sample1.html");
     
        $browser->timeout(10);     
        $browser->agent('User-Agent=Mozilla/5.0 (Windows; U; Windows N
+T 5.1; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4');


#       $browser->form_number(3); 
#       $browser->field("P_CITY"=>"Merced");
#       $browser->submit();

   
        $browser->getFrame('name:','');
      
        $results = $browser->content;    

          print "$browser";
          open(OUT,">Results.html");print OUT $results;close OUT;

[download]

In reply to Crawling a site with Soap and IFrames by PerlNuBe

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.