Afternoon

I'm a big fan of Mechanize as well, enough so that I started porting it to ruby. I used ruby-htmltools (think HTML::Tree), REXML (XPATH & XML trees) and HTTP Access2 (LWP). Building it using this set of libraries gave me a few ideas for W:M-

At the moment, WWW::Mechanize doesn't try to do any general parse; it uses Tokeparser to rip out links and forms, HeadParser to do the head, etc, but doesn't really care much about the rest of the doc. One thing I found in porting, using a generic HTML parsing tool, then converting the result to an XML tree was that I could very easily pick out bits of content from the page. REXML allows the use of XPATH or XML::Twig/HTML::TreeBuilder-type scans of the parse tree, making it very easy to pick out, say, the title, or the number of <LI> elements with class 'green' in the <UL> tag with the id attribute 'foo'.

Though I've never tried it, I'm sure it would be easy to use HTML::TreeBuilder to do something similar with WWW::Mechanize (IIRC it relies on HTML::Parser, too). Then, in your subclass of W:M, you have lovely convenience methods like $mech->title(). Oh, actually W:M has that already .... hmm, how about $mech->keywords()? Or $mech->get_element_by_id('foo')?. I dunno if this was the kind of thing you were thinking of, but I think it does extend the range of possibilities.

cheers
ViceRaid

In reply to Re: Customizing WWW::Mechanize by ViceRaid
in thread Customizing WWW::Mechanize by Ovid

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.