in reply to What happened to FEAR::API?

One less mystery in the universe, Xern replied to my tweet:
@spazm it was pulled off long time ago. bad design. need to come up with a better one.

And now a new question: Why was the design bad, what can we learn from it to move forward? I liked the premise of having a reusable component for site scraping, I feel like I'm so often doing one-off design for scraping.

My most recent scrapers have used WWW::Mechanize::TreeBuilder and HTML::TreeBuilder::XPath. Today I helped a friend put together one with Template::Extract, which was cute but seemed like a step backwards since it generates regexp's instead of using the tree nature of HTML.

Maybe I should add HTML::Selector::XPath to help me generate my XPath expressions. Tuning those is often my biggest pain with an XPath solution. This node and thread has some nice XPath tips: Re: HTML and Xpath.

Replies are listed 'Best First'.
Re^2: What happened to FEAR::API?
by xern (Beadle) on Oct 05, 2010 at 05:23 UTC
    Hi,

    There's actually a redesign of it and some not-so-practical features, such as operator overloading, are taken out.

    Please see here

    Though there is no example or documents on it at the moment. The design is better than what I did before, though it's still quite experimental and the code also uses some cleanup. In this framework, scrapers can be inherited and you don't necessarily need to create a scraper from scratch for any new projects.

    If you have questions or suggestions, welcome to shoot me an email, which is a better way to contact me.

    Thanks :-)
    Yung-chung Lin
[reply]

In Section Seekers of Perl Wisdom