Can Twig process most press and blog sites?

ellinguista has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to scrape forums from the general press and from blogs, based on finding the proper tags that indicate comments. I get frequent Parse errors about Parser.pm. mismatched tag at line 24, column 2, byte 2358 at C:/Perl/lib/XML/Parser.pm line 187 Apparently the formatting of all those sites is not strictly valide XML-wise. What should be done ? Thanks

Comment on Can Twig process most press and blog sites?

Replies are listed 'Best First'.
Re: Can Twig process most press and blog sites? by moritz (Cardinal) on Dec 03, 2010 at 19:39 UTC
Apparently the formatting of all those sites is not strictly valide XML-wise. What should be done ? Use a HTML parser, even better if it's more forgiving. HTML::TreeBuilder and Marpa::HTML come to mind. Perl 6 - second systems done right	[reply]

Back to Seekers of Perl Wisdom