mr_p has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I am trying to parse html text with tags intact. The code I wrote using HTML::Parser strips out the tags.

Is there a way to keep tags intact?

below is my code.

#!/usr/bin/perl
package MyParser;
use base qw(HTML::Parser);
my $main_content="";
sub start {
      my ($self, $tag, $attr, $attrseq, $origtext) = @_;

        if ($tag =~ /^span$/i && $attr->{'class'} =~ /^main-content$/i
+)
        {
            # set if we find <span class="main-content"
            $content_flag = 1;
        }
  }
sub text {
        my ($self, $text) = @_;
        # If we're in <H1>...</H1> or
        if ($content_flag)    { $main_content    .= $text; }
}
my $html = " <html>
    <head>
        <title>Blah</title>
    </head>
    <span class=\"main-content\">
    <bold_text> Here's the body 1 </bold_text>
    <p> para1 </p>
    <p> para2 </p>
    </span>
    </html>";

my $parser = MyParser->new;
$parser->parse("$html");
print "$main_content\n";

[download]

Output I get: 

         Here's the body 1
         para1
         para2

[download]

Output I need is: 

    <bold_text> Here's the body 1 </bold_text>
    <p> para1 </p>
    <p> para2 </p>

[download]

I would like the above output to still have tags, is it possible to do this with HTML::Parser?

Replies are listed 'Best First'.
Re: Parsing HTML with tags intact
by roboticus (Chancellor) on Jan 06, 2011 at 23:52 UTC

    mr_p:

    The first example in the docs gives a good hint on how to do it.

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

[reply]
Re: Parsing HTML with tags intact
by Anonyrnous Monk (Hermit) on Jan 06, 2011 at 23:51 UTC

    Kind of hackish... but as the start handler is being called before the text handler, you could store the tag name in a global variable, which you can then use in the text method to decorate the content as desired: 

    ...
my $tagname;
sub start {
        my ($self, $tag, $attr, $attrseq, $origtext) = @_;
        $tagname = $tag;  # store tag name for later use

        if ($tag =~ /^span$/i && $attr->{'class'} =~ /^main-content$/i
+)
        {
            # set if we find <span class="main-content"
            $content_flag = 1;
        }
  }
sub text {
        my ($self, $text) = @_;
        # If we're in <H1>...</H1> or
        my $tagged_text = $text !~ /^\s*$/ ? "<$tagname>$text</$tagnam
+e>" : $text;
        if ($content_flag)    { $main_content    .= $tagged_text; }
}
...

    [download]

    Output: 

    
    <bold_text> Here's the body 1 </bold_text>
    <p> para1 </p>
    <p> para2 </p>

    [download]

    (Might need some further tweaking (like the $text !~ /^\s*$/ ? ...) to handle edge cases — but you get the idea.)

[reply]
[d/l]
[select]
Re: Parsing HTML with tags intact
by ww (Archbishop) on Jan 07, 2011 at 03:28 UTC
    Your "html" is just a touch odd. It contains two obvious departures from valid html.

    The data provided lacks a <body>...<body> tag pair and <bold_text> isn't an html tag at all.

    Is what you posted merely a misguided, fanciful illustration or are you actually confronted with such data?

[reply]
[d/l]
[select]
      I can have misguided data as such. I have to be able to handle such data.
[reply]

        In that case, perldoc -f HTML::Parser raises the question (for me, anyway): "Is H::P the appropriate tool?

        "HTML::Parser" is not a generic SGML parser. We have tried to make it able to deal with the HTML that is actually "out there", and it normally parses as closely as possible to the way the popular web browsers do it instead of strictly following one of the many HTML specifications from W3C. Where there is disagreement, there is often an option that you can enable to get the official behaviour."

        But I have yet to see a browser that will treat <bold_text> as a tag, rather than simply ignoring it (and merely rendering the enclosed text without additional formatting.

        Update: The doc does -- again, to me and YMMV -- suggest that you could tweak your script to deal with some of the invalid tag-like entries:

        $p->strict_names
        $p->strict_names( $bool )
        By default, almost anything is allowed in tag and attribute names. This is the behaviour of most popular browsers and allows us to parse some broken tags with invalid attribute values like:

        <IMG SRC=newprevlstGr.gif ALT=PREV LIST BORDER=0>

        By default, "LIST]" is parsed as a boolean attribute, not as part of the ALT value as was clearly intended. This is also what Mozilla sees.

        The official behaviour is enabled by enabling this attribute. If enabled, it will cause the tag above to be reported as text since "LIST]" is not a legal attribute name.

        "What may be possible" and "what's provided by this node" are -- however -- two different things. :-\

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom