comment on

I used the following code to get the words from HTML. The javascript removal came from Re: HTML::Strip Problem.

my $text = "";

# HTML::Parser is broken on Javascript and styles (it leaves
# it in the text).
#
# my $parser = HTML::Parser->new(
#       'text_h' => [ sub { $text .= " " . shift; }, "dtext" ]
# );

my $parser = HTML::Parser->new(
       start_h => [ sub{ $_[0]->{text}.=' '; $_[0]->{skip}++ if $_[1] 
+eq 'script' or $_[1] eq 'style'; } , 'self,tag' ],
       end_h   => [ sub{ $_[0]->{skip}-- if $_[1] eq '/script' or $_[1
+] eq '/style'; } , 'self,tag' ],
       text_h  => [ sub{ $_[0]->{text}.=$_[1] unless $_[0]->{skip}}, '
+self,dtext' ]
);

$parser->parse($html_in);
$text = $parser->{text};
[download]

In reply to Re: Extracting keywords from HTML by superfrink
in thread Extracting keywords from HTML by wfsp

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


Perl: the Markov chain saw
	PerlMonks