Extract Web page contents

I sometimes find myself needing to extract the HTML from between the body tags of a document. The following function takes a filename, file handle, or a scalar reference (with the scalar containing the HTML).

use HTML::TokeParser::Simple 1.4;

print get_contents( $some_html );

sub get_contents {
    my $file = shift;
    my $parser = HTML::TokeParser::Simple->new( $file ) 
        or die "Cannot create parser for ($file): $!";
    my $html = '';
    1 while ! $parser->get_tag->is_start_tag('body');
    while ( my $token = $parser->get_token ) {
        last if $token->is_end_tag('body');
        $html .= $token->as_is;
    }
    return $html;
}
[download]

Comment on Extract Web page contents Download Code

Replies are listed 'Best First'.
Re: Extract Web page contents by jdporter (Paladin) on Dec 28, 2002 at 05:42 UTC
`use HTML::TreeBuilder; sub get_contents { HTML::TreeBuilder ->new_from_file($_[0]) ->find_by_tag_name('body') ->as_HTML }` [download] jdporter ...porque es dificil estar guapo y blanco.	[reply] [d/l]
Re: Extract Web page contents by gmpassos (Priest) on Dec 27, 2002 at 11:27 UTC
To get the data between BODY, is more easy to do in this way: `my ($html_body) = ( $html_data =~ /<body.?>(.?)<\/body>/gsi );` [download] But the use of HTML::TokeParser::Simple can be very helpful for more complex things! ;-P Graciliano M. P. "The creativity is the expression of the liberty".	[reply] [d/l]
Re^2: Extract Web page contents by Aristotle (Chancellor) on Dec 27, 2002 at 22:09 UTC
`<body onLoad="alert('>> it's broken! <<')">` Makeshifts last the longest.	[reply] [d/l]
Re: Re^2: Extract Web page contents by Ovid (Cardinal) on Dec 28, 2002 at 19:52 UTC
I used to work with a guy who deliberately put stuff like that in hit HTML to mess with people who didn't do HTML parsing correctly :) Cheers, Ovid New address of my CGI Course. Silence is Evil (feel free to copy and distribute widely - note copyright text)	[reply]
Re: Re: Extract Web page contents by vek (Prior) on Dec 27, 2002 at 13:21 UTC
Parsing *ML with a regex eh? Shame on you :-) -- vek --	[reply]
Re: Re: Extract Web page contents by gmpassos (Priest) on Dec 27, 2002 at 13:47 UTC
ehehehe! I don't have shame to be economic! ;-P I just think that to just get the body we don't need a fat parser! But for complex things the parser is a bless, and the prevoius node show in a simple way how to use it! Graciliano M. P. "The creativity is the expression of the liberty".	[reply]