Regular expressions are a very fragile solution to the "how do I parse HTML" problem.

-David

Thanks for the reply - as it happened (as it so often happens whenever I post asking for help with something) I stumbled upon a regex solution to this problem by accident, but given the majority of people saying regex is not the way to go here, I will definitely look into this and the various other options people suggested

my $test = q{ <a 
href="detail.jsp?key=7147&rc=d_20071128&p=2&pv=1">Next 
Page</a><br/>
<a class="foot"
 href="email.jsp?key=7147">E-mail Story</a> };

print "BEFORE: $test\n\n";

$test =~ s{<a[^>]+>.*?e-?mail[^<]*</a>}{}gim;

print "AFTER: $test\n";
[download]

That strips them outright, but probably you should not use it. It's naive and it will break (like most all regex-based markup parsing). What you want to do is either what erroneousBollock said, or I use XML::LibXML for XHTML. There are many other options in the X(HT)ML space. If you have dirtier HTML, those won't work. You can also do it in a stream with HTML::TokeParser.

The most robust solution is to use HTML::Parser. Regexes are not recommended for parsing HTML although they can be made to work and it is probably a good learning experience.

The . operator should generally be your last choice in a regex. To stay within a tag I would suggest something like this (untested):

s!<a[^>]+href=['"]email[^>]+>[^<]+</a\>!DELETED!gi;

# which becomes this to deal with whitespace issues

s!<\s*a[^>]+href\s*=\s*['"]\s*email[^>]+>[^<]+<\s*/a\s*>!DELETED!gi;
[download]

The key thing we are doing is using the NOT class syntax on the > and < parts of the tags to ensure we match everything but still remain reliably in the tag. The endless \s* are required to deal with the relaxed way HTML deals with whitespace.

does anybody know if any LWP or similar implement DOM

LWP::UserAgent does not provide DOM-level access.

WWW::Mechanize doesn't either, but does parse the HTML for you in order to provide methods like links(), which incidentally, does what you want.

I have a hunch that DOM parsing is cleaner than X(HT)ML parsing

"DOM" is not a manner of parsing, but a manner of access. For methods from the DOM to be able to access data from a tree of nodes, some "parser" code still has to build that tree.

It's certainly cleaner to access data using DOM (or DOM-like) methods, or selector interfaces like XPath or XQuery.

HTML::TreeBuilder::XPath builds an HTML::Tree internally and then provides XPath-like access to that tree.

(HTML parsing) sometimes not being well formed etc

If you're talking about the robustness of parsing HTML, there are many libraries that parse HTML properly even when given invalid input. It's quite orthogonal to how you access the data once you've parsed the document.

-David

parsing vs access methods does get blurry...anyway, in the end we're interested in getting to point B and not that interested in the trip itself...whether parsing or using an access method.
to this point...i think (possibly) best for this problem would be find_link() method of WWWW::Mechanize. not as tedious as Xpath or HTML::Tree etc.
the hardest line to type correctly is: stty erase ^H

#!/usr/bin/perl

use strict;
use warnings;
use HTML::TokeParser::Simple;

my $p = HTML::TokeParser::Simple->new(*DATA);

my ($html, $in_email_link);
while (my $t = $p->get_token){

  $in_email_link++, next if
    $t->is_start_tag(q{a})
      and
    $t->get_attr(q{href})
      and
    $t->get_attr(q{href}) =~ m|email\.|;

  $in_email_link--, next if
    $in_email_link
      and
    $t->is_end_tag(q{a});

  next if $in_email_link;

  $html .= $t->as_is;

}

print qq{$html};

__DATA__
<p>one</p>
<a href="detail.jsp?key=7147&rc=d_20071128&p=2&pv=1">Next Page</a><br/
+>
<p>two</p>
<a class="foot" href="email.jsp?key=7147">E-mail Story</a>
<p>three</p>
<a href="detail.jsp?key=7147&rc=d_20071128&p=2&pv=1">Next Page</a><br/
+>
[download]

<p>one</p>
<a href="detail.jsp?key=7147&rc=d_20071128&p=2&pv=1">Next Page</a><br/
+>
<p>two</p>

<p>three</p>
<a href="detail.jsp?key=7147&rc=d_20071128&p=2&pv=1">Next Page</a><br/
+>
[download]