my $open = qr{
  < [a-zA-Z][a-zA-Z0-9]*
  (?:
    \s+ \w+
    (?: \s* = \s*
      (?: "[^"]*" | '[^']*' | [^\s>]* )
    )?
  )*
  \s*
  >
}x;
[download]

my $close = qr{
  < / \s* [a-zA-Z][a-zA-Z0-9]* \s* >
}x;
[download]

# the following are comments:
# <!-- ab -- cd -->   <!-- ab -->    <!---->
# <!-- ab -- cd -- >  <!-- ab -- >   <!---- >

my $comment = qr{
  <!--              # <!--
  [^-]*             # 0 or more non -'s
  (?:               
    (?! -- \s* > )    # that's not --, space, then >
    -                 # a -
    [^-]*             # 0 or more non -'s
  )*                # 0 or more times
  -- \s* >          # --, space, then >
}x;
[download]

while ($HTML =~ /\G($open|$close|$comment|[^<]+)/g) {
  # do something with $1
}
[download]

# requires the (?{...}) structure
use re 'eval';

while ($HTML =~ m{
  \G
  (
    $open    (?{ $STATE = 'open'    }) |
    $close   (?{ $STATE = 'close'   }) |
    $comment (?{ $STATE = 'comment' }) |
    [^<]+    (?{ $STATE = 'TEXT'    })
  )
}xg) {
  # do something with $1 and $STATE
}
[download]

I love that tree comment. It is a classic. :-)

The tree structure involves such details as the fact that tags must balance and can involve nested tables. Matching nesting with a single regular expression is impossible, you cannot even construct an RE that tests whether parentheses balance!

Another major detail you are skipping over is the fact that there are a lot of constructs which are invalid HTML but the browsers attempt to work around. In a real environment if someone can find some broken construct that escapes your match but gets interpreted the right way by some browser, that is a major oops.

Enjoy your RE, but I won't trust it. At some point you need to stop thinking about this as a matching problem which RE's are designed for) and start thinking of this as a parsing problem (which REs are a useful tool for but are only a piece of the puzzle). As soon as you start saying that you are parsing the document, it becomes trivial to be sure that you are not missing anything and you will guarantee that you put out a valid document which cannot cause confusion.

Random thought: Have you thought about how you will catch someone who just puts <-- somewhere? What if someone designs a tag which you strip out - and in the process turn something you skipped because it was an invalid tag into a valid one? How many more of these can you think of, and how are you going to convince me that you have caught them all?

A non-RE solution can easily - demonstrably - result in output that is absolutely guaranteed to be safe. I stand by my doubt that an RE solution will ever achieve that status.