"can I parse this HTML in a single regex". And the answer is yes!

... with a BIG emphasis on this HTML. HTML is a beast to parse correctly, due to its inheritance from SGML, and due to the error correction / guessing algorithms used in most browsers. Simple regular expressions may work as long as the HTML has a well-known format and does not use too many SGML or encoding tricks.

Just yesterday, I stubled over this nice piece of valid(!) HTML, hand-crafted to defeat most simple-minded string parsers and regular expressions:

    <h1>My-IP-Service</h1>
<h1 class="myip"><!--- >
A comment about the abuse of their service they want to prevent ...
<a href="/netze/tools/whois-abfrage/?rm=whois_formular">nicht ermittel
+bar</a>
<a href="/netze/tools/whois-abfrage/?rm=whois_formular">127.0.0.1</a>
<a href="/netze/tools/whois-abfrage/?rm=whois_formular">198.18.0.15</a
+>
< --><a href="/netze/tools/whois-abfrage/?rm=whois_formular">&#57;<!--
+ >226.180.195.155
< -->&#50;<!-- >
253.159.244.9
< -->&#46;<!-- >
253.239.61.182< -->&#50;<!-- >230.121.254.208
< -->&#50;<!-- >
251.168.157.152
< -->&#52;<!-- >
254.121.189.15< -->&#46;<!-- >
237.24.153.213< -->&#56;<!-- >246.217.119.248
< -->&#46;<!-- >
245.167.107.28
< -->&#49;<!-- >226.204.198.25
< -->&#49;<!-- >
233.167.179.189
< -->&#55;<!-- >
228.193.179.191< --></a></h1>

[download]

(From: http://www.heise.de/netze/tools/ip)

Returning the correct IP address (92.224.8.117 in this case) from this piece of HTML is not impossible, and with enough effort, someone may be able to write a regexp that does the job for this special obfuscation. But with HTML::Parser, it is essentially a no-brainer requiring about 10 lines of code (Sorry, Heise ...). And unless the author finds a way to confuse HTML::Parser without breaking browsers, it will not fail when the author modifies the obfuscation. (Well, changing the H1 tag or its class attribute would break this special implementation.)

#!/usr/bin/perl -w
use strict;
use LWP::Simple;
use HTML::Parser;
my $ip='';
my $wanted=0;
HTML::Parser->new(
    api_version => 3,
    start_h => [sub { $wanted=1 if ($_[0] eq 'h1') && ($_[1]->{'class'
+} eq 'myip') }, 'tagname,attr'],
    end_h   => [sub { $wanted=0 if $_[0] eq 'h1' }, 'tagname'],
    text_h  => [sub { $ip.=$_[0] if $wanted }, 'dtext'],
)->parse(get('http://www.heise.de/netze/tools/ip'));
print "$ip\n";

[download]

So, DON'T use regular expressions to parse HTML or XML. Except perhaps in very special cases where you control how the HTML/XML is generated.

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

In reply to Re^2: Split/Match Question by afoken
in thread Split/Match Question by esmadmin

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.