comment on

I recognize what kind of job you are trying to do, as I've been in a similar situation myself.

I do recommend that you not only check that you have something that looks like a named entity, but that you check that the name you found is actually an actual entity name.

Here is the list of entity names that I currently use, I don't claim to be sure that it is perfect and complete, but it'll be pretty close.

my @names = qw(
  amp lt gt apos quot nbsp iexcl cent pound curren yen brvbar sect uml
  copy ordf laquo not shy reg macr deg plusmn sup2 sup3 acute micro
  para middot cedil sup1 ordm raquo frac14 frac12 frac34 iquest Agrave
  Aacute Acirc Atilde Auml Aring AElig Ccedil Egrave Eacute Ecirc Euml
  Igrave Iacute Icirc Iuml ETH Ntilde Ograve Oacute Ocirc Otilde Ouml
  times Oslash Ugrave Uacute Ucirc Uuml Yacute THORN szlig agrave
  aacute acirc atilde auml aring aelig ccedil egrave eacute ecirc euml
  igrave iacute icirc iuml eth ntilde ograve oacute ocirc otilde ouml
  divide oslash ugrave uacute ucirc uuml yacute thorn yuml OElig oelig
  Scaron scaron Yuml fnof circ tilde ensp emsp thinsp zwnj zwj lrm rlm
  ndash mdash lsquo rsquo sbquo ldquo rdquo bdquo dagger Dagger permil
  lsaquo rsaquo euro Alpha Beta Gamma Delta Epsilon Zeta Eta Theta
  Iota Kappa Lambda Mu Nu Xi Omicron Pi Rho Sigma Tau Upsilon Phi Chi
  Psi Omega alpha beta gamma delta epsilon zeta eta theta iota kappa
  lambda mu nu xi omicron pi rho sigmaf sigma tau upsilon phi chi psi
  omega thetasym upsih piv bull hellip prime prime oline frasl weierp
  image real trade alefsym larr uarr rarr darr harr crarr larr uarr
  rarr darr harr forall part exist empty nabla isin notin ni prod sum
  minus lowast radic prop infin ang and or cap cup int there4 sim cong
  asymp ne equiv le ge sub sup nsub sube supe oplus otimes perp sdot
  lceil rceil lfloor rfloor lang rang loz
  spades clubs hearts diams
);
[download]

Note that the entity names are case sensitive. Oh, and also: some entity names contain digits, for example "frac14". Your code doesn't account for those.

Also note that I included "'", which is a legal XML entity, but actually not a legal HTML entity. So, if you encounter it, you ought to replace it by its numerical entity: "'".

There are various ways to detect if a name is in a set:

You can use them as hash keys for quick detection

my %isname;
$isname{$_} = 1 foreach @names;
if($isname{$name}) ...
[download]

In perl 5.10 or later, you can use the smart match operator to see if the name is in an array
```
if($name ~~ @names) ...
[download]
```
You can use grep to detect if a name is in an array
```
if(grep $name eq $_, @names) ...
[download]
```
With Regex::PreSuf or a similar module, you can build a fast regular expression:
```
use Regex::PreSuf;
my $re = presuf(@names);
if($name =~ /^($re)$/o) ...
[download]
```
In modern perls, a simple dumb regexp of alternatives separated by "|" is automatically optimized so it'll be quite fast, too
```
my $re = join '|', @names;
if($name =~ /^($re)$/o) ...
[download]
```
A regexp like this can easily be incorporated into the real s/// statement, instead of the generic name matcher, so it will only match real entity names. (Don't forget a /\b/ anchor at the end, to avoid matching partial names.)
You can use a large string of names separated with spaces, plus a space at a front and at the end; and use index to search if a name surrounded by spaces, is in the string
```
my $names = ' ' . join(' ', @names) . ' ';
if(index($names, " $name ") >= 0) ...
[download]
```

Here is a proposal of some code, using a hash to detect if a name is a proper name. If a proper entity is matched, a missing semicolon at the end will be added.

s/&((?:(#\d+|0x[\da-f]+)|([a-z0-9]+));?)?/
  $2 ? "&$2;"  # numerical
     : $3 && $isname{$3} ? ($3 eq 'apos' ? "&#39;" : "&$3;")  # named
                         : ($1 ? "&amp;$1" : "&amp;")  # not an entity
/gie;
[download]

In reply to Re: Matching ampersands that are NOT part of an HTML entity? by bart
in thread Matching ampersands that are NOT part of an HTML entity? by Anonymous Monk

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		`&`
	<		`<`
	>		`>`
	[		`[`
	]		`]`

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.