in reply to Matching ampersands that are NOT part of an HTML entity?

I recognize what kind of job you are trying to do, as I've been in a similar situation myself.

I do recommend that you not only check that you have something that looks like a named entity, but that you check that the name you found is actually an actual entity name.

Here is the list of entity names that I currently use, I don't claim to be sure that it is perfect and complete, but it'll be pretty close.

my @names = qw( amp lt gt apos quot nbsp iexcl cent pound curren yen brvbar sect uml copy ordf laquo not shy reg macr deg plusmn sup2 sup3 acute micro para middot cedil sup1 ordm raquo frac14 frac12 frac34 iquest Agrave Aacute Acirc Atilde Auml Aring AElig Ccedil Egrave Eacute Ecirc Euml Igrave Iacute Icirc Iuml ETH Ntilde Ograve Oacute Ocirc Otilde Ouml times Oslash Ugrave Uacute Ucirc Uuml Yacute THORN szlig agrave aacute acirc atilde auml aring aelig ccedil egrave eacute ecirc euml igrave iacute icirc iuml eth ntilde ograve oacute ocirc otilde ouml divide oslash ugrave uacute ucirc uuml yacute thorn yuml OElig oelig Scaron scaron Yuml fnof circ tilde ensp emsp thinsp zwnj zwj lrm rlm ndash mdash lsquo rsquo sbquo ldquo rdquo bdquo dagger Dagger permil lsaquo rsaquo euro Alpha Beta Gamma Delta Epsilon Zeta Eta Theta Iota Kappa Lambda Mu Nu Xi Omicron Pi Rho Sigma Tau Upsilon Phi Chi Psi Omega alpha beta gamma delta epsilon zeta eta theta iota kappa lambda mu nu xi omicron pi rho sigmaf sigma tau upsilon phi chi psi omega thetasym upsih piv bull hellip prime prime oline frasl weierp image real trade alefsym larr uarr rarr darr harr crarr larr uarr rarr darr harr forall part exist empty nabla isin notin ni prod sum minus lowast radic prop infin ang and or cap cup int there4 sim cong asymp ne equiv le ge sub sup nsub sube supe oplus otimes perp sdot lceil rceil lfloor rfloor lang rang loz spades clubs hearts diams );
Note that the entity names are case sensitive. Oh, and also: some entity names contain digits, for example "frac14". Your code doesn't account for those.

Also note that I included "'", which is a legal XML entity, but actually not a legal HTML entity. So, if you encounter it, you ought to replace it by its numerical entity: "'".

There are various ways to detect if a name is in a set:

Here is a proposal of some code, using a hash to detect if a name is a proper name. If a proper entity is matched, a missing semicolon at the end will be added.

s/&((?:(#\d+|0x[\da-f]+)|([a-z0-9]+));?)?/ $2 ? "&$2;" # numerical : $3 && $isname{$3} ? ($3 eq 'apos' ? "'" : "&$3;") # named : ($1 ? "&$1" : "&") # not an entity /gie;