Re^3: Converting HTML special entities to XML

They should always be expanded to UTF-8 and escaped on output. Your HTML parser should just give you Unicode, and whatever XML generator you use should be escaping it automatically for you as appropriate for the target encoding.

Don't attempt to transcode entities and what manually to insert literal bytes into the output XML stream. That way lies madness (and a lot of buggy code; most code dealing with XML out there is quite broken with regard to encodings).

Makeshifts last the longest.

Comment on Re^3: Converting HTML special entities to XML

Replies are listed 'Best First'.
Re^4: Converting HTML special entities to XML by iburrell (Chaplain) on Sep 02, 2004 at 16:38 UTC
It really depends on what kind of processing you are doing. Dealing with the unescaped characters is the safest approach but it requires dealing with charset issues, making sure the output is escaped properly. Dealing with the escape text, in its native charset, is simpler. Character references can help because you don't need to worry about character sets for them; they are always Unicode. In fact, they are the safest way to get Unicode characters in a document with all the charset mangling that goes on.	[reply]
Re^5: Converting HTML special entities to XML by Aristotle (Chancellor) on Sep 02, 2004 at 17:24 UTC
All that is why I'm giving the recommendation that I'm giving. `:-)` You can work with escaped characters avoid going through Unicode, if you wish; but it is hard to get that really right and most people don't. That's why I assert that you should not work with the HTML directly and should not work with the XML directly. It's safest to think of HTML and XML not as a data format, but as an opaque serialization of a data structure. You ask one deserializer for the data structure, and get something unambiguous (ie, Unicode) that you can work to your heart's content with; then you give the still unambiguous result to another serializer that produces conforming output for you. As I said, you can do it differently. Just as you can avoid using strict. It's jut much easier to not shoot yourself in the foot if you stick to that practice. Makeshifts last the longest.	[reply]