in reply to Matching/replacing a unicode character only works after decode()
With the line-feed added at the end, the file was 5 bytes -- just what I would expect for those two characters, given that they are encoded in utf8 (two bytes per character).cat > /tmp/junk.txt μ ^D
If I use any given character-encoding conversion process on that file, and convert from utf8 to iso-8859-1 (or cp1252, which is roughly equivalent), I get the expected 3-byte result:
If you look up those two byte values in a Latin-1 table, you'll see that they are the "Latin capital letter I with circumflex" and the "vulgar fraction one quarter"; but when that two-byte sequence is interpreted as a utf8 character, it turns out to be U+03BC, "Greek small letter mu".0xCE 0xBC 0x0A # now 1-byte/char, the line-feed is unchanged
|
|---|