That`s right. Thanx.
The REVERSE ROOFED a (259) is irrelevant only in the sense that the ROOFED i is in UTF-8 form ONLY when another character is in the String, that cannot be expressed in iso-latin-1 (Whatever that character is). Else, it 'falls back' to iso-latin-1 (238).
Would there be a way to tell perl (perhaps in getting the data from the DBD-ODBC 1.14 with unicode support) to keep (force) the data in UTF-8 form... | [reply] |
Noone else posted anything, so I'll give it a quick go...
It seems you're assuming Perl's internal format is UTF-8.
To be sure, it would help if I knew exactly what you were getting from the database. The "PV = " lines printed by following would provide that info.
use Devel::Peek qw( Dump );
Dump($db_field); # Output sent to STDERR
You should get one of the following four output combinations for chr(259) and chr(238).
259: PV = 0x18e914c "\304\203"\0 [UTF8 "\x{103}"]
238: PV = 0x18e914c "\356"\0
259: PV = 0x18e914c "\304\203"\0 [UTF8 "\x{103}"]
238: PV = 0x18ecffc "\303\256"\0 [UTF8 "\x{ee}"]
259: PV = 0x18d73fc "?"\0
238: PV = 0x18d7444 "\356"\0
259: PV = 0x18d736c "\304\203"\0
238: PV = 0x18ecffc "\303\256"\0
259: PV = 0x18d74dc "\304\203"\0 <- Lack of [UTF8 ...]
238: PV = 0x18d74dc "\356"\0
-
259: PV = 0x18e914c "\304\203"\0 [UTF8 "\x{103}"]
238: PV = 0x18e914c "\356"\0
or
259: PV = 0x18e914c "\304\203"\0 [UTF8 "\x{103}"]
238: PV = 0x18ecffc "\303\256"\0 [UTF8 "\x{ee}"]
The data is in Perl's internal text format (called "UTF8", no dash). It can either be in iso-latin-1 or UTF-8. iso-latin-1 is preferred since it's faster, but it can only be used if every character in the string can be represented by iso-latin-1.
Solution: You need to convert from Perl's internal format to a string of bytes.
use Encode qw( encode );
print(encode('UTF-8', $text));
or
binmode(STDOUT, ':encoding(UTF-8)');
print($text);
-
259: PV = 0x18d73fc "?"\0
238: PV = 0x18d7444 "\356"\0
The data is in iso-latin-1. This can't be the case, since "ă" is working.
-
259: PV = 0x18d736c "\304\203"\0
238: PV = 0x18ecffc "\303\256"\0
The data is in UTF-8 already. Just print it out without re-encoding it.
-
259: PV = 0x18d74dc "\304\203"\0 <- Lack of [UTF8 ...]
238: PV = 0x18d74dc "\356"\0
That means the data in your database is bad, or your database is returning bad data.
The problem should be fixed at the source, but I think it's possible to fix it at this point too.
Update: Added second pair of output combinations, and fixed s/utf8/utf-8/
| [reply]
[d/l]
[select] |
Thanks so much for looking into my problem!
What I get when I dump the database data is the third possibility: (in both cases: isolated "î" and "îã")
259: PV = 0x18d736c "\304\203"\0
238: PV = 0x18ecffc "\303\256"\0
That means it is UTF-8 all the time. So I've learned that the data from the base is correct. But the weird thing is: when I print the î to STDOUT (to get it to go to the browser) it is turned into xEE.... In a CMD box it shows as a Euro-sign (could be handy...) and in the browser it doesn't show but as a square, that is: Paletino Linotyope, the TTF that supports a big char repertoire, doesn't like it. If I look in the source of the HTML page FROM the browser it says î. But when it is accompanied by the ã, than it shows...
If I would make a workaround to change \xEE back to \303\256 (and all the rest), I cannot, at that time, see the difference in the two î's - its only in OUTPUTTING, so it seems, that the isolated î is offered at the browser/STDOUT in iso-latin-1 form...
can you shed more light on this?
| [reply] |