Re: LATIN1 and utf8 strings, joined, mangle the LATIN1

$str contains "weight_\x{2639},weight_\x{c2}", what I would expect from joining "weight_\x{2639}" and "weight_\x{c2}". All I see missing is where you encod your output

# Encode input from/output to STDIN/STDOUT/STDERR as per locale
use open ':std', ':locale';
[download]

# Force UTF-8 input/output for STDIN/STDOUT/STDERR
use open ':std', ':encoding(UTF-8)';
[download]

You'll get better diagnostic output from Devel::Peek's Dump. For example, Dump($str) gives

SV = PV(0x81e829c) at 0x814ecdc
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK,UTF8)
  PV = 0x81fe838 "weight_\342\230\271,weight_\303\202"\0 [UTF8 "weight
+_\x{2639},weight_\x{c2}"]
  CUR = 20
  LEN = 24
[download]

You see both the external and internal representation.

Comment on Re: LATIN1 and utf8 strings, joined, mangle the LATIN1 Select or Download Code

Replies are listed 'Best First'.
Re^2: LATIN1 and utf8 strings, joined, mangle the LATIN1 by brycen (Monk) on Feb 09, 2009 at 21:15 UTC
In the use case, the data is not really output. The joined string is inserted back into the database (as noted above, mangling the output). This application was written prior to perl's support for unicode and/or utf-8.	[reply]
Re^3: LATIN1 and utf8 strings, joined, mangle the LATIN1 by ikegami (Patriarch) on Feb 09, 2009 at 21:33 UTC
So it's being output to the database instead of the screen. `use open` won't help you, but it still needs to be encoded. Sorry, I don't think I can help you because I don't know enough about how database handle encodings, what the DBD expects from you and what the DBD can do for you.	[reply] [d/l]