Re^3: Why Doesn't Text::CSV_XS Print Valid UTF-8 Text When Used With the open Pragma? ("XS")

Your unexpected output seems ISO-8859-1 output of the SPADE charcters. Probably, If you put the output to the text, and See the results in your browser with utf-8 encoding, You'see the SPADE.

print qq("BLACK SPADE SUIT","BLACK HEART SUIT","BLACK DIAMOND SUIT","B
+LACK CLUB SUIT",\n);

#decimail unicode character for above;
my @ary=("&#9824;","&#9829;","&#9830;","&#9827;");
foreach my $target(@ary) {
    $target =~ s/\&#(.*);/$1/;
    print '"' . encode('utf8', chr($target)) . '",';
}
print "\n";
[download]

I mean , this is terminal problem , doesn't it ?

Comment on Re^3: Why Doesn't Text::CSV_XS Print Valid UTF-8 Text When Used With the open Pragma? ("XS") Download Code

Replies are listed 'Best First'.
Re^4: Why Doesn't Text::CSV_XS Print Valid UTF-8 Text When Used With the open Pragma? ("XS") by Anonymous Monk on Oct 02, 2011 at 08:33 UTC
I mean , this is terminal problem , doesn't it ? No. It looks similar but no. The problem, in a nutshell, if you use `warn "$ARGV $_ " for PerlIO::get_layers(*ARGV)` you can see ARGV doesn't get utf8 io layer, only STDIN gets them `$ perl ... utf8wobom.csv >bad utf8wobom.csv unix at ... utf8wobom.csv crlf at ... $ perl ... < utf8wobom.csv >good - unix at ... - crlf at ... - encoding(utf-8-strict) at ... - utf8 at ...` [download] In my non-utf terminal it shows `$ ls -loanh good bad -rw-rw-rw- 1 0 115 2011-10-02 01:31 bad -rw-rw-rw- 1 0 103 2011-10-02 01:31 good $ diff good bad 2c2 < "ΓÖá","ΓÖÑ","ΓÖª","ΓÖú" --- > "├ó┬Ö┬á","├ó┬Ö┬Ñ","├ó┬Ö┬ª","├ó┬Ö┬ú"`	[reply] [d/l] [select]
Re^5: Why Doesn't Text::CSV_XS Print Valid UTF-8 Text When Used With the open Pragma? ("XS") by remiah (Hermit) on Oct 02, 2011 at 14:38 UTC
Probably, Text::CSV::Encoded is what you are looking for. `use Text::CSV::Encoded; my $csv = Text::CSV::Encoded->new ({binary=>1, encoding=>"utf8"}) or d +ie $!; while (my $row = $csv->getline (ARGV)) { $csv->print(\STDOUT, $row); }` [download] This works fine with my perl 5.12.2, with command line ... `perl test.pl test.csv` [download]	[reply] [d/l] [select]
Re^6: Why Doesn't Text::CSV_XS Print Valid UTF-8 Text When Used With the open Pragma? ("XS") by Tux (Canon) on Oct 02, 2011 at 15:52 UTC
Text::CSV::Encoded is indeed a nice extension to the Text::CSV parser family, but it does solve the underlying problem as stated in the original quest. `use encoding ...` is (or now was) broken in combination with XS code. Most bugs that stem from that are workaround-able by using other encode/decode approaches, but it is (or was) still a bug. Enjoy, Have FUN! H.Merijn	[reply] [d/l]