in reply to Outputting Unicode to DOS

Please don't use encoding: it's deprecated.

Instead, either don't recode anything (and store the file in the same character encoding as your terminal uses, likely cp850, check the output of chcp command), or do store your program in UTF-8 and use utf8 (thus your text is stored as characters and you are able to perform unicode-related string operations) and encode the strings you print back to bytes (the characters have to be stored in some encoding, thus if you do not encode them, Perl warns and outputs latin1 or UTF-8), possibly with the help of Encode::Locale: 

use utf8;
use Encode 'encode';
my $str = "abc123äöüß";
print encode cp850 => $str;

[download]
use utf8;
use Encode; # explicitly use for its binmodes
use Encode::Locale;
binmode STDOUT, ":encoding(console_out)";
my $str = "abc123äöüß";
print $str;

[download]
(code is untested; you can also use encode "locale", $unicode_string and binmode STDOUT, ":encoding(cp850)")

See also: perlunitut

Replies are listed 'Best First'.
Re^2: Outputting Unicode to DOS
by thekestrel (Friar) on Aug 28, 2015 at 12:05 UTC
    Perhaps I spoke too soon.. I thought if I fixed printing to the console then it would then be passed correctly to ImageMagick for processing... 
    use utf8;
use Encode 'encode';

my $str = "convert -size 100x25 -background white -fill black -pointsi
+ze 25 label:ÄÖÜß ÄÖÜß.gif\n";

print encode cp850 => $str;
system encode cp850 => $str;

    [download]
    The console (now) correctly outputs the string I want to the console... 
    convert -size 100x25 -background white -fill black -pointsize 25 label
+:ÄÖÜß ÄÖÜß.gif

    [download]

    (This command simply creates a new image file called ÄÖÜß.gif with the text ÄÖÜß written as text in the image)
    ... however the file it create is named wrong and the content is wrong (both Ž™šá). If I type that command exactly on the command line it works. I can only assume that I'm getting the encoding confused as it passes from perl through the shell into ImageMagick?

    I tried open a file handle with the mode set to "|-" to write directly to ImageMagick skipping the shell, but no dice. If you could impart some more wisdom, it would be appreciated.

    Paul.
[reply]
[d/l]
[select]

      (This command simply creates a new image file called ÄÖÜß.gif with the text ÄÖÜß written as text in the image) ... however the file it create is named wrong and the content is wrong (both Ž™šá).
      Ž™šá is exactly what happens if ÄÖÜß is encoded to cp850 ("OEM" encoding on Windows, used in console) and then wrongly decoded as cp1252 ("ANSI" encoding on Windows, used in ANSI versions of WinAPI).

      I think that for system you'll need Encode::Locale's "locale" encoding, as opposed to "console_out": while text for console input/output should be encoded to OEM encoding (CP850 on a German system, CP866 on Russian one), file names and commands for system should be encoded in ANSI character set (CP1252 on a German system, CP1251 on Russian, etc.). You can also try to search for Unicode-related WinAPI wrappers for Perl (Does Win32::Unicode work? Is Win32::Process Unicode-aware?), but I can't give any advice on them.

      In conclusion, try: 

      use utf8;
use Encode 'encode';

my $str = "convert -size 100x25 -background white -fill black -pointsi
+ze 25 label:ÄÖÜß ÄÖÜß.gif\n";

print encode cp850 => $str;
system encode cp1252 => $str;

      [download]
      (or: use Encode::Locale; print encode console_out => ...; system encode locale => ...; so your program is portable across different locales in Windows, though finding a way to use W-suffixed WinAPI functions would be better)

      Since you are using ImageMagick, you can try its Perl binding, although getting Strawberry Perl to build it was not an easy task last time I tried it.

[reply]
[d/l]
[select]
[reply]
Re^2: Outputting Unicode to DOS
by thekestrel (Friar) on Aug 28, 2015 at 06:39 UTC
    Thanks for the reply. Both snippets work like a charm =). Much appreciated.
[reply]

In Section Seekers of Perl Wisdom