Character re-map using regex

zemplen has asked for the wisdom of the Perl Monks concerning the following question:

I am working on an application that uses three character sets (Basic Latin, Latin 1 Supplement, Latin Extended and Cyrillic or ISO 8859-1, 2 and 5). Because the data that was given to me was entered using a non-standard fonts, characters have to be mapped to the Unicode standard. Although this re-map works for me I have found a case where it does not and I see no reason for the failure.

use strict;
use utf8;
${^WIDE_SYSTEM_CALLS} = 1;

no warnings 'utf8';

binmode(STDOUT, ":utf8");

my $text = chr(0x95);# usually a string read in from a file.
print ord ($text), "$text\n";
$text=~s/"\x{0095}"/"\x{017E}"/g;
print ord ($text), "$text\n";
[download]

Comment on Character re-map using regex Download Code

Replies are listed 'Best First'.
Re: Character re-map using regex by shenme (Priest) on Nov 16, 2003 at 16:57 UTC
The double-quote characters were 'seen' when used inside the s///. When I changed the statement to: `$text=~s/\x{0095}/\x{017E}/g;` [download] your code worked. Note that if you are displaying the output in a console window it won't look like what you want. To 'see' the correct output (small cap Z with caron) I had to start up Word, ask for encoded text, then specify UTF-8 coding. (whew)	[reply] [d/l]