I am attempting to do a search and replace on the following Unicode encoded file:

[KEY]
һ
ح
د
ؼ
�^
KeyStr=һحدؼ�^
KeyMap=һحدؼ�^
[DICT]
[download]

This file contains both ASCII and UTF-16 LE (CJK Unified Ideograph) encoded characters. I have a hash containing the characters to search for, and their replacement string, for example:

my %conv =  (
    'U\+003d'    =>    'key1',        
);
[download]

where the key is my character to search for in the form of a Unicode hex value, and the value is what I need to replace it with in the form of a normal utf8 ascii string (so the above had is saying that the equals sign '=' - hex U+003d - should be replaced by the string 'key1'). I use Unicode::String to get the hex values for each line from my input file. My approach is to slurp the data from the input file into a list of utf16 objects:

while (<F>) {    
    push ( @raw, utf16($_));
}
[download]

my @hexout;
foreach $line ( @raw ) {
        # get the hex values for uthe input line
        my $hvs = $line->hex;
        # now loop over the conversions
        foreach my $hexkey ( keys %keyconv ) {
            # if this char to be replaced is in
            # this line, do a conversion
             if ( $hvs =~ /$hexkey/ ) {
                        # replacement chars are utf8...
                        my $newStr8  = utf8($keyconv{$hexkey});
                        # ...so convert it to utf16
                        my $newStr16 = $newStr8->utf16;
                        my $Str16Obj = utf16($newStr16); 
                        # and get its hex value in utf16 format
                        my $newhex = $Str16Obj->hex; 
                        print "Replaced $hexkey with", 
                           "$newStr (hex: $newhex) in line\n",
                           "(hex: $hvs)\n" if $debug;
                        # do the conversion
                        $hvs =~ s/$hexkey/$newhex/g;
            }
        }
        # maintain a list of output hex values.
        push @hexout, $hvs;
}
[download]

Unicode::String->stringify_as( 'utf16' );
my $out16 = Unicode::String->new();
$out16->hex ( join '', @hexout);
print OUTFILE $out16;
close F || die;
[download]

print OUTFILE chr hex foreach ( split /\s*/, $outhex );
[download]