Re: How to Fix Character Encoding Damaged Text Using Perl?

If it's really that straightforward, this kind of reverse pipe-line might do–

perl -CSD -MEncode -le '$moji = decode "UCS-2LE", "Re: URGENT"; print $moji; print decode "UTF-8", encode "UCS-2LE", $moji'

敒›剕䕇呎
Re: URGENT

Comment on Re: How to Fix Character Encoding Damaged Text Using Perl? Download Code

Replies are listed 'Best First'.
Re^2: How to Fix Character Encoding Damaged Text Using Perl? by Jim (Curate) on Jun 15, 2013 at 20:48 UTC
Thank you very much, Your Mother! This works wonderfully, and it makes sense given the known cause of the encoding corruption. use v5.16; use utf8; use open qw( :encoding(UTF-8) :std ); use Encode qw( encode decode ); while (my $damaged_text = <DATA>) { chomp $damaged_text; my $repaired_text = decode('UTF-8', encode('UCS-2LE', $damaged_text)); say $repaired_text; } close DATA; exit 0; __DATA__ 敒›剕䕇呎敌馀⁳潧琠⁯韦겜鯥↽ This prints… Re: URGENT Let’s go to 日本国! …as expected. In this case, `use utf8` is required.	[reply] [d/l]

Replies are listed 'Best First'.

Re^2: How to Fix Character Encoding Damaged Text Using Perl?
by Jim (Curate) on Jun 15, 2013 at 20:48 UTC

Thank you very much, Your Mother!

This works wonderfully, and it makes sense given the known cause of the encoding corruption.

use v5.16;
use utf8;
use open qw( :encoding(UTF-8) :std );
use Encode qw( encode decode );

while (my $damaged_text = <DATA>) {
    chomp $damaged_text;

    my $repaired_text = decode('UTF-8', encode('UCS-2LE', $damaged_text));

    say $repaired_text;
}

close DATA;

exit 0;

__DATA__
敒›剕䕇呎
敌馀⁳潧琠⁯韦겜鯥↽

This prints…

    Re: URGENT
    Let’s go to 日本国!

…as expected.

In this case, use utf8 is required.

[reply]
[d/l]