in reply to Re^5: Decoding Russian text
in thread Decoding Russian text

I thought to mention the Perl encoding detection modules specifically because your script probably doesn't distinguish between single-byte encodings such as KOI8-R and Windows-1251. (Or does it?) vit seems to be asking for automated encoding detection in Perl that can at least distinguish between the common Cyrillic legacy encodings, which are single-byte encodings. In the general case, this is a sticky wicket.

Needless to say, character encoding guessing is guesswork. And disambiguating characters in single-byte encodings is much more guess-y than it is in multi-byte encodings.

(For fun, read Bush hid the facts.)

Jim

Replies are listed 'Best First'.
Re^7: Decoding Russian text
by ikegami (Patriarch) on Jul 14, 2011 at 20:13 UTC

    I thought to mention the Perl encoding detection modules specifically because your script probably doesn't distinguish between single-byte encodings such as KOI8-R and Windows-1251.

    It does, but it relies on human inspection instead of mechanical inspection. Both approaches are useful, but they have different merits and different use cases.

[reply]

In Section Seekers of Perl Wisdom