Re^4: Decoding Russian text

For interactively exploring the character encodings of text, I like BabelPad. It's a Unicode text editor, but it recognizes and automatically detects many legacy encodings.

No one has mentioned the Perl modules Encode::Guess (core) or Encode::Detect (CPAN) yet.

The Cyrillic text is most likely in one of the encodings KOI8-R, Windows-1251, or ISO 8859-5. (Probably KOI8-R, but that's just a guess.)

Jim

Comment on Re^4: Decoding Russian text

Replies are listed 'Best First'.
Re^5: Decoding Russian text by ikegami (Patriarch) on Jul 14, 2011 at 02:07 UTC
Encode::Guess says it can't distinguish between single-byte encodings. I'll definitely start mentioning Encode::Detect which I hadn't heard of before.	[reply]
Re^6: Decoding Russian text by Jim (Curate) on Jul 14, 2011 at 19:27 UTC
I thought to mention the Perl encoding detection modules specifically because your script probably doesn't distinguish between single-byte encodings such as KOI8-R and Windows-1251. (Or does it?) vit seems to be asking for automated encoding detection in Perl that can at least distinguish between the common Cyrillic legacy encodings, which are single-byte encodings. In the general case, this is a sticky wicket. Needless to say, character encoding guessing is guesswork. And disambiguating characters in single-byte encodings is much more guess-y than it is in multi-byte encodings. (For fun, read Bush hid the facts.) Jim	[reply]
Re^7: Decoding Russian text by ikegami (Patriarch) on Jul 14, 2011 at 20:13 UTC
I thought to mention the Perl encoding detection modules specifically because your script probably doesn't distinguish between single-byte encodings such as KOI8-R and Windows-1251. It does, but it relies on human inspection instead of mechanical inspection. Both approaches are useful, but they have different merits and different use cases.	[reply]