in reply to Cyrillic to plain text

What do you mean when you say you can't open the converted data file as plain text? That is, what operation are you using on the converted data that is failing? What, exactly, are you expecting, and what, exactly, is actually happening?

If the data consists of Cyrillic characters, then it is "plain text in Cyrillic" (what else do you expect it to be?), and any process or application that can handle that sort of data should have no problem with it. But if you are using some process that isn't set up to handle Cyrillic characters -- or that expects some different character encoding than what your data file uses (e.g. utf8 vs. utf16 or iso-8859-5 or koi8-r or cp1251 or ...) -- then this is bound to do something strange.

Replies are listed 'Best First'.
Re^2: Cyrillic to plain text
by Arba (Initiate) on Oct 29, 2005 at 20:04 UTC
    I am trying to use HTML::TokeParser to convert a .html file with Cyrillic characterset to .txt file. It works, but when I open this (new) file with notepad (or textpad) there is no readable text. When I open the file with MsWord (there is a file-conversion dialog box) as plain text - it's the same situation, but when the Cyrillic encoding is used - it's there (the readable text). what I want is to open this .txt file with notepad and to have that readable text. P.S.I am an absolute beginner in Perl.
      when the Cyrillic encoding is used - it's there (the readable text). what I want is to open this .txt file with notepad and to have that readable text.

      It sounds like the real question is: "How do I get Notepad to treat and display a plain text file using my Cyrillic encoding?" If there were such a thing as "notepadmonks.org", this question would be for them, not for us.

      Based on what you have said, MS-Word lets you specify what character encoding to use when opening/displaying the file, and when you do this, it actually shows the text the way you want/expect. If notepad can't do this, then it's a problem with notepad, not a problem with how perl is creating the text file.

      If you are able to view/edit Cyrillic text data in notepad with other text files, then the problem is simply that notepad is using one specific character encoding for Cyrillic, and this encoding is different from the one that perl is writing when it outputs the plain text file, and different from the encoding that you specify when you load the plain text into MS-Word.

      In that case, figure out which encoding notepad uses for Cyrillic, and change the perl script to output the text file in that particular character set.

      Try this: create a simple Cyrillic plain-text file using notepad and your keyboard, then use MS-Word to open that file. What character-encoding do you need to select in MS-Word in order to display this test file correctly? And when you opened your perl script's output in MS-Word, which encoding did you choose that time?

      If your notepad output file and your perl output file are using two different encodings, change your perl script so that it will output the file using the encoding that notepad uses. It should be a simple matter of changing the "open" statement that you use in perl for the output file:

      my $filename = 'some_filename.txt' my $output_encoding = 'whatever_it_is_that_notepad_wants'; open( OUT, ">:encoding($output_encoding)", $filename ) or die "$filena +me: $!"; # or, if you are just printing to STDOUT: binmode( STDOUT, ":encoding($output_encoding)" );
      (In case it helps, perl uses names like "koi8-r", and "cp1251", among others, for non-unicode Cyrillic character encodings. Refer to Encode to see how to list all available character sets.)

      (updated code snippet to use a single variable name consistently for "filename")