in reply to Re^5: Japanese character in Linux
in thread Japanese character in Linux

is_urt8 is returning nothing and output of unpack is 81698a94816a8bc9976d.

We tried one more function find_encoding of Encode module and output is Encode::XS=SCALAR(0xaad27d0).

W are not getting actual output when we use encode and decode fucntins.

Output of locale command is as below
LANG=C

LC_CTYPE="C"

LC_NUMERIC="C"

LC_TIME="C"

LC_COLLATE="C"

LC_MONETARY="C"

LC_MESSAGES="C"

LC_PAPER="C"

LC_NAME="C"

LC_ADDRESS="C"

LC_TELEPHONE="C"

LC_MEASUREMENT="C"

LC_IDENTIFICATION="C"

LC_ALL=

Please sugegst where we are going wrong.

Thanx,
Prafull

Replies are listed 'Best First'.
Re^7: Japanese character in Linux
by andal (Hermit) on Jul 08, 2011 at 09:56 UTC

    There are quite a few things that are wrong. First of all, with locale "C" it is not possible to see any Japanese text, or actually any text outside of ASCII. So, it looks like you have to change your locale first. Assuming, that you use xterm as your terminal emulator, try to start it this way 

    LC_CTYPE="en_US.UTF-8" xterm &

    [download]
    Then check the output of locale command in that new terminal window. It should produce something like 
    LANG=
LC_CTYPE=en_US.UTF-8

    [download]

    After that, you should convert your text to UTF-8 encoding. Since you haven't provided Japanese text corresponding to the hexdump, and since the hexdump looks like UTF-16, I'll assume, that it is UTF-16. Then you should output the variable using the following: 

    Encode::from_to($your_variable, "UTF-16", "UTF-8");
print $your_variable, "\n";

    [download]
    If this does not produce the result, then you may have some wrong font for terminal emulator :)

    Actually, sending the variable to terminal probably is not the most important thing. You have to work with that value inside of perl. Depending on what you want to do, you have to convert your variables back and forth. For example, to do pattern matching on it, you should first do 

    use utf8;
my $converted = Encode::decode("UTF-16", $your_variable);
$converted =~ /some Japanese text/;

    [download]

[reply]
[d/l]
[select]

      Japanese charctares are iŠ”j‹É—m

      （株）極洋 Hex Dump is 81698a94816a8bc9976d. We are doing sybase to oracle migration PROJECT> We successfully migraed the data to Oracle using Characterset16 in Sql loader file. Now we are finding issue when application is inserting data from Perl. We are not concrened about displaying the Japanese data in console . We want the Japanese data to be able to handle by perl post migration. Please advise. Thanks Praful
[reply]
[d/l]

        Looks like this site can't handle Japanese as well :)

        Anyway, I believe we can assume, that DBD::Oracle does not do any conversion, so the data obtained from the database comes in UTF-16 encoding, so it should be passed to the database in the same encoding.

        Now, that's the goal. How to achieve it depends on what you are actually doing. I've already told you what you should do to use the obtained data in perl regular expressions. If you want to insert some Japanese text into database, then you would do something like this 

        use utf8;
use Encode;
my $data_to_be_inserted = Encode::encode("UTF-16", "Some Japanese text
+");

        [download]

        I assume, we've clarified the issue with terminal output. If you have some other problems, then it is better to describe them in more detailed way, showing the code that does not work. Otherwise, I can only recommend you to read the documents mentioned in the very first answer to your post.

[reply]
[d/l]

In Section Seekers of Perl Wisdom