mattdeans has asked for the wisdom of the Perl Monks concerning the following question:

Hi,guys How to get the Unicode of a character in perl? Something like JavaScript's charCodeAt provide: The charCodeAt() method returns the Unicode of the character at the specified index in a string. Thanks.
  • Comment on How to get the Unicode of a character in perl?

Replies are listed 'Best First'.
Re: How to get the Unicode of a character in perl?
by davido (Cardinal) on Sep 02, 2011 at 04:53 UTC

    Modern Perls (the more modern the better) have mostly native Unicode support. Be sure to use feature qw/unicode_strings/;, and you may need to binmode FH, ':utf8'; your input and output filehandles (STDOUT, for example) so as not to have "wide character" warnings and problems. That said, chr and ord should just work as you wish, even with Unicode.

    For more elaborate dealings, you may need one of the Encode modules, or Unicode::Collate, Unicode::UCD, Unicode::Normalize, or other modules.

    There is a lot of information out there in Perl's documentation. perlunicode, perluniintro, perlunifaq, perlunitut to name a few. I'm sure that Programming Perl (4th Edition) (due to market in December) will shed additional light on the topic, especially since tchrist is listed as one of the authors. He seems to be waving the Perl Unicode flag a lot these days, so I imagine he will have a lot to contribute on the subject when the book is released. The book Modern Perl deals with Unicode to some degree, and there's a section on it in the Perl Cookbook as well, although the Cookbook was written before Perl really passed puberty with respect to Unicode.

    Be glad you're dealing with Perl. Few languages have such a degree of native Unicode support as modern versions of Perl. C++, for example, requires the 3rd party ICU library (or something similar) just to get part of the functionality that ships with Perl 5.14.


    Dave

Re: How to get the Unicode of a character in perl?
by CountZero (Bishop) on Sep 02, 2011 at 05:06 UTC
    Easy, just the same as you would get the ascii-code of a character, the ord-function gets you its Unicode.

    use Modern::Perl; use charnames ':full'; binmode STDOUT, ':utf8'; my $unicode = "\N{WHITE SMILING FACE}"; say sprintf '%s has unicode %X', $unicode, ord($unicode);

    Output:

     has unicode 263A

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Re: How to get the Unicode of a character in perl?
by moritz (Cardinal) on Sep 02, 2011 at 06:40 UTC
    The Unicode of a character doesn't make sense to me -- Unicode is a concept, a collection of characters and the name of a consortium, not a property.

    There's the number of the codepoint, which you can get from ord. The name of a codepoint can be found with charnames::viacode(ord($character)).

    There are a lot of Unicode properties, which can be queried with regexes:

    if ($char =~ /\p{Sm}/) { print "'$char' is a Math Symbol\n"; }
Re: How to get the Unicode of a character in perl?
by Anonymous Monk on Sep 02, 2011 at 07:10 UTC
    Tutorials: perlunitut: Unicode in Perl
    #!/usr/bin/perl -- use strict; use warnings; use Data::Dumper; use Unicode::UCD qw' charinfo '; my $k = chr 12794; # http://en.wikipedia.org/wiki/Mu_%28kana%29 print Dumper( { ord => ord $k, chr => $k, charinfo => charinfo(ord $k), }), "\n"; __END__ $VAR1 = { 'charinfo' => { 'digit' => '', 'bidi' => 'L', 'category' => 'Lo', 'code' => '31FA', 'script' => 'Katakana', 'combining' => 0, 'upper' => '', 'name' => 'KATAKANA LETTER SMALL MU', 'unicode10' => '', 'decomposition' => '', 'comment' => '', 'mirrored' => 'N', 'lower' => '', 'numeric' => '', 'decimal' => '', 'title' => '', 'block' => 'Katakana Phonetic Extensions' }, 'chr' => "\x{31fa}", 'ord' => 12794 };
      strangedoc wrote:
      use strict; use warnings; use Data::Dumper; use Unicode::UCD qw' charinfo '; my $k = chr 12794; # http://en.wikipedia.org/wiki/Mu_%28kana%29 print Dumper( { ord => ord $k, chr => $k, charinfo => charinfo(ord $k), }), "\n"; __END__ $VAR1 = { 'charinfo' => { 'digit' => '', 'bidi' => 'L', 'category' => 'Lo', 'code' => '31FA', 'script' => 'Katakana', 'combining' => 0, 'upper' => '', 'name' => 'KATAKANA LETTER SMALL MU', 'unicode10' => '', 'decomposition' => '', 'comment' => '', 'mirrored' => 'N', 'lower' => '', 'numeric' => '', 'decimal' => '', 'title' => '', 'block' => 'Katakana Phonetic Extensions' }, 'chr' => "\x{31fa}", 'ord' => 12794 };

      Just to demonstrate how much nicer to look at, and more convenient all around, that the CPAN Data::Dump module is for actually dumping out data than the woefully standard Data::Dumper module is, here is the same code written to use the other module:

      #!/usr/bin/env perl use strict; use warnings; use Unicode::UCD qw(charinfo); use Data::Dump; my $k = chr 12794;  dd {     ord         =>          ord $k,     chr         =>              $k,     charinfo     => charinfo(ord $k), };

      And here is the lovely output:

      {   charinfo => {     bidi          => "L",     block         => "Katakana Phonetic Extensions",     category      => "Lo",     code          => "31FA",     combining     => 0,     comment       => "",     decimal       => "",     decomposition => "",     digit         => "",     lower         => "",     mirrored      => "N",     name          => "KATAKANA LETTER SMALL MU",     numeric       => "",     script        => "Katakana",     title         => "",     unicode10     => "",     upper         => "",   },   chr => "\x{31FA}",   ord => 12794, }

      Isn’t that a lot easier on hand and eye?

Re: How to get the Unicode of a character in perl?
by BrowserUk (Patriarch) on Sep 02, 2011 at 08:10 UTC

    You can code an effective equivalent of charCodeAt() so:

    sub charCodeAt{ return ord substr $_[0], $_[1], 1; }

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.