Between the 5.8.1 Encode module and the various conversion tools available from other sources, I would actually tend to prefer the former, especially for Arabic. Last time I tried iconv to go from (e.g.) iso-8859-6 to utf8, it took the liberty of converting ascii digits to "Arabic script" digits, where this is neither justified nor desirable. (The original 5.8.0 encoding table did the same thing, but they fixed it in 5.8.1. For that matter, maybe iconv has been fixed since then as well -- I have actually seen a few different versions of iconv in operation, whose character-set inventories seemed oddly different.)

My main point is, be careful when doing character-encoding coversion on any non-European langauge; command-line utils (iconv, etc) may perform some replacements that are inappropriate, and yield more "?" (no-such-character) outputs than you would expect -- and sometimes this will be due to unexpected properties of the input data.

Encode.pm might do the same in some cases, when left to its default behavior, but at least you have the ability to change its behavior, and you can create and use alternate character mapping tables if necessary. (Check out "perldoc enc2xs".)

In reply to Re: Character set conversions by graff
in thread Character set conversions by ottaky

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.