I've written a web-based content management system using CGI.pm and DBI::MySQL. A site administrator can type text into a <TEXTAREA> box and it gets saved to the database. On a schedule, it then generates JavaScript files with document.write statements that spit back out parts of the text.

The problem is, the main site administrator tends to write her content in MS Word first, then copies and pastes it into my form. Word does all sorts of fun character substitutions, such as curly-quotes, turning -- into an em-dash, changing ... into a single elipses character, etc. I suspect that some of these characters sometimes either break coming into the database (being stored as type blob) or break within the quotes of the document.write in the .js files I'm generating. (I'm escaping regular single and double quotes before puting them into the document.write).

My question is, how does Perl identify these special characters? They're not 7-bit ASCII. Should I use ord on a test sample, then work out a sustitution table on my own? Or is there a module that already handles this for me? (charnames?)

In reply to Special Character Substitutions by cei

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.