in reply to Re: Perl Character Set
in thread Perl Character Set

Sorry, I'm not sure about encoding. All I can say is that the perl code is save as plain text and user input is via a basic html <input type="text"> box inside a <form> with no enctype specified. The HTML is in latin1 ISO-8859-1.

As for entering directly into the Database, as I said, entering from the console produces prefect results, therefore it is nothing to do with the database as such, just the way in which the data gets to it via Perl and DBI.

Replies are listed 'Best First'.
Re^3: Perl Character Set
by Aristotle (Chancellor) on Nov 01, 2005 at 20:12 UTC

    The point about the source code is to do with your test code. What ends up in the database depends on a) the encoding of the source file and b) the locale your perl runs under. If Perl assumes the source is in a locale-specific 8-bit encoding, it will take the string’s bytes verbatim from the source code and put them in a non-UTF-8 string. Which means the bytes you have in your source would end up in the database.

    But it’s a different question what happens when you stuff user input there. That too depends on what locale Perl is running under (mostly, whether it’s UTF-8), but the other factor is which encoding the user’s input had, which is clearly not necessarily the same as the encoding of your source code.

    And whether entering text directly into the database from the console works correctly depends on what encoding your terminal emulator produces your input as, and what locale the commandline database utility runs under.

    (Can you tell why I so wish everything was UTF-8 already? Layers upon layers, this is an onion of opportunities to screw up…)

    Makeshifts last the longest.

Re^3: Perl Character Set
by radiantmatrix (Parson) on Nov 01, 2005 at 19:43 UTC

    The DB console application may be running with a different encoding than your Perl application, and/or it may be telling the DB explicitly what encoding it is using.

    I would suggest (1)setting your HTML form's enctype explicitly, and (2)learning how to tell your DB that you are speaking a specific encoding. The latter might be a SQL statement, an option to your DBI->connect, or possibly some combination of the two.

    It would help, a little, to know which DBD you're using.

    <-radiant.matrix->
    A collection of thoughts and links from the minds of geeks
    The Code that can be seen is not the true Code
    "In any sufficiently large group of people, most are idiots" - Kaa's Law