powerhouse has asked for the wisdom of the Perl Monks concerning the following question:

Ok, this may be completely unrelated to Perl, however, since my programming is 70% Perl, I figured I'd ask here in case any of you have ran into this issue and know how you fixed it, I'll also ask at the other places that I use, such as MySql and so forth.

Anyhow, I have built a Language Translator for our club, we can give access to certain member that volunteer to translate and that we trust to do it right and not play with the words on the website. Anyhow, When they login to Russian, they see the language tags and a link to preview the current content, or they can search the tables for exact content to edit, once they find it they click edit and perl then delivers a form which shows them the current value in a textarea form field, so they can edit it also it shows the current english content of that tag below it just on the page so they know what to translate.

Anyhow, after they translate it to what they want, then they hit the save changes button, which Perl then writes to the database, here is that code:
$_ctagValue = $dbh->selectrow_array(qq{select `value` from `$_ +tableName` where `lngId` = ?}, undef, $in{lngId}); $_updatedTime = time(); $_updated = $dbh->do(qq{update `$_tableName` set `value` = ?, +`d` = ? where `lngId` = ?}, undef, qq~$in{tagvalue}~, $_updatedTime, +$in{lngId}); if($_updated) { # Insert Tracking... my($stmt,@place); $stmt .= ',' if $stmt; $stmt .= '`Id` = ?'; push(@place,$_cust_id); $stmt .= ',' if $stmt; $stmt .= '`lang` = ?'; push(@place,$_tableName); $stmt .= ',' if $stmt; $stmt .= '`lngId` = ?'; push(@place,$in{lngId}); $stmt .= ',' if $stmt; $stmt .= '`was` = ?'; push(@place,"$_ctagValue"); $stmt .= ',' if $stmt; $stmt .= '`now` = ?'; push(@place,"$in{tagvalue}"); $stmt .= ',' if $stmt; $stmt .= '`timel` = ?'; push(@place,(time() - $in{timel})); $stmt .= ',' if $stmt; $stmt .= '`d` = ?'; push(@place,"$_updatedTime"); $stmt = "INSERT INTO `translation_history` SET $stmt"; $_tracked = $dbh->do($stmt,undef,@place); if(!$_tracked) { &send_Richard_error(qq~Could not track language modifi +cation... attempted this: dbh->do($stmt,undef,~ . @place . qq~); any +errors: ~ . $DBI::errstr); } $sess_ref->attr("show_message","success|I have successfull +y saved the changes to this language tag."); if($in{r} eq "q") { window_redirect("$_surl?pg=$in{pg}&do=$in{do}&view=doS +earchTags&q=$in{q}$inc_sess_id"); } elsif($in{p}) { window_redirect("$_surl?pg=$in{pg}&do=$in{do}&view=$in +{p}$inc_sess_id"); } else { window_redirect("$_surl?pg=$in{pg}&do=$in{do}$inc_sess +_id"); } } else { &send_Richard_error(qq~Could not update a language tag, he +re is what I attempted: (dbh->do(qq{update `$_tableName` set `value` += ?, `d` = ? where `lngId` = ?}, undef, qq\~$in{tagvalue}\~, "$_updat +edTime", "$in{lngId}")) any errors: ~ . $DBI::errstr); $sess_ref->attr("show_message","error|I could not save the + changes you asked me to. I have brought you back to try again. " . $ +DBI::errstr); window_redirect("$_surl?pg=$in{pg}&do=$in{do}&view=editTag +&lngId=$in{lngId}$inc_sess_id"); }
So for instance if I put this into the form:

Принимаю - зарегистрировать меня
(Did not display properly in the preview so I'll put it also in the code below directly into the html):
Принимаю - зарегистрировать меня

and submit that, when I check the database it has this:
Принимаю - з&#10 +72;регистрир&#1 +086;вать меня
So it does not work, I think it is Perl doing that because if I go to phpMyAdmin and edit that tag and put that in manually then save it, it is in there just fine when I check it. However, then when I load the edit field or preview the tag in the language editor in the perl system, I see only a bunch of ??? marks.

I made sure my headers in the browser that perl prints have the correct encoding:
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
That did not make any difference.

Any idea why this is not working?

Again I apologize if this is completely unrelated to perl, however, if you know of a way to get this working then please help.

thx,
Richard

Replies are listed 'Best First'.
Re: Russian Language read from mysql into perl script problem
by ikegami (Patriarch) on Feb 23, 2010 at 21:15 UTC

    and submit that, when I check the database it has this:

    [ I'm going with the assumption that the problem is that the database actually has stuff in the &# notation. If PerlMonks did that, let me know. ]

    It's not Perl that did it. It's the browser that encoded the characters that way because it was asked to encode characters that didn't exist in the encoding it was directed to use (e.g. characters outside of iso-8859-1 when the encoding of the form's page is iso-8859-1).

    I made sure my headers in the browser that perl prints have the correct encoding

    If the problem is as I described, you shouldn't have the problem if the page containing the form is encoded using UTF-8.

    What content type do you specify in the actual HTTP header. You've only shown the the content type you tell the browser the header should have contained.

    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

    Unrelated to your question: You claim your content is HTML, but that "/" at the end is illegal HTML.
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    should be
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

      Ok, I changed it to:
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      then I went into the table and had it save the content in the russian language again, now when I went to the editor and viewed it again I see this:

      Current Content(on Page):
      ???????? - ???????????????? ????
      Content in Database(in MySql)As displayed here:
      Принимаю - зарегистрировать меня

      So it still does not seem to work.
      when I pulls the russian language directly out of the datbase for some reason it just shows the ?? on the page, when I view the source I also only see the ?? on the page, so it seems to be Perl, because the source code should be what is in the table right?

      I don't know... still confused by it.

      Thank you and I do appreciate any further advice.

      thx,
      Richard

        Ok, my assumption was wrong. My question is still relevant, so I'll ask it again, with some additional questions.

        • What content type do you specify in the actual HTTP header. You've only shown the the content type you tell the browser the header should have contained.

        • How do you encode the HTML you send to the browser?

        • What do you get if you Dump the value you get from the database? (Output sent to STDERR, so you'll probably have to check your logs.)

Re: Russian Language read from mysql into perl script problem
by ikegami (Patriarch) on Feb 23, 2010 at 21:17 UTC
    [ Duplicate post, ignore ]