Hi Monks :)
I have a problems with encoding on my web platform.
This app using flat files for database.
When I started to develop , I used this code to open files:
open FH,">$path";
after few weeks we started to get traffic from other countries and names on other languages.
I get broken chars with existing code , and changed it to this only on inputs(>)
open FH , ">:utf8",$file_path;
it worked like that few months
The most traffic we get is from 'facebook login' button that uses perl api to get the data.
I recognized that on post forms, perl gets another data with broken chars
I already spent a few days on reading articles about encodings in perl and I still don't get the point how to work with that in the right way
For now I have changed all the inputs and output filehandles to this,because I read that it's the strict UTF in this article http://perlgeek.de/en/article/encodings-and-unicode
open FH, "<:encoding(UTF-8)", "$dir/private_data";
After that I get problems on posts and data that come from facebook, after many tries I got a code that works well. But the problem , that I doesn't understand why it works .. here is the code
    if($params{FORM_POST}){$_=decode 'UTF-8',$_ for(@$ref_userdata);}
    if($params{FROM_FACEBOOK}){$_=encode 'UTF-8',$_ for(@$ref_userdata
+);}
    open WH,">:encoding(UTF-8)","$dir/private_data" or die $!."$dir/pr
+ivate_data";
    print WH "$_|" for(@$ref_userdata);
    close WH;

[download]

Now I see that old database that writen in regular way and old utf8 get a broken chars.
When I doing cat private_datawith the new :encoding(UTF-8) in putty i get also not right encoding that looks like that|||Ð&#65533;иÑ&#65533;оÑ&#65533;лава|Ханева|female|03-08-1982|
I feel that I'm like in circle of searchings that will never end
Don't really know what to ask, just need a help with this.
Thanks a lot !!!

In reply to Mess with UTF-8, utf8 and raw encoding on live working platform by AlfaProject

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.