Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Monks, I have to get the content of the web pages in languages other than English and translate part of the content to English Language. I am using WWW::Mechanize to fetch the web content and able to extract the content I need to translate. I use google translate API to translate. Here the code that tries to translate from Russian to English. Problem here is when extract the string into $deal_title the content of the string is changed to some format .Not the original string. I want to retain the original content i.e Russian sentance in $deal_title so that i can translate. I tries Encode module to encode to UTF8 and passed it to API. It is not working.
It would be of great help if anyone can share their wisdom to solve this issue.
Thanks.

Here is method call & module code:

my ($stat,$title_translated); ($stat,$title_translated) = Translate2Eng::Translate($deal_tit +le,'ru','en','AIzaSyAaWvD1pbiJYf0AlkVp4-l4KVkkSrkxQow');
package Translate2Eng; #This module translates the non-english stings to english #using the Google Translate API. Limitation of this API is #that it can translate only 1,00,000chars/day/key use strict; use warnings; use REST::Client; use JSON; use HTML::Entities qw(decode_entities); use Data::Dumper; sub Translate{ my $string = shift; my $src_lang = shift; my $des_lang = shift; my $API_key = shift; my $client = REST::Client->new(); #print "--https://www.googleapis.com/language/translate/v2?key=$API_ke +y&source=$src_lang&target=$des_lang&q=$string---"; eval{ $client->GET("https://www.googleapis.com/language/tran +slate/v2?key=$API_key&source=$src_lang&target=$des_lang&q=$string"); }; if ($@) { return(0,$@); } my $response = $client->responseContent(); my $json_text = from_json( $response ); my $text = ${$json_text->{data}->{translations}}[0]; my $translatedText = $text->{translatedText}; $translatedText = decode_entities($translatedText); return (1,$translatedText); }

Replies are listed 'Best First'.
Re: Processing Non-English Language Strings
by pvaldes (Chaplain) on Aug 18, 2011 at 17:35 UTC
    mmmh... use Convert::Cyrillic; ? perl can support koi8-r for unix, cp1251 for windows and mac for macs, all of them common encodings for russian characters. Maybe you can reduce the problem using specifically one of them instead utf8; you have a module in cpan for detect cyrillic characters also.