Hello again nysus,

I just tried that on my local DB and it works. I found the module Text::Unidecode it should do what you need.

#!/usr/bin/perl use utf8; use strict; use warnings; use Text::Unidecode; my $encode = unidecode("What\x{2019}s up with the water ??"); print $encode . "\n"; __END__ $ perl test.pl What's up with the water ??

Update: Not to forget you need to define also the column in your table as:

`Column` VARCHAR(150) CHARACTER SET utf8 NOT NULL UNIQUE,

You do not the parameter NULL UNIQUE I just usually add them on my columns so I will avoid duplications etc.

Update2: Sample of the whole code that I tested:

#!/usr/bin/perl use DBI; use utf8; use strict; use warnings; use Data::Dumper; use Config::Simple; use Text::Unidecode qw( unidecode ); $|=1; #flush every time the program my $path = 'conf.ini'; my %config = (); my $checkExist = ""; sub mysql { Config::Simple->import_from("".$path."", \%config) or die Config::Simple->error(); my $dbh = DBI->connect("dbi:mysql::".$config{'MySQL.host'}.":".$co +nfig{'MySQL.port'}."", "".$config{'MySQL.user'}."", "".$config{'MySQL.pass'}."", { 'PrintError' => 1, 'RaiseError' => 1 , 'AutoInactiveD +estroy' => 1 } ) or die "Could not connect to ". $config{'MySQL.host'} .": ". $DB +I::errstr ."\n"; my $databases = $dbh->do("SHOW DATABASES LIKE '".$config{'MySQL.db +'}."'") or die "Error: " .dbh->errstr. "\n"; if ($databases eq 1) { printf "Database: ". $config{'MySQL.db'} ." exists not creating: " +. $config{'MySQL.db'} ."\n"; } else { printf "Database: ". $config{'MySQL.db'} ." does not exist creatin +g: ". $config{'MySQL.db'} ."\n"; $checkExist = $dbh->do("CREATE DATABASE IF NOT EXISTS `".$config{' +MySQL.db'}."`") or die "Could not create the: ".$config{'MySQL.db'}." error: " +. $dbh->errstr ."\n"; } # End of else $dbh->do("USE ".$config{'MySQL.db'}."") or die "Error: " .dbh->errstr. "\n"; my $tables = $dbh->do("SHOW TABLES FROM `".$config{'MySQL.db'}."` WHERE Tables_in_".$config{'MySQL.db'}." LIKE '".$config{'MySQL.table'}."'") or die "Error: ".dbh->errstr. "\n"; if ($tables eq 1) { printf "Table: ".$config{'MySQL.table'}." exists not creating: ".$ +config{'MySQL.table'}."\n"; } else { printf "Table: ".$config{'MySQL.table'}." does not exist creating: + ".$config{'MySQL.table'}."\n"; $checkExist = $dbh->prepare("CREATE TABLE ".$config{'MySQL.table'} +." ( `ID` INT NOT NULL AUTO_INCREMENT, `data` VARCHAR(150) CHARACTER SET utf8 NOT NULL UN +IQUE, PRIMARY KEY(`ID`) );"); if (!$checkExist->execute()) { die "Error: ". $checkExist->errstr ."\n"; } } # End of else $checkExist = $dbh->prepare("INSERT IGNORE INTO `".$config{'MySQL. +table'}. "` (`data`) VALUES (?)"); # my $encode = "What\x{2019}s up with the water ??"; # my $encode = encode_utf8("What\x{2019}s up with the water ??"); my $encode = unidecode("What\x{2019}s up with the water ??"); if (!$checkExist->execute($encode)) { die "Error: ". $checkExist->errstr ."\n"; } my $statement = "SELECT * FROM `".$config{'MySQL.table'}."` WHERE +1"; my $hash_ref = $dbh->selectall_hashref($statement, 'data'); $checkExist->finish(); $dbh->disconnect() or warn "Error disconnecting: $DBI::errstr\n"; return $hash_ref; } # End of mysql sub my $output_ref = mysql(); my @data = keys %$output_ref; print Dumper $output_ref, \@data; __DATA__ $ perl mysql.pl Database: PerlMonks exists not creating: PerlMonks Table: Data exists not creating: Data $VAR1 = { 'What\'s up with the water ??' => { 'ID' => 1, 'data' => 'What\'s up wi +th the water ??' } }; $VAR2 = [ 'What\'s up with the water ??' ];

The conf.ini file:

[MySQL] user=username pass=password host=localhost port=3306 db=PerlMonks table=Data

Hope this helps, BR.

Seeking for Perl wisdom...on the process of learning...not there...yet!

In reply to Re^3: Storing UTF-8 data into database from scraped web page by thanos1983
in thread SOLVED: Storing UTF-8 data into database from scraped web page by nysus

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.