Re^2: DBIx returns question marks

Hi All,

thanks for excellent ideas I'll start working on them right now (in a slow sunday night way). Meanwhile, I've discovered that when I do this on dev machine:

show variables like "%character%";
show variables like "%collation%";
[download]

I get this:

character_set_client    utf8
character_set_connection    utf8
character_set_database    utf8
character_set_filesystem    binary
character_set_results    utf8
character_set_server    utf8
character_set_system    utf8
character_sets_dir    /usr/share/mysql/charsets/
----------
collation_connection    utf8_general_ci
collation_database    utf8_general_ci
collation_server    utf8_general_ci
[download]

And on my staging machine is this:

character_set_client    utf8
character_set_connection    utf8
character_set_database    latin1
character_set_filesystem    binary
character_set_results    utf8
character_set_server    utf8
character_set_system    utf8
character_sets_dir    /usr/share/mysql/charsets/
------------------
collation_connection    utf8_general_ci
collation_database    latin1_swedish_ci
collation_server    utf8_general_ci
[download]

That cannot be good. I'll play during the night and post my results in the morning.

Thanks a lot, Stepamil

Comment on Re^2: DBIx returns question marks Select or Download Code

Replies are listed 'Best First'.
Re^3: DBIx returns question marks by stepamil (Acolyte) on Aug 08, 2011 at 13:41 UTC
Hi, me again, Thanks Corion for giving me this list. This one's a keeper :) I went through it and here's what I have: I've connected staging machine to use dev DB so both scripts are using the same DB. I've also removed DBIx from equation because I have the same behaviour on DBI, so I'll change the thread title shortly. Corion: Check the data format in your script as it writes to its output. Is it properly encoded from Unicode to the target encoding? Compare the hexdumps. Me: Dev -> var is UTF8, endoded:\x{7f16}\x{53f7} hex:e7bc96e58fb7, Stg -> var is not UTF8, var:??, hex:3f3f Corion: Check the data format in your script as it reads the data from the database. Is it utf8 everywhere? Compare the hexdumps. Me: No, the ?? variable is not UTF8 when I get it from staging machine Corion: Check the data format in your database in all tables. Are the tables/columns declared to be utf8/Unicode everywhere? Compare the hexdumps. Me: I'm using the same database, but different machine. Dev is using local DB and staging is using dev's DB. So that should be covered. Corion: Check the data format when it is written from your script to the database. Do they write utf8/Unicode everywhere? Compare the hexdumps. Me: Same as previous answer - same DB, different machine Corion: Check the data format from where your script gets its data. Is it properly decoded from the source to utf8 everywhere? Compare the hexdumps. Me: Also covered by previous answer Well, fighting with this continues. My more experienced colleague is coming from holiday this afternoon. He'll be fresh pair of eyes on this issue. Stepamil	[reply]
Re^4: DBIx returns question marks by Corion (Patriarch) on Aug 08, 2011 at 13:55 UTC
Are you sure that the `libmysql` versions between the two machines are the same? It seems to me that the two machines read different things from the database, which means that they either have problems due to one going over the network and the other being local, or due to being otherwise different versions.	[reply] [d/l]
Re^4: DBIx returns question marks by stepamil (Acolyte) on Aug 08, 2011 at 17:45 UTC
The issue is fixed!!! I've went through the thread once again and checked DBD::mysql. On dev was 4.018 version, but on stg it was 3.x. After the update everything worked perfectly. I just cannot belive that I somehow (while checking) missed to notice this. Thanks a lot for your help. It has been a tricky one for me, and, apart from learning the cause to this, I learned extra on utf8 difficulties. Stepamil	[reply]