Re^6: How to avoid decoding string to utf-8.

Hi Haj, Thank you for the reply.

1.Data comes from the Database, as it is same as you look on the web application.
2. Yes, Perl code is running as part of web application.
3. TestStr is basically coming from database which got inserted while submit Form from the application itself, but at the time of showing this string on the web application this issue occurs.

as I said earlier, I have strings with mixed encodings, which means that one string is differently encoded with another due to upgrade of application from legacy application.

Thank you.

Comment on Re^6: How to avoid decoding string to utf-8.

Replies are listed 'Best First'.
Re^7: How to avoid decoding string to utf-8. by haj (Vicar) on Oct 12, 2020 at 13:54 UTC
So you appear to have strings with different encodings in your database. That's really bad, because you won't get correct results from database queries until you get this fixed. I have difficulties to understand why the regular expression does not change the result unless you are several levels of encoding away from the truth. This can happen if during the upgrade someone tried to fiddle with encoding until the result "looks right" in the browser - but what you actually have now is just a cancellation of errors. Encoding matters in the transfer from the browser form to the web application, when writing from the web application to the database, and in the opposite direction when reading from the database and when sending the data to your browser. Please tell us how you control encoding in these four places. For obtaining some data for debugging, please print the data - good and bad - like this (also suggested by ikegami earlier in this thread): `printf("%vX", $testStr)`;	[reply] [d/l]

Replies are listed 'Best First'.

Re^7: How to avoid decoding string to utf-8.
by haj (Vicar) on Oct 12, 2020 at 13:54 UTC

So you appear to have strings with different encodings in your database. That's really bad, because you won't get correct results from database queries until you get this fixed.

I have difficulties to understand why the regular expression does not change the result unless you are several levels of encoding away from the truth. This can happen if during the upgrade someone tried to fiddle with encoding until the result "looks right" in the browser - but what you actually have now is just a cancellation of errors. Encoding matters in the transfer from the browser form to the web application, when writing from the web application to the database, and in the opposite direction when reading from the database and when sending the data to your browser. Please tell us how you control encoding in these four places.

For obtaining some data for debugging, please print the data - good and bad - like this (also suggested by ikegami earlier in this thread):

printf("%vX", $testStr)

[reply]
[d/l]