Since you still didn't reveal what you did to control encoding at the database or web level, I can only guess.

• It looks like the database content is hosed and contains strings in different encodings. You can't reliably SELECT records from these data.
• It seems that you did either not tell your database driver to handle UTF-8, or you omitted to decode content from your web form and write doubly encoded stuff to your database. In both cases, you need that first step of decoding after reading from the database.
• The regular expression takes care for stuff which has been inserted with a second level of UTF-8 encoding. Whenever the substitution succeeds, you found bad data in your database, inserted by either your new code, or by the legacy application. You can capture the return value of the substitution to check whether a substitution took place to identify data which need to be fixed in your database. With ikegami's suggestion to use utf8::decode you can achieve the same goal, a true return value from utf8::decode indicates broken data.
• The final encoding step is required if you print a web response with a charset of UTF-8 without don't specifying an I/O layer for that encoding. Again, without knowing what your code does, I can't say for sure.

Finally, if that code only seems to work, be sure to write a test suite with Unicode data, preferably also including strings with characters which can not be encoded in one byte. Also check the contents of your database with some "non-Perl" code, like the psql command line tool for PostgreSQL or whatever your database engine provides. Without that, your database operations will always be guesswork and the next migration will most likely go wrong as well.

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.