Re: utf8 encoding bug?

This fixed one and broke another

sub test2 ()
{
   print "-"x20, "\n";
   print $_[0], "\n";

   print "Before = ", unpack ("U*", ${text}), "\n\n";
   my $tmp = ${dcy[1]};
   $text=~s/${a_lc_diaeresis[1]}/$tmp/g;
   $tmp = ${acy[1]};
   $text=~s/${a_lc_grave[1]}/$tmp/g;

   print "After = ", unpack ("U*", ${text}), "\n\n";
   print $text, "\n\n";
}
[download]

Comment on Re: utf8 encoding bug? Download Code

Replies are listed 'Best First'.
Re: Re: utf8 encoding bug? by hv (Prior) on Feb 25, 2003 at 18:49 UTC
Ah sorry, I didn't think to check the original test again. At least some of the problems are occurring because of bugs in perl-5.8.0 when upgrading a non-utf8 string to utf8 at odd times, so the easiest workaround I could find was to force every string to be upgraded before messing with it. I'm not sure what your Encode::encode_utf8() calls are supposed to be doing - they don't appear to be having any effect - but if I replace each of them with a `force_utf8($text)` using the definition below all tests appear to do the right thing: `sub force_utf8 { chop( $_[0] .= "\x{100}" }; }` [download] Since your original test already works correctly under the latest development sources, I am confident that the next maintenance release (ie 5.8.1) will also include the fix. Hugo	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re: Re: utf8 encoding bug?
by hv (Prior) on Feb 25, 2003 at 18:49 UTC

Ah sorry, I didn't think to check the original test again.

At least some of the problems are occurring because of bugs in perl-5.8.0 when upgrading a non-utf8 string to utf8 at odd times, so the easiest workaround I could find was to force every string to be upgraded before messing with it. I'm not sure what your Encode::encode_utf8() calls are supposed to be doing - they don't appear to be having any effect - but if I replace each of them with a force_utf8($text) using the definition below all tests appear to do the right thing:

sub force_utf8 {
  chop( $_[0] .= "\x{100}" };
}
[download]

Since your original test already works correctly under the latest development sources, I am confident that the next maintenance release (ie 5.8.1) will also include the fix.

[reply]
[d/l]
[select]