in reply to limiting length of utf8 string in bytes

There's a snippit for this in the output_message function of cbstream.rb. (This code has quote a few places where it does ugly hacks with character encodings, because it was written back when only ruby 1.8 existed. Nowadays we have ruby 1.9 which has a better system for handling strings with various encodings than perl.) It's not directly applicable here, but the principle is the same.

Assumes $str contains the decoded string. Then, after

$str =~ /\A(.{0,383}[\x00-\xbf]|)/s or die;
$1 should contain at most 384 bytes and not end with an incomplete utf-8 character.

Update: ikegami's right, the above regex is wrong. (I still believe the one in cbstream is right, but does something different.)

Replies are listed 'Best First'.
Re^2: limiting length of utf8 string in bytes
by ikegami (Patriarch) on Dec 15, 2009 at 14:36 UTC
    That's wrong. It can chop up 3 and 4 byte chars. See my reply to the OP for the fix.