I've chatted with Pete Sergeant; I don't think that RTF::Tokenizer will be developed further. It is useful, though, if you are careful to preprocess special characters before running the tokenizer. It really, really doesn't like \~ codes for non-break spaces.

--
$,="\n";foreach(split('',"\3\3\3c>\0>c\177cc\0~c~``\0cc\177cc")) {$a++;$_=unpack('B8',$_);tr,01,\40#,;$b[$a%6].=$_};print@b,"\n"