RTF is fundamentally a ANSI text file. A character set specification is used to set the code page used and may be one of:

• \ansi - the default
• \mac - Apple Macintosh
• \pc - IBM(R) PC Code Page 437
• \pca - IBM PC Code Page 850

However the encoded characters you show suggests that the code page being used is compatible with cp1252 (the wikipedia article's mention of cp1256 is misleading and wrong as far as I can tell). The following may be the voodoo you are looking for:

use strict;
use warnings;
use Encode;

my $x = "foo à, è, ì, ò, ù bar";
my $x1 = encode('cp1252', $x);

$x1 =~ s<([\x00-\x1F\x7F-\xFF])> <"\\'" .(unpack("H2",$1))>eg;

print $x1;
[download]

use Modern::Perl;
use utf8;
use RTF::Writer qw/rtfesc/;

my $escaped = rtfesc('aà eé iî oö uü');
say $escaped;
[download]

a\'e0 e\'e9 i\'ee o\'f6 u\'fc

You will need to find out the encoding of your input data. The \u suggests to me that the characters are likely UTF-8-hex-encoded unicode code points. You will need to find out what encoding / codepage RTF actually uses, and encode to that target. See perluniintro, and whatever RTF spec.

Also, it would be interesting to hear from you how Unicode::Escape fails for you and where it misses the RTF specs (and where the RTF specs are to be found). I don't find the Unicode::Escape documentation talking about RTF at all, so maybe there is some finer point I'm missing.

\u is used for Unicode documents according to Wikipedia:
http://en.wikipedia.org/wiki/Rich_Text_Format#Character_encoding

\' is used for Windows1256 encoded, and no mention is made of Mac-encoded even tho' Word for Mac uses it.

Unicode::Escape fails because it seems to encode higher order characters differently than RTF editors do.

I don't care who is right, it's just not the same and so doesn't work, and of course this presents problems when I'm filling my templates with UTF8 data and trying to filter it. :(

$x =~ s/([\x00-\x1F\ x7F-\xFF])/"\\'" .(unpack("H2",$1))/eg;
                    ^
[download]

Assuming you're talking about Latin-1 characters (as the range up to just \xFF suggests), it should suffice to get rid of the indicated space...

my $x = "foo à, è, ì, ò, ù bar";

$x =~ s/([\x00-\x1F\x7F-\xFF])/"\\'" .(unpack("H2",$1))/eg;

print $x;    # foo \'e0, \'e8, \'ec, \'f2, \'f9 bar
[download]

Does it suffice for you? Because for me I get the following:

my $x = "foo à, è, ì, ò, ù bar";

$x =~ s/([\x00-\x1F\x7F-\xFF])/"\\'" .(unpack("H2",$1))/eg;

print $x;    # WHAT I WANT: foo \'e0, \'e8, \'ec, \'f2, \'f9 bar
# What I get
foo \'c3\'a0, \'c3\'a8, \'c3\'ac, \'c3\'b2, \'c3\'b9 bar
[download]

Vexing...

Judging by your output, your problem is that your string (source code) is UTF-8 encoded, but you have not told Perl about it.

use utf8 if your source code is in UTF-8.  In case the data comes from elsewhere, decode it properly before using it.

The UTF-8 encoding of the character 'à' (for example) is the two bytes c3 a0, so if you don't tell Perl that those two bytes are supposed to be decoded into one character, you'll have them incorrectly interpreted as the two latin-1 characters \xc3 and \xa0.

Old thread, but just in case anyone happens to stumble against this thread as I did, here is an addition that might be useful.

RTF specs admit non-ansi Unicode characters, by escaping them in decimal format. So in case your string has some non-ansi characters, you will also need to convert them to escaped. Fortunately, Encode can do it for you. Here is an example:

use Modern::Perl;
use utf8;
use Encode;

my $text = "Gen 1:1 Ἐν ἀρχῇ ἐ&#
+960;οίησεν ὁ θεὸ
+ς τὸν οὐρανὸ&#95
+7; καὶ τὴν γῆν.";

$text = encode('cp1253', $text, sub{ sprintf "\\u%d\?", shift }); # cp
+1253 is for Greek)
$text =~ s/([\x00-\x1F\x7F-\xFF])/"\\'" .(unpack("H2",$1))/eg;

say $text;
[download]

Output will be:

See also: Re: Save yourself, start all projets with UTF-8 encoding