I can't reproduce your stated result; I don't see a backslash in the string. I am using Config::General version 2.50 which I just downloaded from CPAN:

$ cat rcfile
foo   16&\#x2032; roll
$
$ cat 894796.pl
#!/usr/bin/env perl

use warnings;
use strict;
use Config::General;

my $conf   = Config::General->new('rcfile');
my %config = $conf->getall();
print $config{foo}, "\n";

$ ./894796.pl
16′ roll
$
[download]