The core of the problem is that you don't realise that interpolation inserts the string in literally.

$ perl -E'
   say "\x61";
   $x="\\x61"; say "$x";
   $x="\$x"; say "$x";
'
a
\x61
$x
[download]

If you want to treat the content of a Perl variable as Perl code, you need to use eval EXPR.