You are starting with a variable that contains a JavaScript literal minus the quotes (e.g. "\","3","4",0"), and you want code to generate the string JavaScript would produce by that literal (byte E0₁₆).

This has nothing to do with UTF-8 or Unicode. The byte may be part of the UTF-8 encoding of some string, but md5_hex doesn't care about that.

my $chapid3= '\027';
my $chapchalange3= '\340\174\012\314\214\070\070\231\377\005\016\132\2
+70\024\241\163';
s/\\(\d{3})/chr(oct($1))/eg for $chapid3, $chapchalange3;

print "\nExample2=".Digest::MD5::md5_hex($chapid3."AAA".$chapchalange3
+);
[download]

And I see no utf8 flag for the internal representation?

That flag should never matter. It only does in the presence of bugs. md5_hex is not buggy.

perl -MTest::More=tests,1 -MDigest::MD5=md5_hex -e'
   $_ = chr(0xE9);
   utf8::downgrade( $dn = $_ );
   utf8::upgrade(   $up = $_ );
   is(md5_hex($up), md5_hex($dn));
'
1..1
ok 1
[download]

If anything, I would suspect a problem if the flag was on.