You ask why people would keep configurable things in source code? Well every time you try to pull configurable things out you pay the following costs:

• Code does not magically become non-code by being moved to a configuration file. And the more you move there, the harder it becomes to read. I could submit many Apache config files for examples.
• You have to add a layer of indirection to your code, making it harder to understand. Your code example is a demonstration.
• You increase how much "action at a distance" there is. Action at a distance makes code harder to read, errors harder to spot, and synchronization errors much more common.

This is not to say that moving stuff into configuration is necessarily bad. In fact it is good if you need to tweak that configurable thing frequently.

With that said, your program is the same as:

#! /usr/bin/perl -ni
BEGIN {
    @ARGV = my $file = "hai";
    die "File '$file' doesn't exist!" unless -e $file;
}

print
    if s/HAI WORLD Times ([0-9]+).*/
          "\t\t<exML LOLZ = \"" . unpack(\'a1\', $1) . "\" />"
        /eg;
[download]

Yeah, it is a hack. But which hack would you prefer to maintain?

And a random note. Rather than eval you can just use a closure:

$replaceString = sub {
    "\t\t<exML LOLZ = \"" . unpack(\'a1\', $1) . "\" />"
};
# time passes

    $array[$elementIdx++] = $replaceString->();
[download]

This has a number of advantages, including much better performance, and the fact that errors are not accidentally trapped. (Trust me, if you're evaling code from a configuration, you want to know if the configuration is bad.)