This will do what you want, I think:

my $test="my_variable_name";
my $default_value="the value I set";
$$test = "old value";
print "the value of \$my_variable_name = $my_variable_name\n";
$my_variable_name = "A new value";
print "the value of \$my_variable_name = $my_variable_name\n";
[download]

... but don't do that :)

Update: Perhaps I read & posted too quickly, does this do what you want?

my $test="my_variable_name";
my $default_value="the value I set";

$default_value = *$test;
$$default_value = "old value";

print "the value of \$my_variable_name = $my_variable_name\n";
$my_variable_name = "A new value";
print "the value of \$my_variable_name = $my_variable_name\n";
[download]

Update2: Thanks ikegami. You'll notice I omitted the obligatory "use strict;" and "use warnings;", because when they're not omitted:

$ perl symbol.pl
Global symbol "$my_variable_name" requires explicit package name at sy
+mbol.pl line 12.
Global symbol "$my_variable_name" requires explicit package name at sy
+mbol.pl line 13.
Global symbol "$my_variable_name" requires explicit package name at sy
+mbol.pl line 14.
Execution of symbol.pl aborted due to compilation errors.
[download]

$/  =  q#(\w)#  ;  sub  sig { print scalar reverse  join  ' ',  @_  } 
+ sig
map { s$\$/\$/$\$2\$1$g && $_ } split( ' ', ",erckha rlPe erthnoa stJu
+" );
[download]