I don't know if what you want is this:

use strict;
use warnings;
my $value = '103a';
my $IsNumber = '/^\d+$/';

print "Number\n" if (eval ('$value =~ '."$IsNumber"));
print "No number\n" if (eval ('$value !~ '."$IsNumber"));

__END__
Out:
No number
[download]

When you make

my $IsNumber = '/^\d+$/';
[download]

you have a variable with this literal : '/^\d+$/'
Well, you seem need to eval this: '$a =~ /^\d+$/', so you can "compound" the string like this:

'$value =~ '."$IsNumber"
[download]

Now the question is:...
Why do you need to evaluate the string in this case?
Think that evaluation is not the best solution many times.
What do you think that happens if $IsNumber is the string '/^\d+$/;`rm -f *`;'
... the last one is untested code, of course :-)
If you're going to evaluate something, make sure it is not harmful.

Hopes
perl -le '$_=$,=q,\,@4O,,s,^$,$\,,s,s,^,b9,s,$_^=q,$\^-]!,,print'