eval "package $package; &$code; 1" or die $@;

I think you're missing a "\" before "$code". But then the code will be called via a code reference, and the contents of the code reference are still in the old package (where they were compiled):

sub run_in_package (&$) {
    my($code, $package) = @_;
    eval "package $package; &\$code; 1" or die $@;
}

run_in_package {
    showcaller();
} 'foooo';


sub showcaller {
    warn scalar caller;
}
[download]

The text of the warning will be "main" (not "foooo").