You could use s/// to both pattern match and insert the source directory in one step.

my $dir = '/path/of/dir';
opendir my $dh, $dir or die "Can't opendir '$dir': $!\n";
unlink grep s{^(.*$mypattern)}{$dir/$1}i, readdir $dh;
[download]

You could get better error reporting (which would have been useful here) if you call unlink in a loop with the "do or die" idiom instead.

# opendir as above
foreach my $file ( grep /$mypattern/i, readdir $dh ) {
    my $path = "$dir/$file";
    unlink $path or die "Can't unlink '$path': $!\n";
}
[download]