It's simpler than that with glob, default arguments, and a little logic.

for (glob "$path/*">) {
    unlink if -f and -M _ > $cutoff;
}
[download]

That takes advantage of $_ as default argument for -X and unlink. The "_" handle saves a stat call.

That omits the tree expansion, which is easily handled by wrapping in a recursive sub.

(Added)

sub unlink_all (
    my ($path, $cutoff) = @_;
    local $_;
    for (glob "$path/*") {
        unlink, next if -f and -M _ > $cutoff;
        unlink_all($_, $cutoff) if -d _;
    }
    1;
}
[download]