Check the whole path, I tested it, seems like you need access rights (x) to all directories in the path.

mkdir -p noaccess/access
touch noaccess/access/delete.me
chmod u-x noaccess/
perl -e 'unlink("/home/tom/scratch/perl/noaccess/access/delete.me") or
+ die("oops $!")'
oops Permission denied at -e line 1.
chmod u+x noaccess/
perl -e 'unlink("/home/tom/scratch/perl/noaccess/access/delete.me") or
+ die("oops $!")'
[download]

regards,
tomte

Hlade's Law:

If you have a difficult task, give it to a lazy person --
they will find an easier way to do it.