I bet that's because perl opens the file, reads it, closes it and then runs the code.

$ cat > hw.pl
#!/usr/bin/perl
print "Hello, World!\n";
sleep 5;
print "exiting...\n";
$ chmod 755 !$
chmod 755 hw.pl
$ strace -o abc -tt -ff -s 500 !$
strace -o abc -tt -ff -s 500 hw.pl
$ _

(stripped strace output)
21:33:16.679617 open("/home/hue/lang/perl/hw.pl", O_RDONLY|O_LARGEFILE
+) = 3
21:33:16.681557 read(3, "#!/usr/bin/perl\nprint \"Hello, World!\\n\";\
+nsleep 5;\nprint \"exiting...\\n\";\n", 4096) = 72
21:33:16.681985 read(3, "", 4096)       = 0
21:33:16.682080 close(3)                = 0
21:33:16.682336 write(1, "Hello, World!\n", 14) = 14
21:33:16.683833 nanosleep({5, 0}, {5, 0}) = 0
21:33:21.663964 write(1, "exiting...\n", 11) = 11
[download]

Doing an lsof -n|grep hw shows that no process (except hw.pl itself) has the file opened, so when one unlinks it, the operation performs immediatly.