If you call a sub in a loop, you are timing how long it takes to call the sub as well as the time it takes to execute the contents of the sub.

Note that Benchmark goes to some lengths to try to subtract out the time taken by the "calling the sub" part. Of course, these attempts can certainly be unsuccessful (to the point of Benchmark declaring defeat or just to the point of Benchmark producing meaningless results; though I think that interpretation is often valid even when one has worked around the issue as you described).

which is important when you're timing something that's as fast as sub calls

There is very little that I would characterize as "important" when trying to time something in Perl that is about as fast as a sub call. Well, other than the idea of "you are likely wasting your time". :)