I think you should worry less about speed. The numbers you show are not a significant time difference for a statement which will only run once in a non-trivial program. Particularly not for a disk I/O operation which may be influenced that much by head position or system traffic.

A better criterion is to consider how soon your program should know about your subs. By loading at compile time with use, perl knows the location and properties of your subs at compile time. With that, for instance, subs prototypes become effective, where they would be ignored if loaded at runtime. That may even gain you some speed at runtime.

Note: Don't go stick prototypes on everything! They're much misunderstood and rarely necessary.