Aside from pre-sizing your hashes as mentioned by derby above, there really isn't much you can do in perl to make hash-access faster. That's different in C++ and Java because those languages implement hashes in C++ and Java respectively. Perl hashes are implemented in (already pretty efficient) C.

You could - maybe - implement a more efficient hashing algorithm in C that's tailored to your application, but I would be very surprised if it made any significant dent in your application run time.

In my experience, if you want serious speed ups, and your program speed is really limited by CPU time (instead of IO speed) and your algorithm is already efficient (see also What is "Schwarzian Transform" (aka Schwartzian), for example), you can gain a lot from porting your heavy lifting inner loops over to C. And you probably still will use perl's hashes then.