It's generally not something to worry about with Perl. Can you provide a stripped down sample of your code that demonstrates where the speed issue is and a description of the context in which the code is used?

Generally finding an appropriate data structure and algorithm have much more impact on execution time than micro-adjustments such as hash optimization. Your code would have to be doing virtually nothing but accessing the hash for any optimization there to make any measurable difference at all to execution time.

You may find tools such as Devel::DProf and Devel::FastProf help focus on the areas of code that really need attention.