No, there isn't really any more elegant way. This is because the keys to a hash are (in theory) unordered, so saying something the "the second key" is meaningless.

Either use a parallel array or sort the keys before use:

delete( @hash{(sort(keys{%hash}))[2..4]} );

will delete the 2nd through 4th sorted keys from the %hash.