Buy a faster computer. Don't make huge lexically declared variables.

You're asking Perl to do more and more, of course it's going to take longer and longer. Each occupied slot in the HV* structure is pointing to a scalar SV* instance, each of which has to be freed.