I would love to eliminate the temporary hash by using anonymous hashes.

We often get such questions, yet Perl provides a mechanism to hide away complex code: subs! Move your code info a sub, and you won't have temporaries lying around. (As mentioned, this has already been done for you as List::Util's uniq.)