Why are you reinventing a well-solved problem? If your white- and black-list data is growing to the point where your standard "small and slow" isn't working, maybe it's time for a database.

A SELECT .. JOIN construct lets the DB software find the intersections, and it can do so remarkably fast.

Besides, what exactly is wrong with "huge and fast"? Disk space isn't generally a huge concern when you're talking about the sizes of CPAN modules...