in reply to Re: Bloom Filter or other mehod to store URL's?
in thread Bloom Filter or other mehod to store URL's?
Hmmm... our intranet server hosts about 10 million pages. If i do a no-brain hash, it quickly fills my 1 GB of mem. Even on disk a 10 GB hash file doesn't sound nice.