in reply to Data indexing in BerkeleyDB hashes

If it is to spider one domain, you could use the escaped relative url. It could cut down the key lengths quite a bit. (Don't know if this is an issue with Berkeley)

If you want to get really sidetracked , check out The Google Whitepaper

-Lee

"To be civilized is to deny one's nature."