in reply to Re^3: Best way to look-up a string amongst 100 million of its peers
in thread Best way to look-up a string amongst 100 million of its peers
Pre-bucketing and splitting the data can have several advantages over an all-in-one-btree file. Notably, inserts can be faster because there is no rebalancing to be done. And if you split far enough to make the resultant files small, then there is no need to sort the individual files as a linear search is fast enough.
In effect, using the first n chars to index into the appropriate bucket and then linear searching is exactly equivalent to hashing. Ie. You described the same process perl uses for its internal hashes and that CDB uses. Except the hashing part of the process is faster.
|
|---|