Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Re: Re: A short meditation about hash search performance

by liz (Monsignor)
on Nov 15, 2003 at 21:57 UTC ( [id://307388]=note: print w/replies, xml ) Need Help??


in reply to Re: Re: A short meditation about hash search performance
in thread A short meditation about hash search performance

So what it says is that the chance run into the worst analysis I given, is probably reduced.

Indeed. The impetus for the random key hashing scheme, was the potential for a DOS attack when a fixed key hashing scheme was used. So 5.8.1 introduced a random seed for hashing keys. However, for long running perl processes (think mod_perl), it was thinkable that the hash seed was "guessable" from performance of the program on various inputs. Since there was a binary compatibility issue as well, schemes were tried out to fix both.

Once people realized you're really talking about a general performance issue, it started to make sense to make the algorithm self-adapting depending on the length of the lists of identical hash keys.

Abigail-II did a lot of benchmarking on it. Maybe Abigail-II would like to elaborate?

Liz

Update:
(If that's the case, the document liz provided shall not be there, as the queue length would always be 1 ...

A same hash key list length of 1 for all hash keys, would be optimal if there were no other "costs" involved. However, the re-hashing of existing keys is not something to be done lightly, especially if the number of existing keys is high. So you need to find the best possible combination of same hash key list length and re-hashing. In that respect, the ideal same hash key list length is not 1!

Replies are listed 'Best First'.
Re: A short meditation about hash search performance
by Abigail-II (Bishop) on Nov 16, 2003 at 02:54 UTC
    Abigail-II did a lot of benchmarking on it. Maybe Abigail-II would like to elaborate?
    The benchmark was fairly simply: take about million different words, insert them in a hash, measure how long it takes and what the average chain length is. They are all common words, it's the combination of various English wordlists I once grabbed from a puzzle site, and long list of Dutch words. No specially prepared input. The average chain length was 1.27 on 5.8.0, 5.8.1, 5.8.2-RC1 and 5.8.2-RC2. The only interesting thing was the time - it took about 17.5 seconds on 5.8.0, 5.8.1 and 5.8.2-RC2, and less almost 4 seconds less on 5.8.2-RC1.
    A same hash key list length of 1 for all hash keys, would be optimal if there were no other "costs" involved. However, the re-hashing of existing keys is not something to be done lightly, especially if the number of existing keys is high. So you need to find the best possible combination of same hash key list length and re-hashing. In that respect, the ideal same hash key list length is not 1!
    I do not agree with the latter conclusion. The best possible combination of max chain length and re-hashing depends on the ratio number of inserts vs number of queries (for the sake of simplicity, let's not consider deletes). The lower this ratio is (that is, the more queries you have), the more time you can spend on inserts to get a better overal performance. That is, if you have enough queries, it pays to have max chain length of 1.

    Abigail

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://307388]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (4)
As of 2024-03-29 06:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found