in reply to Does Search::InvertedIndex module live up to billing?

G'day all - back at it.

So I built a new database with the new keys:

CREATE TABLE `responses` ( `respondent` smallint(5) unsigned NOT NULL +default '0', `response` smallint(5) unsigned NOT NULL default '0', PR +IMARY KEY (`respondent`,`response`), KEY `respondent` (`respondent`), + KEY `response` (`response`) ) TYPE=MyISAM
Populated it with my test data, and tried my search
SELECT COUNT(*) FROM theTable A JOIN theTable B ON (A.respondent = B.r +espondent) WHERE A.response = ? AND B.response = ?
And it's averaging 0.2 seconds this morning while the previous two-column version is averaging 0.13 sec and the far-fetched one-table-per-response db is averaging 0.09 sec.

Anyway, EXPLAIN on this query in this new database says

+-------+--------+-----------------------------+----------+---------+--------------------+------+--------------------------+
| table | type   | possible_keys               | key      | key_len | ref                | rows | Extra                    |
+-------+--------+-----------------------------+----------+---------+--------------------+------+--------------------------+
| A     | ref    | PRIMARY,respondent,response | response |       2 | const              | 3372 | Using where              |
| B     | eq_ref | PRIMARY,respondent,response | PRIMARY  |       4 | A.respondent,const |    1 | Using where; Using index |
+-------+--------+-----------------------------+----------+---------+--------------------+------+--------------------------+
I suppose next I'll try reversing the order of the index declarations, but is that really going to give me the boost to more than double speed?

Bit arrays and bit masks sound interesting, but Googling "bit array" & "bit mask" doesn't give me much. Any good reads on this somewhere?

Replies are listed 'Best First'.
Re^2: Does Search::InvertedIndex module live up to billing?
by punch_card_don (Curate) on Oct 22, 2004 at 15:02 UTC
    Found the PDL pages....interesting...