Create custom indexes for quick search words in db filled with large texts (stripped htmls).
This code create (VERY FAST) two text files which later will be quickly loaded in db with "LOAD FROM FILE" query.
iWords.txt contain links id_txt<->id_word: "id_txt,id_word".
Vocab.txt contain unique words: "word,id_word".
iWords.txt must be processed with "sort | uniq" after this code.
(This code was written long before MySQL added ``FULLTEXT SEARCH'' feature.)
Authors: me and asdfgroup.
sub FetchText { # doing something like this: # $sth->fetchrow_array("SELECT id_txt, txt FROM Text"); } ($,, $\) = ("\t", "\n"); open I, ">iWords.txt" or die $!; open V, ">Vocab.txt" or die $!; while (($id, $_) = FetchText()) { $_=lc; print I $id,$word{$_}||=(print(V $_,++$w),$w) for /\b[a-z]{3,30}\b/g; } close I; close V;