guarriman has asked for the wisdom of the Perl Monks concerning the following question:

Hello.

Working with Apache+Perl+MySQL, I'm developing a website to allow users to find within a product catalog. Product info is stored within a MySQL table (ID, name, price).

These users usually make typos and mistakes when typing the product name, and I want to suggest them other names, similars to the typed one. For instance: mp3 playr --> mp3 player

Does anybody know any algorithm or method to perform it? Thank you very much.

  • Comment on A method to suggest other names when user makes typos

Replies are listed 'Best First'.
Re: A method to suggest other names when user makes typos
by sauoq (Abbot) on Nov 03, 2005 at 10:43 UTC
Re: A method to suggest other names when user makes typos
by Corion (Patriarch) on Nov 03, 2005 at 10:39 UTC

    If a "plain" or "direct" search fails, you could try the following things:

    • Note the search in a log and manually create an alias every day for the top 10 failed searches. This will improve the perceived results of your users vastly, but requires lots of manual work every day.
    • Normalize all your products by using some phonetic normalization, do the same for your search term and then do the search again. This works nice if the first method still doesn't provide a satisfactory result.
    • Apply the Levenshtein distance or some other distance to select the "closest" matching/available word. For example, an interesting metric could be to see how many characters need to be inserted or swapped to get from one word to the next. This is expensive to compute but could yield interesting results too.

    The idea of manually adjusting the search results comes from a talk by Martin Pauly which discussed the work he did for Blackstar Video, I think. It is a nice idea on how to gradually improve a very simplicistic approach in my opinion.

Re: A method to suggest other names when user makes typos
by inman (Curate) on Nov 03, 2005 at 11:18 UTC
    You could try a combination of a real search engine and some query manipulation to improve the results. If you take your query as an example, you can process this into wildcard trigrams and pass the query to the search engine. The search engine would use it's ranking algorithm to work out which matches were closest. There are a number of free search engines available. The precise technique will depend on the ability of the search engine chosen.

    e.g. search for *mp3* *pla* *lay* *ayr*. The search engine should search for all of these expressions and then rank the results according to which documents matched the most

    This technique is used on commercial websites to make up for inaccurate spelling. e.g. brittany spears

Re: A method to suggest other names when user makes typos
by ghenry (Vicar) on Nov 03, 2005 at 23:16 UTC

    You could consider doing this with AJAX

    There's an example using Catalyst. There's Prototype.js, Rico and maybe a few things on JSAN.

    Won't be as quick a solution as the other nodes in this thread, but it's quite a cool way to do it and a bit like Google Suggest ;-)

    Walking the road to enlightenment... I found a penguin and a camel on the way.....
    Fancy a yourname@perl.me.uk? Just ask!!!