New semantic search technology

I just noticed on NewsForge an article on semantic searching - the idea of searching for 'fruit' and finding documents that contain 'apple' or 'orange' in them.

After implementing their first algorithm in Perl and C++, they found the algorithm couldn't scale. So they rewrote it with a new algorithm, this time in just Perl, and now have their scalability.

I thought this was interesting both from just a pure geek perspective (I've wished for something like this from google a few times), but also from a pure Perl perspective - proving, once again, that the speed/scalability of the underlying algorithm generally matters more than the language it is implemented in.

Comment on New semantic search technology

Replies are listed 'Best First'.
Re: New semantic search technology by shotgunefx (Parson) on Sep 26, 2006 at 11:28 UTC
I wanted to try a crack at something a little bit like this awhile back. There was an article in an old TPJ about Lingua::Wordnet that got me thinking along those lines. Never got much past thinking about the design though. -Lee "To be civilized is to deny one's nature."	[reply]
Re: New semantic search technology by jZed (Prior) on Sep 28, 2006 at 03:42 UTC
I could be wrong, but I think you only summarized the first part of the story ... 1) the Perl/C++ mixture wouldn't scale so they replaced it with a Perl only version that did scale. and then 2) it wasn't fast enough so they rewrote it entirely in C++ which is what they now distribute. Still, it's a very interesting project, thanks for bringing it to our attention.	[reply]