I'm currently too (re)writing a search engine in perl.

I'm using POE for simultanous http requests to index the sites. The most important thing is to pay attention to your database layout, because I had to modify it three times, because some features I decided to add required it.
And make a good plan before you start writing it. Be sure how you want to rate the indexed pages.
Edit: You might also have a look at the google programming contest group. (http://groups.google.com/groups?q=google.public.programming-contest). People have mentioned there some ideas on how to rate a page.