Your spec is vague. This is what I'd suggest given what you're asking: http://www.google.com/search?q=keyword+site%3Aperlmonks.org

If you are spidering a site, you better have permission or ensure it's allowed by the site's ToS.

If you're doing something ethical but for some reason can't rely on Google: Mech or LWP::UserAgent would be fine. 5,000 pages would probably only take about 20-30 minutes to spider without parallel requests but either way you might really be hammering a website. A dynamic site getting constant requests like that can be smothered depending on its server/architecture.

KinoSearch 0.3 or better is great for search engines after the fact. With KSx::Simple + $mech->content(format => "text") you could have a lightning fast basic search engine done with like 20 lines of code (update: search part only). I know that's a valid estimate because I did one, against a backend DB, last night. :)

Update: fixed mod://link (thanks rowdog)