Dear nysus:

I want to create a robot that spiders a handful of small sites a couple of times a day (...) I'd really appreciate some ideas that will save me from reinventing the wheel

Have you seen WWW::Mechanize? I have used it a few times, with excellent results.