Lots of good suggestions here, and I can't help but add a few of my favorites, in no particular order:

• Go on a (preferably strenuous) hike. Somehow, the physicality of that, combined with the literal change of scenery, really seems to break my mind out of whatever rut it might be in.
• Use the time for introspection rather than coding. By that I mean things like: What did I learn from my last project? How could I have done it better? Are there tools I could learn or build that would make my life easier in the future? How can I lay the groundwork to make me more effective on my next project?
• Stop by a big bookstore and browse through some of the non-computery sections: architecture, history, biology, etc. Pull out an interesting-looking book or two, grab a cappucino from the cafe, and curl up on one of those comfy chairs for a couple of hours of skimming and contemplation about something outside my usual frame of reference.
• Visit CPAN. Look over a couple of modules that I've never had occasion to use before. Read the documentation, read the code, build something (even something trivial) using those modules.
• Talk to coworkers (including nonprogrammers!). Find out what they really do. Learn what makes them tick. Learn more about the business your company is really in and who your customers are.
• Visit some sites that talk about programming languages I don't use every day. Learn a few things about Lisp, Prolog, Smalltalk, whatever. Maybe try doing something I've done in Perl in one of those languages to see how the thought process and implementation differs.

I also keep a running list of little projects and tools that aren't important enough to do right away, but that I'd like to write someday. Then if I'm got some time on my hands and I'm raring to code, I've always got a few things to choose from.

But at the same time, I'd urge you to look at downtime not just as a chance to actually do something new, but as a chance to improve your ability to do things in the future -- by learning new tools, interacting with new people, and thinking about new topics. Even if none of them have any direct relationship to what you're working on now or in the immediate future, I think you'll see benefits in the long run.