To add my own anecdote to this thread, I very regularly see periods where the site is completely unresponsive (read timeouts) from both at work (with a nasty HTTP proxy) and at home (straight DSL connection). Generally things work fine for 5-10 minutes, and then there will be a 1-3 minute period where I cannot get a response from the site at all. When it does respond, though, it's generally pretty speedy.
Though I stopped trying to pursue this when I asked others if they were seeing similar problems. Until this thread, everyone I'd spoken to had stated they saw none of the problems I saw.
Could this be an issue with an ISP slightly upstream from PM? That might explain a significant number of us seeing frequent problems while others (whose data may be routed separately) see no problems. *shrug*