I think the comments regarding the hardware are a bit dated since right now the current hardware is itself a bit dated. There are efforts under way right now to migrate to new hardware and we should see a performance jump then.
Another thought has been to add some javascripty functionality similar to slashdot and use.perl.org to cut down on the full page loads the server has to do for comments, etc.
See also this thread: