Once you have something that "works", release that to the User and let them work with it for a while. They are the arbiters of whether the solution is 'fast enough'. If they are happy with it, you are done; even if you could coax an extra 20% reduction ibn the run time if you only ....
A couple of points:
- happiness isn't binary; users might be mildly happy with adequate performance, but estatic with fast performance, or grudgingly accepting of poor performance
- User acceptance is important; change is bad! Often, when you're redesigning the entire system for performance, you can make the system much faster and more consistant if you revise the end-user paradigm. You can't do that if you hand off the first system to the end users: you'll have user acceptance issues like crazy the minute you try to change anything. Users don't like wasting time on training, and training costs are real costs. The minute something hits production, remember, you'll have to support it for years, so choose carefully!
- Management hates testing time. If you want to rebuild and retest a system to do the same thing as an old system because you didn't build it fast enough in the first place, they're going to get angry with you. Given that testing takes twice as long as development, you're going to waste a *lot* of time if you don't develop the system right the first time...
These points aren't irrefutable; there are often exceptions, and they're mostly grounded in business culture, which varies from place to place. But in my experiences, those are the real world factors that get in the way of the rosy, ivory tower view of how software gets engineered; expecting things to be otherwise leads to bitter disappointment.