Re^4: Optimisation isn't a dirty word.

we wasted several programmers years trying to do an in-house optimization as part of a code cleanup.

Seems to me, part of the problem is right there. If you're doing code clean-up, you're attempting to make your code more maintainable, readable, modularized etc. If you're doing optimization, you're attempting to make it more efficient. Doing both at the same time is inordinately difficult and makes failure at both more likely. I'd posit that even if you're a wizard who knows every nook and cranny of perl and can mentally juggle thousands of lines of code, you're probably better off doing clean-up and optimization in several alternating passes (I'm certainly no such wizard, so this is pure speculation on my part, but it makes sense).

This is also where I have most problems with the OP. While I agree that performance optimization is often important and probably does not get enough good press these days, I think it is dangerous to want to do too much of it too soon. It's hard enough coming up with a program design which is flexible yet simple. Strongly leaning towards optimization at the same time makes things infinitely harder and there aren't many people who can pull it off well. Also, I think it'll always be easier to refactor maintainable code for optimization than it will be to turn optimized but messy code into something another mortal can understand and maintain.

Comment on Re^4: Optimisation isn't a dirty word.

Replies are listed 'Best First'.
Re^5: Optimisation isn't a dirty word. by Anonymous Monk on Oct 25, 2005 at 21:45 UTC
Seems to me, part of the problem is right there. That was actually minor, compared to a lot of other stuff. A fellow coder who reviewed the system afterwards privately wondered whether the system architect was fully sane or not. I think it's possible to design in efficiency; like the OP suggests. I just think the approach we took for it was absurd (the designer waffled constantly; trading speed for baroque extensions, and simplicity for speed, until we had something that was neither simple nor fast; and any extensibility it might have gained was lost under a blanket of complete incomprehensibility brought on by excessive complexity. ) Also, I think it'll always be easier to refactor maintainable code for optimization than it will be to turn optimized but messy code into something another mortal can understand and maintain. Sure, but it's easier to refactor optimized, maintable code than it is to refactor messy, non-optimized code as well. By definition, "messy" code is hard to refactor. It's certainly possible to write maintainable and optimized code. If it wasn't, you couldn't refactor it; so why not strive to write it clearly in the first place? The effort required to make optimized code maintainable is often higher, but therefore so is the incentive to make it maintainable. A good programmer knows that confusing code will probably cost the company money someday, and avoids writing it. If he can't make the code clear, he at least puts in extra, full page documentation, complete with formal proofs of algorithmic correctness if neccessary. Much of the extra effort involved in optimizing code goes not into the algorithms themselves, but rather into presenting the code and the documentation in as clear, straightforward, and understandable fashion as is possible given the complexity of the implementation. In any program, the most important thing to communicate is usually NOT the code itself: it's the intended results of the code. If the intention of the code is clear, then someone else can fix my bugs, even if I die or get hit by a bus. If no one knows what a given section of code is going to do, or how, or why, well, it's nearly impossible to find or fix any flaws in the methodology, isn't it? So, it's true in a sense that optimizing code is "less important", in the sense that it takes more time (and hence money) for the same output. On the other hand, sometimes optimization means "getting rid of what you don't really need", and simplifying your design; and that can be a big win for performance AND development time. Just some thoughts... -- AC	[reply]

Replies are listed 'Best First'.

Re^5: Optimisation isn't a dirty word.
by Anonymous Monk on Oct 25, 2005 at 21:45 UTC

Seems to me, part of the problem is right there.

Also, I think it'll always be easier to refactor maintainable code for optimization than it will be to turn optimized but messy code into something another mortal can understand and maintain.

Sure, but it's easier to refactor optimized, maintable code than it is to refactor messy, non-optimized code as well. By definition, "messy" code is hard to refactor.

It's certainly possible to write maintainable and optimized code. If it wasn't, you couldn't refactor it; so why not strive to write it clearly in the first place?

The effort required to make optimized code maintainable is often higher, but therefore so is the incentive to make it maintainable. A good programmer knows that confusing code will probably cost the company money someday, and avoids writing it. If he can't make the code clear, he at least puts in extra, full page documentation, complete with formal proofs of algorithmic correctness if neccessary.

Much of the extra effort involved in optimizing code goes not into the algorithms themselves, but rather into presenting the code and the documentation in as clear, straightforward, and understandable fashion as is possible given the complexity of the implementation.

In any program, the most important thing to communicate is usually NOT the code itself: it's the intended results of the code. If the intention of the code is clear, then someone else can fix my bugs, even if I die or get hit by a bus. If no one knows what a given section of code is going to do, or how, or why, well, it's nearly impossible to find or fix any flaws in the methodology, isn't it?

So, it's true in a sense that optimizing code is "less important", in the sense that it takes more time (and hence money) for the same output. On the other hand, sometimes optimization means "getting rid of what you don't really need", and simplifying your design; and that can be a big win for performance *AND* development time.

Just some thoughts...
--
AC

[reply]