I have not completely bought into the XP way of doing things, but I find many of their arguments compelling. One is to never, ever scrap a project and rewrite from scratch. Rather, they advocate having a good test suite and then continually refactoring the code until it looks like you want it to. This doesn't mean necessarily stopping structural changes, just that they are tightly controled.

A system that is functional has one undeniable trait: it works. You may not like how it works. You may need to do extensive checks to make a two-line change. You may hate its user interface. But it works. Likely, the programmer before you had already encountered subtle issues in the problem domain that aren't immediately obvious. If you don't know about them, you'll end up hitting them in your new version. The result of these refinements may end up in a morass that is no better than the orginal.

I don't fully buy the no-rewrite rule, myself.

I actually subscribe to the opposite theory: plan to write two - the first one is just throw-away. If I could just stop all requirements from coming in for a month, I would be able to finish the other half of my rewrite of our entire perl infrastructure on our team. The first design had holes in it large enough to drive a truck through. The new design answers all the problems of the old design, plus plugs the holes. But it requires a significant rearchitecture.

As far as the programmer before me encountering problems that aren't readily apparanet... doubtful. That previous programmer was me ;-)

This is a massive effort to move from shell scripts that had everything hardcoded in wierd places, ... to perl code that was completely data-oriented. I worked on the shell scripts for a few years before we started the move to perl, so I'm intimately aware of the pluses and minuses of the original code. I designed the first perl infrastructure, handling all the pluses and solving all the minuses of the old shell code. Yes, all of them. But, reality has a way of side-swiping you. In the design, I only took care of all the immediate issues, without looking to future issues. Only by putting it to some use (we delivered two major releases of our off-the-shelf business software based on the original infrastructure) have we figured out what needs to be kept, what needs to be rewritten, and what simply needs to be discarded. Most of what needs to be kept is concepts, not code.

That's just the way I look at it. But I didn't suggest rewriting ('porting') the shell code into perl to management until I did understand the subtle issues in the problem domain. So maybe this is just an opinion given different assumptions... and not an outright disagreement.

If I could just stop all requirements from coming in for a month . . .

But this is exactly the problem XP looks to solve. People doing Waterfall development often say "if we could just get the customers to solidify their requirements . . . ". XP people, OTOH, say that practical experiance shows that customers never solidify their requirements, so let's figure out ways to deal with it.

In general, I can't come up with many good reasons against XP (though there are plenty of ignorant critisim writen against it). Rather, there is just something that doesn't seem quite right about it.

"There is no shame in being self-taught, only in not trying to learn in the first place." -- Atrus, Myst: The Book of D'ni.

I agree with each of your points. I was writing from the perspective of being in the process of creating a project, not attacking one which is written, especially by someone else.

Oddly enough, this is no longer an issue that concerns me. With proper test suites in place, I have, on a number of occasions, made large, sweeping changes in my code. Knowing you'll catch any bugs you make with a test suite substantially reduces "fear based programming."

As a side note, I noticed this:

I woke up in the middle of last night worrying about whether I'd trashed a data variable I was merrily carrying along through a control sequence just because it was ...

That sounds suspiciously like "tramp data." Tramp data is data that is passed, unused, through a call chain so that the later methods/subroutines can get access to it. This is a Bad Thing. Your code becomes more difficult to refactor because subs now take extraneous arguments. Also, if you accidently munge the data, it can be difficult to track down where this happened.

Instead, use access subroutines to handle this:

my $TRAMP;
sub my_data {
  $TRAMP = shift if @_;
  return $TRAMP;
}
[download]

You can then take the variable out of the call chain. Later, if you find something is altering this data in an unexpected manner, you can put data validation here, dump stack traces, etc.

That is you propose to make the tramp data a member of the object? This is a bit similar to making it a global variable - that is you make it here global in the object (dynamic scope, instead of global in the module - lexical scope).

But if you give it a fancy name, "Singleton", then the global is acceptable...to some.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco.

Rule 1 has a caveat! -- Who broke the cabal?

Well, in this particular case, I showed a non-OO way of handling this, so it's not really a data member of an object, but it can be (in this case, it would be class data.)

The particular point, though, is that extraneous arguments should not be passed through a call chain. Providing an access method is merely one step in a refactoring. Perhaps it's all that's needed. Perhaps a greater refactoring is needed. Who knows? It's much cleaner than tramp data, though.

Cheers,
Ovid

New address of my CGI Course.

I don't use OO Perl much at all, but OO's data access principles are definitely a part of my design philosophy. I spent far too many years writing assembler code to fear designing access strategies.

That said, I'm not above getting lazy. It's rarely a call chain pass that I use, but more usually the 'globalization' of a data item that is plucked by one sub after another without controls on what happens in the middle. In OO, the data item is intenal to an object, and the object is supposed to protect itself. It's all too easy to neglect the sanity checks as you add code.

In this particular case, I was modifying a working string in several steps, and added a modification in the middle that broke that chain such that the data was not moving from a known state to another known and stable state. My subprocedures were too finely grained, in other words, and I inserted another grain in the middle that broke it.

That sounds suspiciously like "tramp data." Tramp data is data that is passed, unused, through a call chain so that the later methods/subroutines can get access to it. This is a Bad Thing. Your code becomes more difficult to refactor because subs now take extraneous arguments. Also, if you accidently munge the data, it can be difficult to track down where this happened.

Instead, use access subroutines to handle this:

my $TRAMP; sub my_data { $TRAMP = shift if @_; return $TRAMP; } You can then take the variable out of the call chain. Later, if you find something is altering this data in an unexpected manner, you can put data validation here, dump stack traces, etc.

This is just using a function call to re-implement a global variable. Perl has real built in globals. Use them.

I've seen sooo many people try to dodge using global variables, then ending up with an obfuscated version of the same thing.

"No, it's not a global: it's a single hash that contains all our data, passed down to every function".

"It's not a global, it's stateful accessor function"

"It's not a global, it's a class method" (but used maintain state information about unrelated objects)

"It's not a global, it's a tied hash that references a accessor function" (my favourite)

And so on, ad naseum.

The K.I.S.S. principle says that if you need to store global data, just put it in a global variable, unless you have a good reason not to. If you're shoving too many things down your call stack, perhaps you need to re-design your call stack. I'm very much a fan of using the scope you need, rather than jumping through hoops to avoid it because of some "rule" that says globals are always bad.

Unnecessary use of globals is very bad: avoiding globals when they're useful or necesseary is just as bad.

End of rant.
--
AC

I would argue that if one is afraid to make changes to the underlying (structural) code, then that is the time when one REALLY needs to sit down and restructure the code to clarify and clean it up.

Indeed, thank you for following through more fully on my thoughts. (By the way, who let you into my brain? Get out of there!) Just to clarify my original point a bit, when I said:

They fear causing new bugs. This is a legitimate fear, and definitely should be considered.

What I meant was that causing bugs is something one should legitimately worry about, not that changing important code should be feared. I further explained that I like to use test suites to help avoid creating new bugs, but your Meditation touches on another technique that can help. Namely, writing good, clean code in the first place, or doing a shakedown on code that isn't good and clean. Nice Meditation, in any case. ++

Ha-ha, thank you for taking this in the spirit intended, revdiablo++. I was going to just respond in the original thread but decided this deserved airing in a separate forum. Your expansion above is exactly right. As both Larry and Brian D. Foy have stated, having the courage to whip your code into shape is an essential attribute of a successful programmer.

In my ORSP classes, we point kids towards thinking for themselves, but an equally important attribute is trusting yourself to be able to come up out of the water once you step off the deep end.

Here's a contrarian view:

Things You Should Never Do, Part I

I don't always agree with Spolsky (in particular, he has a much higher opinion of MS than I do), but still I think he has an interesting, down-to-earth perspective on software development.

Now, to quote Linda Richman: Discuss!

First, there are architectural problems. The code is not factored correctly. The networking code is popping up its own dialog boxes from the middle of nowhere; this should have been handled in the UI code. These problems can be solved, one at a time, by carefully moving code, refactoring, changing interfaces. They can be done by one programmer working carefully and checking in his changes all at once, so that nobody else is disrupted. Even fairly major architectural changes can be done without throwing away the code. On the Juno project we spent several months rearchitecting at one point: just moving things around, cleaning them up, creating base classes that made sense, and creating sharp interfaces between the modules. But we did it carefully, with our existing code base, and we didn't introduce new bugs or throw away working code.

What I said I advocated is shaking and restructuring, not a clean sheet rewrite. Much the same as Spolsky advocates in the passage quoted above. I agree with Spolsky's points, especially on large team projects.

My sense is that an initial code development project should have two or three clean sheet revs, but it is rare that I'll look at a rewrite after that, especially with code which has been through the mill with users. The only time I'm willing to consider that is when we're doing a complete architectural revamp, but, even then, I'll have my coders do a completion phase with the old codebase such that everything is consistently implemented in the old paradigm and tested.