There are three things that I think should be foremost in one's mind:

1. Design
2. Design
3. Design

Failure to take the time to really map out the data and logic flows of an application are one of the biggest problems that programmers seem to face when writing anything more significant than "Hello, World".

There are plenty of things to consider, but here are some other things that I look for (incomplete list, of course):

• Well-documented APIs.

  Simply listing what everything will accept and what it will return can clarify a lot of issues.

• Orthogonality.

  Each package, module, subroutine, whatever, should have ZERO side-effects. For instance, if you turn on your car's headlights, your wipers should not come on. Further, because there is a well-established API for the headlight switch, you can replace it with a generic brand and it shouldn't affect anything else in the car. This is orthogonality. If you have a module with a well-defined API, you could throw away all of the code and rewrite it from scratch and have nothing else in the system change, so long as you don't change the API.

• Always use source control.

  You are going to hate life if, after 7 revisions, you've realized that you made a horrible mistake and can't go back (or find out who to blame). Source control handles this for you.

• Do not assume that your code works.

  Prove it. Insert bad data into the system. Did the QA team catch all of the bugs? Assume that your data source will spit out garbage. Will your code catch it? Will your code realize that 13 is probably not a valid month? Test and retest all of your assumptions. Every time you release a build, you'll need to rerun ALL tests. Resting on your laurels is a good way to rest on your sofa collecting unemployment.

• Use bug tracking software.

  At the very least, use a discussion board. All too often developers will use email. When this happens, knowledge is not shared. If Alice has already fixed a bug similar to Bob's and she sees the bug on the bug tracking software or discussion board, she can communicate this. Otherwise, she may never hear about it. Knowledge sharing is important. Facilitate it.

• Learn how to normalize a database.

  Even if you never have to do this, you'll be amazed at how often normalization rules apply to program design. Plus, if you have a clueless DBA (I once had a DBA try to use a user name as a primary key), you can spot the issues.

Cheers,
Ovid

Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.

• Documentation standards strictly enforced
• Code reviews
• Design and analysis taking roughly 1/2 the time spent developing the product
• Proof-of-concept of any components with unknown or risky implementation issues
• Proper documentation of abstractions, including class diagrams, sequence diagrams, and other tools from the UML toolkit
• Full specifications, preferably with use cases, cross-referenced and in a format that your coders can easily understand and absorb
• Iterative development cycles
• Et cetera ...

My advice borne of experience is that the bigger the project, the more "proper" techniques you need to use and the harder you need to push back on management who will pressure you for a quicker development cycle at the expense of proper development methodology.

So just do what you always knew you were supposed to do on the smaller projects, but never had time or management buy-in to do.

In terms of getting management buy-in, I would recommend doing your best to convince them that as a project grows in size, its complexity grows exponentially; thus, company will save money in the long run on any big project by investing in proper design and development practices up front and dramatically reducing maintenance and development costs in the long run. If they're unwilling to do that, I would question whether the project should be pursued in the first place. After all, if they balk at taking the time to do things right, they are implicitly telling you that they don't expect the project to be around very long.

I want to know what is expected, what is required and when it needs to be finished.

And if I might be politically incorrect, if the end client is a moron or not. Stupid clients always make a bad project, hence I will turn away otherwise nice work if the client doesn't seem to have a grasp of reality.

EEjack

Failure to take the time to really map out the data and logic flows of an application are one of the biggest problems that programmers seem to face when writing anything more significant than "Hello, World".

Another interesting tack is: in a language where there is no distinction between data and logic, who needs such mapping out? And such a language is Prolog... the actual algorithms may be heinously slow, but hey at least the whole prototype is fleshed out... it, it allows you to design and run your design: the specification is the implementation in Prolog.

I have to strongly disagree with your statements princepawn, based on my experience. You say:

I have found with Perl it is better to just dig in and avoid a lengthy overy intellectual design phase. And then just evolve the code as need be. Perl is naturally flexible and should be used as such.

I am afraid that choice in language it is unfortunately irrelevant. The "maping out the data and logic flows" in programming projects that I have dealt with had little, if anything, to do with actual implementation detail, or the toolset (or toolsets) which we had chosen to utilize. While I am a proponent of a softare development methodology which leverages some more RAD philosophies (I think XP is closer, but not completely there yet), the "map of the data and logic flow" is usually instrinsically tied to business processes as well as client requirements. In most *large* application development projects, the problem domain is not sufficiently documented so that one actually understands the nature of the problem they are attempting to solve, so without an upfront phase of "discovery" where the problem set is enumerated, and the relationships between the entities are at least partially established.

To do otherwise, is a lack of feduciary responsibility, and is one of the keystone reasons why over 80% of software projects fail (CHAOS Report, Standish Group.)

I agree with eduardo.

When designing *really* big application, you just cannot be too cautious in design phase. If you hope your code will "evolve" - good luck.

Trying to "evolve" code might work is application where you have knowledge of problem area, so your "feeling" about desig decision will be based on your knowledge and intuition - and yes on your luck.

OTOH, *really* big applications are far too complex to rely on plain intuition and luck. What is worse, there might not be ane person understanding the whole picture. So even XP programming will have setbacks.

Just an example from my experience: I was working recently on a huge project for a big bank. When I started, we were supposed to go live in 2 months. When I left 2 years later, we were 2 months live. And most of the time we were some 2-3 months before going live.

Why? Because bank itself lacked understanding how different deparments collaborate. There were couple of users who know in detail how works department they worked in, couple even had experience in work for more than one dept.

But sometimes part of experience was outdated, or not detailed enough - they knew only about problems for "junior" positions.

Also, big problems have most likely previous solution, which is not adequate, and you need to replace it, but previous solution formed perception of users how solution to their problems should look like. So sometimes you cannot provide better solution, but solution they believe is closer what they used to have. Especially if you are in time constraints as we were, there is not enough time to research and provide better solution - you just go ahead and try to simulate part of what they have without much understanding.

I was added to one obscure part of project, what another (decent programmer, but without design experience) was implementing - to help him. We found that in whole company there was only one person completely understanding this obscure, but very basic and important feature of the system (custom enhancement of big system). He was very helpful, but his time was very valuable and not easy to get.

After much research, we found that database contained various types of wrong data from different stages of implementation of custom part, which he did not want to distract us, and was forced to add features what he needed but decided not to implement in old system, because of expected conversion to new system.

And of course this person was very smart when talking, but completely not capable express himself in writing. So you cannot imagine how bad written specifications were...

It was big struggle.

Summary: In *really big application*, do not expect that there is one person, or even a group of persons, who understands it as a whole, or it will be easy to solicit correct specifications. To overcome this migt be substantial part of problem.

pmas

To make errors is human. But to make million errors per second, you need a computer.

Here is an example of why this is a bad idea.