This can divide into two questions:

1. What design documents must I produce in order to be in compliance with my companies accepted methodolgies, and how can I make those documents as effective as possible?
2. What design documents give me the most "bang for the buck"?

I'll assume you're asking the second question.

Either way, design documents can become a big burden on a project, particularly when you try to do a bunch of design up front, but the design migrates as you start to build it and get more experience (or the customer changes their requirements). I favor being minimalistic, and try to avoid throw-away work. The more pages of design doc you write, the more you'll have to update as the system grows. Write down too little, and the design isn't communicated. The trick is to find a balance.

For database systems, the document that I've seen get the most use is a current E-R (Entity Relationship) diagram, stuck up on a wall where everyone can see it, and where people can gather around it and exchange thinking. At lot of micro design can happen by getting the right people in front of the diagram for a stand-up design session.

The key point is that this diagram has to be visible and current. That means that people get to write on it as changes get made, and that you'll need to periodically produce a new version.

The E-R diagram is logical. If you find that you need physical design details, tack up the DDL next to it. I suspect that you'll find that fewer people will refer to it.

Another useful document I've found is a simple log of issues and resolutions. It should provide a way of answering "why did we decide do it that way?" questions, and record enough information about assumptions or trade-offs to be useful when new information is available. A lot of decisions get made in a project, and many can pass by some people unnoticed or unappreciated. The logs I've kept have always proven valuable later in the project.

O wise monk,
Thank you for your most wise reply.

I am afriad that our project has has swung too far towards the
Write down too little, and the design isn't communicated.
end of the spectrum. And I need to convince the team that this
is a problem. But I am not up on current design document stuff.

At my job we are to embark on the design and implementation of a software system that is part data warehouse and part data mining system.

For a project of that size and complexity, I'd strongly recommend that you write a throwaway prototype before you do any detailed, intense design work. By all means write an overview before your prototype... but keep in mind that you probably don't know enough about the problem to do a detailed requirements plan or interface specification until you've tried it at least once, in real code. (If you, or other people on your team, have written data warehouse/data mining systems before, all bets are off.)

I'm a big fan of learning by doing: write little throwaway prototypes for each feature as you go. They don't have to be complete, or easy to use, or fast... their purpose is to give you some idea of what a solution looks like, so that you can refine it for your finished product. If you keep the parts separate and the interfaces between them clean, you can refine incrementally as you go with minimal pain, and you'll probably end up keeping some prototype code, but "plan to throw one away; you will, anyhow".

It just so happens that Perl makes an excellent prototyping language.

--
F o x t r o t U n i f o r m
Found a typo in this node? /msg me
The hell with paco, vote for Erudil!

Thanks Fox,
We actually have done a "pilot" system.
It is a mess. There is no documentation
or design documents other than the looking
and the source and some very badly written
MS word files called "the specs".

I guess what I am ask for what are the formal
definition of a software design. What should a
Functional Specification address? Who should write
it? One person? A user? A team of users?

after being bitten and seeing some of my coworkers bitten one too many times, i sat down and wrote up an outline for a Requirements Specification. it's basically a list of questions that really need to be answered before development begins to make sure that the developers and the customers are all operating with the same assumptions. the Requirements Spec is in addition to a regular functional spec (the Joel article that guha mentions is excellent for that).

depending on the size of the project, i'll either insist on a full requirements specification being written, or, for smaller projects, just take a printout of that outline to the initial meetings and use it as a checklist to make sure everything that needs to be asked is.

don't skimp on at least gathering the requirements. i'm not convinced that they always need to be written up and signed off on but you definitely need to go and systematically make sure you're all on the same page. be precise and annoyingly thorough. i've had meetings with customers before and they say that a "small" class will be using the application. i imagine that that means around 20 - 40 students. in their mind a "small" class is 400 students. only an order of magnitude difference. not finding that out till the end of the project can be a painful experience.

anders pearson

This document which is less entertaining but still something to think about when considering design documents. At work it is regarded as the law.

I have also found this document on Functional Specification valuable AND rather entertaining.

Having this general design in-hand and signed off on by your audience will prepare you for the unavoidable eventualities of users who are dissatisfied with the product or claim that the product lacks intended features.

That sounds you're preparing for war with your customers from the outset. Don't go there! Expect that they'll change their mind about things, and be prepared to adapt. Part of being prepared is being light on your feet with documentation, to minimize that amount you'll need to change when new requirements come in and you have to make changes.

If you're concerned about schedule dissatisfaction, prepare a short document for each change request that recaps the changed requirements, and details the consequences (in terms of schedule and resources) of the changes. Get the customer to sign these.

Or better, engage your customer more frequently, and have them help prioritize the list of what they want to see next. If you're following one of the Agile methodologies (e.g., XP), you'll be in a position to "release" fairly frequently. Giving the customer the chance to reprioritize lets them change their mind, and greatly reduces the chance of them coming back at you with a "this isn't what we wanted!" argument.

After the requirements have been signed off on by the userbase, more detailed, technical design documentation can be prepared.

Don't rely on the users to be any more prescient than the programmers. Rather than having them sign off on what they think they want ahead of time, work with them (or at least one of them, who becomes the users' representative to the development team) during the project. That way, when you discover that feature X will be N times more difficult to implement than you thought, you can ask them just how useful X would be, and whether it's worth it. Or the user will, after actually using the product for a little while, discover that one of the interface widgets they insisted upon is actually a pain to use. Better to find that before the whole user base starts to whinge....

Moral of this story: Requirements change. Plan for it.

--
F o x t r o t U n i f o r m
Found a typo in this node? /msg me
The hell with paco, vote for Erudil!

After the requirements have been signed off on by the userbase, more detailed, technical design documentation can be prepared. Having this general design in-hand and signed off on by your audience will prepare you for the unavoidable eventualities of users who are dissatisfied with the product or claim that the product lacks intended features.

While everyone's experiences will vary on this subject, this was the way it was done at a consulting firm I worked for. We tried to document everything up front (mistake #1), have the user sign off on them (mistake #2) and when the user wanted changes, the Project Manager had to say "You signed off on this...if you want changes it will cost you $x".

The customers would typically piss and moan about this until the sales people (weasels) would buckle under the pressure and tell our boss to tell us just to do it. This of course would lead to projects being a) overbudget (internally we would mark these extra hours as "unbillable") or b) late.

I am now pretty convinced that attempting to document things up front and having a user sign-off is not the way to go about things. They will inevitably want changes...oh...and they'll want the changes in the same amount of time as the original project.

Granted, I will write up as much documentation as I need to get the job done, but that's about it. Luckily, as an independant doing mostly sub-contracting work, I can bypass most of the issues I've had in the past and focus on what needs to be done, not what a bunch of sales weasels want. I'm starting to see the value in what the Extreme Programming people have to say although I haven't had the opportunity to try any of it yet.

mike