Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Common Software Development Mistakes

by eyepopslikeamosquito (Archbishop)
on Aug 07, 2011 at 13:06 UTC ( [id://919075]=perlmeditation: print w/replies, xml ) Need Help??

Cleaning up some old files today, I noticed some old notes I'd written on Classic Software Development Mistakes. Rather than throw them out, I thought I'd share them here.

Adding manpower to a late software project makes it later

-- Brooks' Law

This is probably the most common mistake I've seen over the years. Throwing more people at late projects seems like a reflex reaction for most managers. While Brooks reaffirmed his law in his Mythical Man-Month Anniversary Edition twenty years after he first proposed it, Steve McConnell has argued, unconvincingly in my view, that Brooks' Law no longer applies:

The time that existing staff spends with new staff certainly does take productive time away from the project, and time spent training can be frustrating to existing staff members who are already feeling pressured to complete their own work. But the loss is nothing like the ratio needed for Brooks’ Law to be true.

-- Steve McConnell

While McConnell's argument sounds logical, the crucial point here is how capable and well-trained are the new staff thrown at the troubled, late project? If they are capable and familiar with the domain, fair enough. Yet in my experience, capable staff tend to already be extremely busy. Moreover, their bosses are unlikely to release them. That only leaves B-grade performers (whose bosses may be only too happy to let go for a few months) and new hires. Now, throwing untrained and mediocre developers at a troubled project has a devastating effect on morale. Existing project staff, already under extreme pressure, now find themselves spending inordinate amounts of time training and spoon-feeding their new group of hairdressers and telephone sanitizers ... which typically leads to the better project team members resigning, the less capable ones being unable to secure a new job. Only the dead wood left behind. That is a terrible long term strategy for any company.

Sadly, I'm yet to meet a high level manager who has actually read The Mythical Man-Month. Or who knew what Brooks' Law is.

Of course, blaming it all on Brooks' Law is an oversimplification. Often the project is not really "late" at all, but a victim of overly optimistic scheduling and poor estimation. Poor hiring and training can also cause a project to take longer than it would were a stronger team available.

I'm not the sort of engineer that Google usually looks for. You see, I don't have a computer science degree from an elite university, or indeed any degree at all. I don't have any pretensions toward being a computer scientist, though I'm familiar with enough of the concepts and terminology to be able to work with them. I'm more the type to know and use tools -- preferably popular free and open source tools -- that other people have built, but of course Google's not very interested in that. I've also spent a lot of my time in the tech field on teaching, mentoring, encouraging software teams to adopt best practices, building relationships with other teams and with users of our software, advocating openness and transparency, and so forth, none of which Google's hiring practices care about or look for. It's all algorithm pop-quiz, and I'm crap at those.

-- Skud Blog, A note to Google recruiters

As illustrated by the above blog, Google's infamous hiring practices are often criticized for being ridiculously onerous with an over emphasis on excelling at algorithm pop-quizzes. Certainly, Google set the hiring bar very high and must surely reject many candidates who would have turned out to be model employees if given the opportunity. Yet I agree with their overall hiring philosophy: that it is far better to reject good candidates than to hire poor ones.

A common hiring mistake I've seen over the years is for a non-technical manager to hire developers without asking developers to be involved in the hiring process and without asking candidates to write any code. If you do that, it's very easy to hire developers who can hardly code at all. These sort of hiring mistakes are incredibly expensive and demoralizing because working with mediocre programmers usually frustrates the stronger developers, sometimes to the point of resignation, while sacking new hires can be a very unpleasant and traumatic experience for all concerned. Not to mention the cost (time, money and opportunity) of needing to re-hire.

Most product managers that I have worked with would rather ship a failure on time than risk going late.

-- Alan Cooper in The Inmates are Running the Asylum

Knowing the "right" product to build is hard, very hard. Who would have thought that twitter would be such a success? Well, it sounded like a daft idea to me. Actually, it still does. :) So product managers are an easy target. The two most common product mistakes I've seen are shipping a failure on time (as indicated by the above Cooper quote) and building the wrong product. Another more subtle strategic mistake is trying to build too many products thus spreading your resources so thinly that none of them are any good.

A classic example of building the wrong product, mentioned in Patrick Copeland's innovation at Google talk, was Bill Gates' unshakable belief that users wanted to see a Windows "Start" button on their mobile device. As you might expect, most Microsoft employees were reluctant to challenge the company founder on what product to build. Maybe pretotyping would have helped here.

I could rabbit on and on, but that's enough from me. I'd love to hear about the software mistakes you've seen. To help jog your memory, I list below a summary from McConnell's Classic Mistakes Enumerated.

People-related mistakes

  1. Undermined motivation.
  2. Weak personnel.
  3. Uncontrolled problem employees.
  4. Heroics.
  5. Adding people to a late project.
  6. Noisy, crowded offices.
  7. Friction between developers and customers.
  8. Unrealistic expectations.
  9. Lack of effective project sponsorship.
  10. Lack of stakeholder buy-in.
  11. Lack of user input.
  12. Politics placed over substance.
  13. Wishful thinking.

Process-related mistakes

  1. Overly optimistic schedules.
  2. Insufficient risk management.
  3. Contractor failure.
  4. Insufficient planning.
  5. Abandonment of planning under pressure.
  6. Wasted time during the fuzzy front end.
  7. Shortchanged upstream activities.
  8. Inadequate design.
  9. Shortchanged quality assurance.
  10. Insufficient management controls.
  11. Premature or too frequent convergence.
  12. Omitting necessary tasks from estimates.
  13. Planning to catch up later.
  14. Code-like-hell programming.

Product-related mistakes

  1. Requirements gold-plating.
  2. Feature creep.
  3. Developer gold-plating.
  4. Push me, pull me negotiation.
  5. Research-oriented development.

Technology-related mistakes

  1. Silver bullet syndrome.
  2. Overestimated savings from new tools or methods.
  3. Switching tools in the middle of a project.
  4. Lack of automated source-code control.

Perl Monks References

References

Replies are listed 'Best First'.
Re: Common Software Development Mistakes
by iguanodon (Priest) on Aug 07, 2011 at 18:16 UTC
    ++. I always enjoy reading your entertaining and thought provoking posts. I'd follow you on twitter if I didn't think it was such a daft idea :)

    Throughout my career (a small number of jobs but a lot of different management regimes) I have also not met a high level manager who has heard of Brooks' Law or The Mythical Man-Month. Most of the grunts doing the work and most of their immediate supervisors understand it, but at a certain level I guess it's just not required reading.

    So when we inevitably run into problems (pick any ten from the list above) and we're asked why we don't "just bring on some contractors", the favorite response is "Nine women can't make a baby in one month". It always gets laughs on the conference calls, then the next question is "but seriously, why don't you just bring in some contractors?"

      It helps immensely if one is also conversant with the scale of the project that Brooks is writing about. OS/360 was a monumental achievement: arguably, all IBM's non-Linux OS's are still based on the OS/360 concepts, though we've gone from a Cambrian weird wonder to a velociraptor at this point. (edit: typo)
Re: Common Software Development Mistakes
by JavaFan (Canon) on Aug 08, 2011 at 09:38 UTC
    While McConnell's argument sounds logical, the crucial point here is how capable and well-trained are the new staff thrown at the troubled, late project? If they are capable and familiar with the domain, fair enough.
    I think the fallacy here is to assume that anyone working on the project needs to be an as top-notch programmer as you are. Usually, that's not the case. Many projects only need "expert" knowledge for a fraction of the project, but the rest is just "grunt" work. There's often the odd webpage, conversion script, monitor, cronjob wrapper, test suite, documentation, etc, that can be offloaded to a "lesser" programmer, freeing up "expert" time. If only all they do is fetch coffee, or read Perlmonks for you.
    A common hiring mistake I've seen over the years is for a non-technical manager to hire developers without asking developers to be involved in the hiring process and without asking candidates to write any code.
    Has that actually worked for you on a structural bases? Is it worth the time it takes? (Cause not only does the developer-to-be write the code, you need people to judge it). How do you deal with the fact many candidates are under stress? Are you giving them new, non-trivial tasks, or do you give them (part of your) existing code base, and ask to modify it - the latter is for many companies a far more realistic scenario.
      Has that actually worked for you on a structural bases? Is it worth the time it takes? (Cause not only does the developer-to-be write the code, you need people to judge it). How do you deal with the fact many candidates are under stress? Are you giving them new, non-trivial tasks, or do you give them (part of your) existing code base, and ask to modify it - the latter is for many companies a far more realistic scenario.

      From the articles I've read, even a trivial programming problem is useful. It proves that they can code, not just that they say they can. If you want to go detailed and dig into how well they can code and if they can work with your coding style all the more power to you, but that does (as you note) take time and effort on the part of the interviewers. But even a simple 'take two numbers from the user, switch which variables they are in, add them together, and print the result' is better than hiring them and then finding out they are totally incompetent at anything coding related.

        I will often ask candidates to write down a few lines off code on a white board, or a piece of paper, but in those cases, I usually don't really care about the details of the code, as I will ask them to explain what they are doing.

        But I wasn't getting the impression that was the kind of coding being referred to by the OP. I've heard from other companies giving coding tasks that exceed the size of a whiteboard - but I've never heard of any research done about the value of this. We've discussed this at our company (that is, giving potential hires a coding task), but we're rejected it; the cost of creating a task that's representative of the work we are doing is just too great. (And since we get in lots of candidates, we'd need to cycle between tasks, as people do pass around the questions that have been asked). We care more about people fitting in the development methodology than about their exact level of Perl coding (which can be trained).

      I think the fallacy here is to assume that anyone working on the project needs to be an as top-notch programmer as you are. Usually, that's not the case. Many projects only need "expert" knowledge for a fraction of the project, but the rest is just "grunt" work.
      Good point. For fun, let's analyze a specific scenario. Let's assume that the "Perl 6 project" is "late" and that we have some (below average) developers available to throw at it with the goal of "finishing" it sooner. For this hypothetical scenario, let's loosely define "finished" as all Synopses complete and all implemented in at least one Perl 6 implementation with similar performance and stability to perl 5.14. I don't want to get distracted with nit-picking the "definition of done" here. The point of this little exercise is to gain insight into Brooks' Law: where does it hold, where should it be repealed? To simulate what I typically see in commercial projects, let's further assume that these new developers have little prior knowledge or experience in Perl.

      Now, adding new "below average" developers to perform highly skilled work, such as finishing the Synopses or improving the Perl 6 parser seems counter-productive to me, a classic case where Brooks' Law holds. In which of the following areas would adding more people help finish the project sooner?

      • Writing the Perl 6 specification
      • Perl 6 development of parsing, compiling, ...
      • Perl 6 runtime development (run machine, garbage collector, introspection, foreign code, parrot, ...)
      • Perl 6 library development
      • Writing Perl 6 documentation
      • Development of installers
      • Development of tools (e.g p5 to p6 translator, pod2html, ...)
      • Development of Perl 6 CPAN
      • Writing test suites
      • Using Perl 6 and reporting bugs and issues found
      • Maintaining the bug database, issue tracking, ...
      • System administration
      • Licensing
      • Marketing, maintaining Perl 6 web sites, wikis, ...
      • Release engineering
      • Project management
      I may have missed some key areas above or broken things down wrongly. Please feel free to suggest improvements.

      I suspect that most open source projects (such as Perl 6) are well ahead of most commercial ones in terms of documentation and in partitioning the system into independent components. Accordingly, I feel Brooks' Law is less applicable to open source projects than to commercial ones.

Re: Common Software Development Mistakes
by sundialsvc4 (Abbot) on Aug 08, 2011 at 21:02 UTC

    I submit that the mistake which is most commonly made is to say, “the job of a software developer is to write code.”   In the last few years, the word, engineer, has quietly dropped out of this job description ...

    And let me be quite blunt here:   if the word, “engineer,” is in the job-description, bean counters might feel a little ill-at-ease at “outsourcing” the work to people whose primary calling-card, to the bean-counter, is that “they’re cheap.”   The word, “engineer,” is usually preceded by words such as “licensed” and “professional.”   Even the most insipid bean-counter probably would not advocate hiring someone “who works for cheap” to build a bridge or a skyscraper.   But they would hire abjectly-unqualified people to work on critical software systems, whose consequences of failure (or even of error) might be much higher than that of an inadequate physical structure or device.

    There is no such thing as “a software developer.”   The writing of computer software is always an engineering undertaking.   Even though the finished product is intangible, the effects of it are not; and, the risks of it are not.   The mere skill of “writing code” is no more a description of the task than “proficiency with a hammer” is to a professional cabinetmaker.   Yet, whereas even the simple re-paving of a surface street (or the building of a cabinet) would be routinely undertaken with scrupulous discipline, critical software projects are often undertaken by workers who are barely trained ... and with total disregard to both engineering and project-management practice.

    And let me be very clear about one thing:   I have had the privilege of working with certain third-world software engineers who were, and who are, unquestionably brilliant.   I do not believe that “where you were born” is an indicator of “how good or how prepared you are.”   (The thought is obviously absurd.)   Nevertheless, people who aren’t qualified, are being hired, under conditions which clearly affirm that the people who are making those hiring decisions “know not what they do.”   People are being drawn halfway around the planet only to have their hopes (and skills) dashed on the rocks.   I do not like to see good people being thrown under a bus.   And, quite separately, I don’t like seeing any software project that has doomed itself to failure for lack of discipline, knowledge, or design.

    Sometimes, I think that, as an industry, we have done this to ourselves.   The electrical and mechanical engineering disciplines ... hell, the (unionized) electrical and mechanical trades ... have done a good job of educating the public, and the bean-counters, that the things which they do are not an idle undertaking.   But, in the software industry, I don’t think we’ve done ourselves justice.   And there is a very long string of dissatisfied customers to show for this.

      (great thoughts sundial)

      one of the reasons we've done this to ourselves is that software (even in its most trivial applications) is actually quite complicated, but we put a gui on it, and make it look easy.

      One of the reasons that designs fail is that we fail to take that complexity fully into the design, and therefore fail to parcel it out to the right people, with the right know-how, and to top it all off we put some money-man or non-software boss on top of it all and expect them to be useful to us.

      Sure the money's nice, but they really don't contribute much by way in terms of expertise.

Re: Common Software Development Mistakes
by zakame (Pilgrim) on Aug 08, 2011 at 06:43 UTC

    This, plus this article on c2 describes what I'm noticing at my workplace now. It seems to be growing worse here as there's a previous cycle of people coming in and leaving so frustrated at the way the company works. Perhaps it may be the norm in such a line of work as ours, but still, I wish it could be a better place.

Re: Common Software Development Mistakes
by raybies (Chaplain) on Aug 08, 2011 at 12:09 UTC
    This may seem unfair or unimportant to some, but I know one problem we have here is English (or native) language skills. If I can communicate with my coworkers in plain speech, then it makes my job a whole lot easier, as we do a lot of phone conferences. It's very difficult to get what the customer wants--adding a language hurdle is one of those problems that I've found really slows down our development time. When his part of the project is mentioned in a meeting he does the same vapid nod, head-bob he does when you think he's understanding you... sigh...

    We hire a lot of new guys here. All but one of our native speakers has been able to make significant contributions to our group(and the guy who didn't work out was a 15+year engineer who really should've retired 10 years ago...he's been moving from organization to organization, finding new and unique ways to do nothing for quite some time). It takes a while sometimes--sometimes just shadowing us, but it beats the shadow who can never tell me if they actually understood what I've been demonstrating.

Re: Common Software Development Mistakes
by davies (Prior) on Aug 08, 2011 at 12:56 UTC

    I'm inclined to accept McConnell's argument that "Controlled projects are less susceptible to Brooks’ Law than chaotic projects". But I'm also inclined to believe that well controlled projects are less susceptible to being late in the first place. And (yes, the distinction's not binary, but all the same...) if project management is chaotic, then it is reasonable to expect the management of the addition of staff to be as chaotic as everything else. In that situation, I'll bet on Brooks every time. But if a well controlled project gets thrown out by something external - the Japanese tsunami for a tragic example - then I could easily imagine the addition of staff improving matters.

    Regards,

    John Davies

    Update: I don't accept that Linux disproves Brooks. Brooks's point related to adding manpower to a late project.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://919075]
Approved by Limbic~Region
Front-paged by Limbic~Region
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2024-03-28 08:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found