I've been estimating programming and project tasks for a whole bunch of years, and while my estimates have gotten better over time, they're still all over the map in some areas. Lately, I've been pondering why this is, and what I can do to improve. What follows is a progress report of sorts, in the hopes that it generates some thinking and discussion.

To help with the pondering, I use a really simple scheme for collecting data1: For each task on the immediate or near horizon, once I'm confident that I understand the task well enough to estimate it, I record

  1. an estimate, in ideal (uninterrupted, adequately fed and caffienated) programming hours,
  2. a confidence rating, from "wild guess" through "fairly comfortable" to "confident",
  3. a timestamp

Re-estimation is allowed and encouraged. If I learn something more about a task (or have a sudden inspiration or second thoughts) before I start the task, I record a new estimate. When I start work on a task, at the end of each "session" (typically at the end of the day), I'll record

  1. how many hours of work I actually applied to the task,
  2. how many (ideal) hours I now believe remain,
  3. a timestamp,
  4. any relevant notes that'll help me later

The bookkeeping overhead is fairly light--on the order of a few minutes a day. The tough part is reconstructing how much time was actually spent on task.

With data in hand (or in Excel, on in a database) at the end of an iteration, it's easy to cull the data and make graphs. Here's where things get interesting. With graphs, it's easy to spot patterns.

A pattern that's surprised me is a tendency on some types of tasks--generally tasks where there are two or more technical unknowns that need to be worked through during the task--for my confidence to drop mid task, and for a creeping period of apparent non-progress to set in, as the work remaining seems to stay constant no matter how much time I put it. A chart for this effect looks something like
| rrc e | rr c f | r c f | r c o | rrrr c r |r c t | c | c +------------ MMMMLLLMMNHH time ->

Here, 'c' represents work completed, 'r' represents work remaining (plotted as a stacked bar on top of work completed), and L, M, and H are confidence indicators.

At a coarse level, the data suggests that I should either be more aggressive in breaking tasks in smaller pieces, or I should bump an estimate up about 15% for each additional (n>1) technical unknown in the task.

The data also confirms something I've known but haven't admitted: I'm really, really bad at estimating how much effort is required to get a UI looking good. I can accurately estimate how long it's going to take to prototype a page in a web app, for example, but the "fit and finish" work almost always take a lot longer than I think its going to. But now I know, and can adjust my estimates.

At the risk of going overboard with data collection, I'm thinking about keeping a running count of significant interrupts that happen while I'm on task. I know interruptions are a problem, but don't have a good handle on how much they cost.


---
1 I make no claims of originality on my data collection scheme. I have a vague recollection that the Personal Software Process (PSP) does something similar, but at a higher cost.

Replies are listed 'Best First'.
Re: On Improving One's Estimates
by simon.proctor (Vicar) on Feb 23, 2004 at 21:42 UTC
    The most accurate way I have ever done it is this:

    Add weeks till you stop feeling ill.

    Subtract days till you stop feeling guilty.

    I have never missed a deadline.
Re: On Improving One's Estimates
by flyingmoose (Priest) on Feb 24, 2004 at 03:05 UTC
    My favorite method used to be "think of how long it will take, double it, and add 50%". Err, 2.5x what it will really take. But even that isn't good enough anymore, since I'm getting whacked from nine billion directions at once.

    My estimation is usually fairly accurate since I've been doing the same thing for over two years now, the one thing I can't handle is outside requests and changes in priority. So I usually estimate things as "will take 2 days of uninterrupted development time", which usually means it will take a week sharing between tasks, or in worst case two weeks. I'll convey both the 2 day and 1-2 week number to project/program management, such that they are aware the task isn't hard, I'm just doing a lot of things at once.

    I'd be interested in other systems people have for dealing with estimates as they relate to multi-tasking. My organization usually has 5-6 ongoing projects that affect me at any one time.

    Some folks I know have resorted to a priority queue system, and this works fairly well, though it can be hard for some to accept. Essentially you maintain a list of ordered priorities, and then when some one comes in with a request you let them know where it is in your queue. Thus a relatively simple item may take a month to get accomplished.

    Currently, I'm on the "cooperative multitasking" model of estimation. For those that play with OS's, you will know this doesn't work very well. You have to let everyone know there is a very good chance their "process" will be delayed indefinitely.

Re: On Improving One's Estimates
by cchampion (Curate) on Feb 23, 2004 at 23:12 UTC

    A similar method of setting an estimate is reported in this article.

      A dependency is when you have two tasks, one of which must be completed before the next one can begin. I've found that with software, the dependencies are so obvious that it's just not worth the effort to formally keep track of them.
      Usually, I like what joel writes. Not 'cause i agree or disagree, but because he usually thinks things out.

      I've also been known to be proven wrong.

      Why to use MS project or something that has equivalent functionality.

      • Dependencies are dependencies. They are on resources, which are time and people. You can't make them vanish.
      • Tracking them are of the utmost importance. For instance, take a simple junior-senior level programming relationship. One may do the architecture and API type functionality. The second, may do more interface programming and code maintenance. Yes, the dependencies are more obvious, but when it gets beyond these two people, what happens then? How do you keep track of who can do what?
      • Time is a resource. When people take vacations and what not, you must make good estimates. Even if you don't factor in the warm-up time after a vacation, you are better off than if you never counted it in. You have things like holidays as well to worry about. What if one of your team-mates is jewish, and needs to take other holidays? It's a pain in the ass to do in excel. Especially when you wish to tell QA something will be ready, and your deployment team as well.
      • Time estimates are hard to do when you don't do them often enough. I know in an ideal time situation, it will usually take me n*3 to complete something. If something is really hard, n*5. Good software schedule management programs will take estimates and remember how on track you usually are.

      While it may be a little more complex than excel, learning how to use the tools, especially when it's not just one developer doing something and nothing else, it becomes invaluable.

      Sorry Joel, but you are wrong.


      I used to be a funny character, now I'm just 4 bits.
Re: On Improving One's Estimates
by pboin (Deacon) on Feb 25, 2004 at 12:48 UTC

    A few thoughts on timelines:

    1. Always let people know right away that spec changes are going to cost in time or money, and sometimes both.
    2. Always remember the Good-Fast-Cheap triangle (pick two). It's a law that you can't change.
    3. Most importantly, always remember that letting someone bully you into changing an estimate/schedule w/o paying more in time or money, you're ultimately screwing both you, the developer and he, the customer, even if he doesn't realize or acknowldege it!

    Do everybody a favor, and be a bastard about it.

Re: On Improving One's Estimates
by castaway (Parson) on Feb 24, 2004 at 11:49 UTC
    I like the re-estimating idea.. I read that joelonsoftware article a while back and thought he had a good point with that..

    What really gets me is the fact that (here, at least) I get told off for booking time on things like 'learning' (== keeping up to date on techniques etc. ), and told I should book it all on a project of some sort.. Fine by me, but that means I then need to over-estimate (which I cant, cos the current project got estimated by someone else, and there are no extra days, stupid tight time budgets), or run out of time and look bad.

    Luckily, no UIs in this one, but some 'unknown techniques' which Ive had to learn first, those are the real pains..

    (I should start doing re-estimates tho.. tho Im absolutely terrible at estimating time at all, or indeed remembering how long other similar things took me.. time to start making notes I guess..)

    C.

Re: On Improving One's Estimates
by astroboy (Chaplain) on Feb 25, 2004 at 11:18 UTC
    One good resource for estimating projects is Steve O’Connell’s wonderful “Rapid Development” – my favourite IT book of all time. It discusses two ways of estimating projects – past experience and estimation methodologies that can be encapsulated in software tools. I prefer the former, although if you like the latter, O’Connell offers a free tool on his website: www.construx.com

    Estimations based on past experience requires all team members keeping accurate records. In my last job, the organisation introduced a rule that required everyone to enter their time into the time database daily – the customers were allowed to log in over the web at any time and get reports of their projects up to the previous day. Well, a huge fuss ensued (and I was one of the loudly indignant ones!), but it died down pretty quickly. It really only required five minutes per day.

    This method worked well for us, as there are only so many permutations of project types that we engaged in. It also works well for because our projects tended to be quite small – two to 10 members per project, 3 to 12 months in length. We know how long it would take to gather requirements, and write the tech spec. Then after we divided the work into the number and complexities of GUI screens, stored procedures, Perl/shell scripts, migrations, OS/DB installation etc, we could calculate the work based on how long it had taken us to do the same number of items in the past . This proved to be reasonably accurate in terms of determining billable hours/cost to the customer.

    In addition we used to build in a contingency – typically 30% when we had to liaise with 3rd party suppliers (there are so many unknowns and gotchas), less if it was all in-house.

    After all the dependencies etc are worked out using MS Project equivalent we can then work turn our billable hours/estimations into delivery estimations. If the deadline is more important to the customer than having the wizzy features, we may simplify the requirements/spec and the estimates would be adjusted accordingly.

    Once the project was under way, each team member had ownership of their own work. Everything was based around weekly status reports which are submitted to the customer - which would include a breakdown by team member of the following
  • hours spent on each project component
  • what was accomplished
  • what the member intends to accomplish in the next week
  • what they intended to accomplish the previous week, but didn’t
  • what they worked on that was not planned for at all.

    The advantage of this is if anything was encountered that would adversely affect the project, we could take immediate action. So if we knew the budget was wrong, or the time lines weren’t right, we could alert the customer early on, rather than waiting until the problem became unmanageable. This is anathema to many people – they never admit problems, and just hope that with a bit of overtime or a few extra contractors that the issues will disappear. I found customers actually liked these issues raised as soon as they were discovered, but maybe we’ve been lucky with our customers.

    This estimation system works well for us, but may not be everyone’s cup of tea. If your organisation’s culture isn’t amenable to this level of scrutiny, then you’ll probably have a hard time with it. (Believe me - if I had my way, it never would have been introduced at my last company). I doubt it will work for large projects. It may not work well if you have a wide variety of expertise. If you suddenly have a junior whose productivity is quite low, then you can’t use the time database to estimate their work, and it becomes a suck it and see for their work.
      If your organisation’s culture isn’t amenable to this level of scrutiny, then you’ll probably have a hard time with it.

      The problem I have with the type of scrutiny you describe (and I've been there before) is that the information flow is entirely too much of a one-way street. Information gets extracted and shovelled into the gaping maw of Microsoft Project, with little feedback other than "hurry up!" to the troops. This serves the interest of the top-level stakeholders, but fails to help me recognize patterns in my estimation behavior.

      Without seeing these patterns, how can I improve? It does me little good for a spreadsheet somewhere to say "When Dave says X, add 30%". There's little instructive value in that number. Is that 30% on all estimates? On some? Am I more accurate when estimating some classes of tasks than others? Are there patterns I need to know to recognize, so that I can avoid the "just another day, i'm almost there" trap?

      The last time I looked at O'Connell (a few years back), it seemed that he was good on the planning and execution side, but had big gaps in the individual improvement area. If that's changed, I'll revisit his site.

        These are all valid points. I think this estimation technique does not apply to all projects, but I would like to offer the following feedback:

        "When Dave says X, add 30%." Using the above methodology, the tasks are broken down to a reasonably fine granularity. You certainly want Dave's input, but basically, you run a report against the database and you see how long Dave took last time he did the same task.

        Detailed time recorded in a database allows you to query estimated vs actual time, by person by task, so revisions to the estimation process can be made over time. It also helps when answering your question: "Am I more accurate when estimating some classes of tasks than others?" as you have metrics that will tell you. (I think this is why O'Connell feels that estimates based on past experience only works if you have hard data.)

        RE: "Information gets extracted and shovelled into the gaping maw of Microsoft Project..." To be honest, I'm not a big fan of MS Project. It does help to get an overview of dependencies, and shows what tasks can be done in parallel with the available resources. But I find it unwieldy and cumbersome. So while we use it, it isn't the guiding tool in the project. (I also think that some project managers -- particlularly those who have fallen into the role from non-technical backgrounds -- use it as a crutch in lieu of actual project management. Hey, if you're fiddling with charts then you're justifying your income, right?)

Re: On Improving One's Estimates
by chance (Beadle) on Feb 24, 2004 at 18:18 UTC
    my method:

    1. think about problem, and scribble down list of steps involved in solving it. Write down how long each step will take, keeping in mind 'breakage'. I.e. two six hour tasks are probably going to take me two days to do, unless I'm willing to put in a twelve hour day. Add up times.

    2. refine guess by making sure I included time for debug,documentation, installation, testing, etc .... as appropriate.

    3. If I feel pretty confident that I haven't forgot anything, double the estimate

    4. If I think I might have forgot something, or done too much hand waving on any particular step, triple the estimate.

    you might think there would be recursion problems, but this is the most accurate method I've ever used. And remember, this is not an upper bound, this is something like the median amount of time it will take. The original guess from step 1, in a very half assed way, is about 1 standard deviation. Don't know how much these multipliers vary on a person-to-person basis.