#YesEstimates

Humans are constantly estimating.  We predict the future more than any other species.  We estimate where our foot will land with our next step.  We estimate how long it will take to commute to work.  We estimate whether we have time to complete the home improvement project before guests show up for dinner.  A vocal faction in the Agile community has emerged proclaiming that estimates are always wrong, estimation is always bad and people that ask for estimates are foolish.  I respectfully disagree.

A Twitter debate raged the other day between #noestimates zealots and people who think that estimates can be useful.  I’m in the latter camp.  I want to explain why I think thoughtful and reviewable estimates can be valuable, in a medium not constrained to 280-character sound bites.

What is an estimate?

An estimate is a prediction or educated guess about the quantity, size, or volume of something.  In a software context, we estimate how much work we think will be needed to build a component and often convert that effort estimate into an approximate duration estimate for scheduling, budgeting, and resource planning purposes.

Why is estimation challenging?

Estimation is hard for several reasons.  Most fundamental is that many people disagree about what is meant by the term “estimate”.

• To some, an estimate is a promise, “I guarantee I can complete this task in 50 hours.”
• To others, and estimate is a forecast, “With what we know today, I think this task will take about 50 hours.”
• To another group, an estimate is a target, “We are hoping to get this task done in 50 hours.”

For my purposes, task estimates are not promises or targets, estimates are predictions based on a best guess about where we are now and what it will take complete a task.  Because there are no facts about the future, our predictions are based on assumptions and subject to error.

Some of our assumptions will be wrong, and that will reduce the usefulness of our estimates.

Estimates can be wildly distorted if there isn’t clarity and consensus about definition of the work being estimated.

Estimates suffer from optimism, people imagining things going more smoothly than is likely (a special case of an assumption issue). 

I think the most challenging thing about estimation is that most people have minimal coaching in how to make effective estimates.  Combined with toxic environments in which people are punished for inaccurate forecasts, we end up with a crop of well-intentioned people who are hesitant to make estimates for fear of reprisal or who excessively “pad” estimates to avoid trouble – rendering the estimate useless.

Why Do We Need Estimates?

Here on Earth, it is common for executives to go through a decision process to decide which projects to pursue.  The conversation usually sounds something like this:

If we had a software product that could do <thing> it would be valuable.  If the value of the software exceeds the likely cost, then I think we might want to pursue it.

What about the time value of money?  When we get the software will factor in to how valuable it is.

Good point.  I think it would be valuable if we got get it by <date>.

How does the value change if we could deliver it a month earlier than <date>?  How would the value change if we couldn’t deliver until a month after <date>?

Good point.  If we choose to pursue the project, we will need an initial forecast of the amount of time needed to help us decide whether to continue to pursue.

What about risk?  No project is guaranteed to succeed.  How much tolerance to we have for project failure or delays?

Another good point.  Let’s describe what we want as clearly as we can, then involve experienced subject matter experts and a project manager and ask them whether they can build a credible plan to develop software that does <thing> by <date> with <allocation of people and money>.  We will ask them to identify likely risks and document their assumptions.  After our investment in their preliminary planning, we can make a more informed decision about whether we wish to pursue the project based on their estimates of cost and schedule, as well as their risk findings.

But what if their estimates are flawed?

Estimates are never perfect.  We will monitor progress, look for trends, and pull the plug if the project value appears to be in jeopardy.

I don’t understand the context of people whose executives don’t care how much something will cost or how long it will take.  Do they work for organizations with unlimited resources?   

If we step away from the software context, would you hire a contractor to remodel your house who told you, “I don’t know how long it will take or how much it will cost, just keep giving me money and I’ll let you know when we’re done.”?