Thinking for a Change

Embarassing moment #239: Turning up late to teach an estimating and planning course.

Danger: Estimator at work!

Estimation is (sometimes) useful, but it has its dangers. Estimation provides us with useful information, to base management decision on. This information and the way we get it can be misused. Maybe that’s why so many people are so afraid of estimating and look for ways of avoiding it.

What are some of the problems?

Appearing too certain

We all feel uncertain when we have to estimate something, so how can we appear too certain?

If I ask you to estimate how long something will take, what do you answer? If you give me one number, my immediate reaction will be “Wrong!” If I think about it, I will ask you “Are you certain? Which factors influence that estimate?”

A number is always the wrong answer to any estimation question. Are you so certain that this work will take exactly so long? What about all risks and the myriad variables that influence that work? How much do you know about the customer, the project, the team, the technology. Admit it, at the start of the project you don’t know a lot.

The “Cone of Uncertainty” show us how the uncertainty range of estimations changes as we learn more. It teaches us to always quote a range when we’re asked to estimate. The more uncertainty, the larger the difference between low and high estimate. That’s the good news. The bad news:

• These Cone of Uncertainty ranges are best case values. The two lines show how badly good estimators can estimate. The shaded area shows how the ranges converge for bad estimators.
• The X axis isn’t time. Your estimates don’t improve with time, they improve with information, knowledge and experience. The X axis should display your level of knowledge.
• The cone isn’t symmetrical. Most of us tend to underestimate the amount of work required.
• For some reason, we tend to use ranges that are too small to adequately cover the uncertainty, even when nobody pressures us to do so.

For estimation purposes, remember these two definitions:

• Accuracy: how close to the real value a number is. “I estimate this blog entry will take me between 2 and 4 hours to write.” Not bad: it actually took me a bit more than 3:30 (based on the timestamps of the edits).
• Precision: how exact a number is, independently of its meaning. “I estimate this blog entry will take me 4 hours, 45 minutes, 23 seconds and 860 milliseconds to write.” Wrong! And I don’t need to know about seconds and milliseconds!

Mistaking estimation and commitment

“We don’t need a range, we need one number, one date, one deadline to communicate with other parties!” Right, but that’s not an estimate, that’s a commitment.

Given that we estimate the product to be delivered between D1 and D2, which delivery date do we commit to when we talk to our customer? That’s a management and commercial decision, based on the confidence we have in the estimate, commercial pressure, deadlines, how much risk we want to take, the cost of failing to deliver as committed…

Developer: This project will take between X and Y days to complete.
Salesperson: I sold the project based on 2/3 of that number of days. You’re responsible if we don’t deliver in time.
Developer: I think that’s your responsibility. We’ll deliver the project as fast as we can, but we can’t guarantee it’ll take less than Y days.

Spending too much time estimating

Estimating takes time and we want to get that estimate just right, because we’re afraid of what might happen when we get it wrong. So we do some more investigation, another spike, try another estimation method… In the limit, we could just do the project, that would give us a perfect estimate.

Estimates are information to base management decisions on. What kind of decision? Depending on the type of decision, we need more or less accuracy, we need to spend more or less time. In any case, estimating shouldn’t take more than a few percent of the team’s time.

Manager: I want to know how long <some project> will take. Give me an estimate.
Developer 1: Yes boss! I’ll get right to it!
Developer 2: What kind of estimate do you need? What’s the required accuracy? What information do you have? How much time can we afford to spend on this?

Not looking at the big picture

Many tutorials on estimating give the advice to break down the work in small bits and then estimate those. The advice isn’t bad because, as we’ll see later, having an estimate based on estimates of smaller pieces improves our accuracy. But breaking down the work in too small pieces has a lot of drawbacks:

• All that breaking down takes a lot of work and then you have to manage all these small pieces. If you do this too soon, you’ll have the wrong pieces anyway and you’ll have to carry them along until you discover they’re wrong.
• You could break down the work below that which can be usefully validated with acceptance tests. I tend to avoid estimating tasks, because there’s often no clear and independent way to verify that the task has really been done. What’s the use of estimating something nobody’s waiting for? I prefer to estimate deliverables and user stories where a “customer” can meaningfully say “This is (not) done.”
• Nobody wants to know when a specific story will be done. We want to know when a Minimal Marketable Feature will be done, when the release will be ready, when the deliverable is ready to hand over to another team. If we break down the work too far, we’ll waste time tracking that detail and lose sight of the real goal. If we keep looking at the big picture, the law of large numbers will work to our advantage.

Destructive negotiations

Developer: We estimate the project will take between X and Y days to complete.
Manager: That’s too long! Your estimates are wrong! Redo the estimation and don’t come back until they’re right!

Those who estimate are often faced with people who are a lot better negotiators than they are: managers and salespeople. Moreover, the estimates are often based on quicksand and the estimators have a poor track record, so there are few facts that can be used in the negotiation. The only way to win this game is not to play: estimates are not negotiable. Project contents are negotiable, approaches are negotiable, resources are negotiable, commitments are negotiable. Estimates aren’t negotiable, but up for review and improvement when we get new information.

Developer: I understand that we need to make a commitment that’s earlier than the worst case estimate. Can we have a look at the project to see if there’s a way to change it so that we can deliver upon that commitment?

Playing with models

Some estimation techniques are based on models with several variables. For example, we could have some parameters that increase the estimate if we work with an inexperienced team, new technology or in a new domain.

On the one hand, the models and their variables allow us to tune the estimation methods and be aware of the many variables that influence our projects. On the other hand, these variables can be manipulated to come closer to an expected outcome. Because the number “came out of the model”, they gain undeserved authority.

Project Manager: The model predicts this project will take between X and Y days.
Manager: Can I have a look at your assumptions? Why do you rate your team as “Inexperienced”? Those guys we brought in last month are very experienced! Let’s change that parameter. And why do you say that this is new technology? Surely, web applications are almost the same as the mainframe applications we usually make? A consultant told me so. Let’s change that parameter. Ah! The estimate looks a lot better. Still  a bit on the high side, though.
Project Manager: … (Stunned silence. Thinks: I still have Y days to get out before the shit hits the fan. Those estimates are really useful.)

We also play with Mental Models, models of how others will act. For example:

Manager: I always halve the developers’ estimate, because I’ve noticed a tendency to pad their estimates and then I get blamed because we’re outbid by our competitors.
Developer: I’ll best increase the estimate, because our manager has this tendency to cut our estimates and then we get blamed if we don’t deliver in time.

Where do these mental models come from? Probably from some small dysfunction a long time ago which has grown into a serious vicious cycle. And we’d best not talk about these mental models, because that might cause all sorts of nasty conflicts!

Corporate amnesia

Once a late project is (finally!) done, the team is disbanded and everybody tries to forget this awful experience, the death marches to reach impossible deadlines, the daily re-estimations, the frayed tempers and the shortcuts that will haunt the maintenance programmers.

And then another project comes along that’s quite similar. So, we estimate by analogy: if our project is similar, the effort required should be quite similar. But nobody knows (or wants to admit they know) how much effort the project actually took. We use the original, documented estimate as our estimation basis. Result: another late project, by design.

Developer: Wow, this estimate with your new estimating technique is too high! We can’t present these estimates to our management!
Estimator: Why not?
Developer: They’ll never believe these estimates. Previous’ projects estimates are much lower.
Estimator: And did you deliver these projects as planned?
Developer: No. Never.
Estimator: And then what happened?
Developer: We just continued developing until it was done.
Estimator: I wonder why your estimates aren’t believed…

Is it all bad news?

With all of these dangers, it’s a wonder that any project actually delivers as planned.

If you have anything to do with estimations (either as a consumer or producer of estimates), read Software Estimation: Demystifying the Black Art. It not only contains the bad news, but also very practical advice on how to deal with it and become a more successful estimator.

There are ways to become better at estimating. But that’s the topic of a future entry.