|
|
Embarassing moment #239: Turning up late to teach an estimating and planning course.
 Danger: Estimator at work!
Estimation is (sometimes) useful, but it has its dangers. Estimation provides us with useful information, to base management decision on. This information and the way we get it can be misused. Maybe that’s why so many people are so afraid of estimating and look for ways of avoiding it.
What are some of the problems?
Appearing too certain
We all feel uncertain when we have to estimate something, so how can we appear too certain?
If I ask you to estimate how long something will take, what do you answer? If you give me one number, my immediate reaction will be “Wrong!” If I think about it, I will ask you “Are you certain? Which factors influence that estimate?”
A number is always the wrong answer to any estimation question. Are you so certain that this work will take exactly so long? What about all risks and the myriad variables that influence that work? How much do you know about the customer, the project, the team, the technology. Admit it, at the start of the project you don’t know a lot.
The “Cone of Uncertainty” show us how the uncertainty range of estimations changes as we learn more. It teaches us to always quote a range when we’re asked to estimate. The more uncertainty, the larger the difference between low and high estimate. That’s the good news. The bad news:
- These Cone of Uncertainty ranges are best case values. The two lines show how badly good estimators can estimate. The shaded area shows how the ranges converge for bad estimators.
- The X axis isn’t time. Your estimates don’t improve with time, they improve with information, knowledge and experience. The X axis should display your level of knowledge.
- The cone isn’t symmetrical. Most of us tend to underestimate the amount of work required.
- For some reason, we tend to use ranges that are too small to adequately cover the uncertainty, even when nobody pressures us to do so.
For estimation purposes, remember these two definitions:
- Accuracy: how close to the real value a number is. “I estimate this blog entry will take me between 2 and 4 hours to write.” Not bad: it actually took me a bit more than 3:30 (based on the timestamps of the edits).
- Precision: how exact a number is, independently of its meaning. “I estimate this blog entry will take me 4 hours, 45 minutes, 23 seconds and 860 milliseconds to write.” Wrong! And I don’t need to know about seconds and milliseconds!
Mistaking estimation and commitment
“We don’t need a range, we need one number, one date, one deadline to communicate with other parties!” Right, but that’s not an estimate, that’s a commitment.
Given that we estimate the product to be delivered between D1 and D2, which delivery date do we commit to when we talk to our customer? That’s a management and commercial decision, based on the confidence we have in the estimate, commercial pressure, deadlines, how much risk we want to take, the cost of failing to deliver as committed…
Developer: This project will take between X and Y days to complete.
Salesperson: I sold the project based on 2/3 of that number of days. You’re responsible if we don’t deliver in time.
Developer: I think that’s your responsibility. We’ll deliver the project as fast as we can, but we can’t guarantee it’ll take less than Y days.
Spending too much time estimating
Estimating takes time and we want to get that estimate just right, because we’re afraid of what might happen when we get it wrong. So we do some more investigation, another spike, try another estimation method… In the limit, we could just do the project, that would give us a perfect estimate.
Estimates are information to base management decisions on. What kind of decision? Depending on the type of decision, we need more or less accuracy, we need to spend more or less time. In any case, estimating shouldn’t take more than a few percent of the team’s time.
Manager: I want to know how long <some project> will take. Give me an estimate.
Developer 1: Yes boss! I’ll get right to it!
Developer 2: What kind of estimate do you need? What’s the required accuracy? What information do you have? How much time can we afford to spend on this?
Not looking at the big picture
Many tutorials on estimating give the advice to break down the work in small bits and then estimate those. The advice isn’t bad because, as we’ll see later, having an estimate based on estimates of smaller pieces improves our accuracy. But breaking down the work in too small pieces has a lot of drawbacks:
- All that breaking down takes a lot of work and then you have to manage all these small pieces. If you do this too soon, you’ll have the wrong pieces anyway and you’ll have to carry them along until you discover they’re wrong.
- You could break down the work below that which can be usefully validated with acceptance tests. I tend to avoid estimating tasks, because there’s often no clear and independent way to verify that the task has really been done. What’s the use of estimating something nobody’s waiting for? I prefer to estimate deliverables and user stories where a “customer” can meaningfully say “This is (not) done.”
- Nobody wants to know when a specific story will be done. We want to know when a Minimal Marketable Feature will be done, when the release will be ready, when the deliverable is ready to hand over to another team. If we break down the work too far, we’ll waste time tracking that detail and lose sight of the real goal. If we keep looking at the big picture, the law of large numbers will work to our advantage.
Destructive negotiations
Developer: We estimate the project will take between X and Y days to complete.
Manager: That’s too long! Your estimates are wrong! Redo the estimation and don’t come back until they’re right!
Those who estimate are often faced with people who are a lot better negotiators than they are: managers and salespeople. Moreover, the estimates are often based on quicksand and the estimators have a poor track record, so there are few facts that can be used in the negotiation. The only way to win this game is not to play: estimates are not negotiable. Project contents are negotiable, approaches are negotiable, resources are negotiable, commitments are negotiable. Estimates aren’t negotiable, but up for review and improvement when we get new information.
Developer: I understand that we need to make a commitment that’s earlier than the worst case estimate. Can we have a look at the project to see if there’s a way to change it so that we can deliver upon that commitment?
Playing with models
Some estimation techniques are based on models with several variables. For example, we could have some parameters that increase the estimate if we work with an inexperienced team, new technology or in a new domain.
On the one hand, the models and their variables allow us to tune the estimation methods and be aware of the many variables that influence our projects. On the other hand, these variables can be manipulated to come closer to an expected outcome. Because the number “came out of the model”, they gain undeserved authority.
Project Manager: The model predicts this project will take between X and Y days.
Manager: Can I have a look at your assumptions? Why do you rate your team as “Inexperienced”? Those guys we brought in last month are very experienced! Let’s change that parameter. And why do you say that this is new technology? Surely, web applications are almost the same as the mainframe applications we usually make? A consultant told me so. Let’s change that parameter. Ah! The estimate looks a lot better. Still a bit on the high side, though.
Project Manager: … (Stunned silence. Thinks: I still have Y days to get out before the shit hits the fan. Those estimates are really useful.)
We also play with Mental Models, models of how others will act. For example:
Manager: I always halve the developers’ estimate, because I’ve noticed a tendency to pad their estimates and then I get blamed because we’re outbid by our competitors.
Developer: I’ll best increase the estimate, because our manager has this tendency to cut our estimates and then we get blamed if we don’t deliver in time.
Where do these mental models come from? Probably from some small dysfunction a long time ago which has grown into a serious vicious cycle. And we’d best not talk about these mental models, because that might cause all sorts of nasty conflicts!
Corporate amnesia
Once a late project is (finally!) done, the team is disbanded and everybody tries to forget this awful experience, the death marches to reach impossible deadlines, the daily re-estimations, the frayed tempers and the shortcuts that will haunt the maintenance programmers.
And then another project comes along that’s quite similar. So, we estimate by analogy: if our project is similar, the effort required should be quite similar. But nobody knows (or wants to admit they know) how much effort the project actually took. We use the original, documented estimate as our estimation basis. Result: another late project, by design.
Developer: Wow, this estimate with your new estimating technique is too high! We can’t present these estimates to our management!
Estimator: Why not?
Developer: They’ll never believe these estimates. Previous’ projects estimates are much lower.
Estimator: And did you deliver these projects as planned?
Developer: No. Never.
Estimator: And then what happened?
Developer: We just continued developing until it was done.
Estimator: I wonder why your estimates aren’t believed…
 Is it all bad news?
With all of these dangers, it’s a wonder that any project actually delivers as planned.
If you have anything to do with estimations (either as a consumer or producer of estimates), read Software Estimation: Demystifying the Black Art. It not only contains the bad news, but also very practical advice on how to deal with it and become a more successful estimator.
There are ways to become better at estimating. But that’s the topic of a future entry.
What’s expected of me in this new Agile team?
When we work with teams transitioning to Agile, we have to address the transition from existing roles and the anxiety surrounding the change:
- What will be expected of me?
- What can I expect of others?
- Will my skills still be valuable?
- How can I contribute?
- I’ve heard that there are only developers and customers on Agile projects. Don’t they need testers, designers, analysts, architects, managers…?
Portia and I have written a number of Agile Team Roles and Responsibilities in the form of User Stories and Acceptance Criteria. We use these as a start to help the team define the roles they need to deliver value.
Just remember
- These are only a start to a conversation about roles within the team.
- Role does not equal person. A role can be filled by many people; a person can be in one or more roles. A team where many people can take on a role is more flexible, smarter, more resilient and more efficient.
- Roles change fluidly over time. In an agile team, people will take the responsibility for a role as the need arises.
How do you add value to your team?
Is estimating due to lack of trust?
Rob Bowley’s “Estimation Fallacy” session at SPA 2009 contained some discussion about trust. In particular, this pithy statement was made:
Estimates are due to the lack of trust between the business(*) and development team.
There have been a number of descriptions of “estimation-less” approaches. Is it simply a case of lacking trust?
It’s probably a bit more complex than that.
Valid reasons to estimate
Why would we estimate the effort and cost of a project, a release, an iteration, a feature?
To evaluate if the benefit of the project outweighs the cost. Say we have an estimate of the value a project or product may bring. I’m prepared to pay you 1.000.000€ if you deliver certain product. Do you do this project? Only if it will cost you less than that amount to deliver it. Of course, we should try to break the product down in smaller Minimal Marketable Features so that we can deliver sooner and get value earlier.
To make scope tradeoffs. Say we have a given budget and time: we need to show a demo at an industry event in 3 months with the current team. Which features shall we keep, which can we postpone? Which features will give us most “bang for the buck”? Of course, we start by prioritising by value, but three lower valued/lower cost features may have more effect than one higher value expensive feature.
To know how to distribute resources over multiple projects. Say we’ve got 20 projects we could start and we have 50 developers. Which project(s) do we start? How do we know where the developers will have the most effect? Of course, we start the fewest possible projects simultaneously, but there’s a limit to the number of people that can work on one project.
To make reliable promises to others. “Can we deliver this game before Christmas? If so, we need to get the marketing campaign started.” Of course, it would be easier if we were all in the same multi-functional team or if we had a real cross-company value stream. But sometimes we do have to coordinate with others or agree on contracts.
To know when to exercise our Real Options. Real Options tells us we need to make a decision at the optimal decision time. This is the deadline minus the implementation time of the most costly option. How do we know the implementation time?
No estimation? Really?
I agree that spending inordinate amounts of time on estimating is a waste of time. But no estimation? If I look at the “estimation-less” methods I see a few different approaches:
- Break down the work in approximately even-sized amounts of work. In effect, you can say that each item now has a cost of ‘1’. Then you can simply count the number of items and use throughput and cycle time to make your estimations. How do you know that the items are approximately even-sized?
- Count the number of user stories, even if you know they have different sizes. In the end, it doesn’t matter much, some stories will go faster, some stories will go slower. Over a release many of those differences cancel each other out. What do you call the number of stories you typically deliver per release?
- Give each item a high level estimate (in points, gummi bears or t-shirt size) and track (or estimate for the first few releases) how many of those you can reliably deliver. Concentrate on those items were estimates matter.
The only really estimation-less method I see is one where features are prioritised purely based on value and then pulled for implementation one by one, without caring how long it would take to implement. I think this is what’s described by Joshua Kerievsky. Even then, he talks about micro-releases, so the features must have been broken down into reasonably small pieces.
That’s about how I work on projects where I’m both developer and customer: user stories are prioritised by value and released in micro releases. The features are small enough to be implemented in a couple of hours. In any case, I set a timebox and revert if implementation takes too long. I know the value; I know how much I want to spend; most of the time I correctly estimate that the cost of the feature is within my budget.
Trust: necessary but not sufficient
Estimations are useful but fraught with many problems. Trust between everyone involved helps a lot, but it doesn’t eliminate the need for estimates in most circumstances. Let’s look at some of the issues related to estimating and see what the root causes are. Maybe we can do something about the root causes.
And why don’t we talk about estimating (business) value?
(*) I really, really hate the use of the expression “ The business“. IT, the development team is part of the business. If we want more trust, can’t we start by seeing them as one team?
Where do we intervene and what do we do?
When we first start to improve processes, the situation is daunting: we can see so much that can be improved. Where do we start? What will have the biggest effect? And when we’ve done that, what do we do next?
The “Theory of Constraints” gives us a simple and powerful framework to guide our process improvement: “The Five Focusing Steps”.
Step 0: What is the goal?
The first and most difficult step is to determine (and agree on) the goal of the “System”.
Before we can do that we must determine what the System is. As a rule of thumb the System equals the “Sphere of Influence” of our Client: everything that our Client has the authority change.
To determine the goal of the System, we must ask ourselves “Who uses the results the output of the System and what do they value?”. We try to find metrics that measure the amount of valuable output we produce. The Theory of Constraints calls this the “Throughput” of the System.
On the “B” project, our goal was to release valuable features that were actively used by our users. Our measurement was the “Business Value” estimate that our onsite customer put on each User Story.
What is your system? What is its goal?
Step 1: Where is the bottleneck?
The fundamental insight of the Theory of Constraints is this: “the output of any system is determined by one bottleneck.” A chain is as strong as its weakest link. If we want to make the chain stronger, we need to work on the weakest link.
At the start of an improvement process, the bottleneck is easy to spot. A bottleneck resource is:
- Always busy. But busyness isn’t the only (or even a good) criterion by which to recognise bottlenecks. Many systems are mistakenly optimised to get high utilisation (or rather busyness) out of all resources.
- Work piles up in front of them.
- Downstream resources are regularly idle.
The development team was the bottleneck. We had a list of features that was piling up in front of us. It would take us several releases to implement our backlog. Users were getting impatient for features to be implemented and deployed.
Where is your bottleneck?
Step 2: Exploit the bottleneck
If the output of the system is constrained by the output of the bottleneck, we must first try to increase the output of the bottleneck. Any idle time of the bottleneck reduces output of the system. What can we do?
- Remove any non-value adding work.
- Remove or limit interruptions. Remove impediments.
- Let the bottleneck resource work at a steady pace.
- Provide high quality tools and materials.
- Carefully prioritise the bottleneck’s work so that they always work on the most important tasks.
- Ensure that there’s always enough work to do for the team (the backlog), so that they don’t become idle through lack of input.
- As team lead, I received and prioritised all incoming requests and questions. As I could deal with most of them, the team wasn’t interrupted.
- Production issues needed to be handled quickly. All issues were handled by me and one developer. The bugfixer role rotated after every bug. This provided a nice balance between keeping the team focused on developing the current iteration and ‘feeling the pain’ of production issues.
- I made sure that everyone worked at a sustainable pace. No more overtime!
Warning: don’t shield the team from useful information like input from the customer, production issues, installations and feedback from users.
We start by fully exploiting the bottleneck because this type of improvement is relatively easy:
- It requires no extra cost or investment.
- Only one resource is involved.
How can you exploit your bottleneck?
Step 3: Subordinate every other decision to the bottleneck
When we’ve fully exploited the bottleneck, we must subordinate every other decision to our decision to exploit the bottleneck. All the resources that aren’t bottlenecks have, by definition, some slack. Use that slack to support the bottleneck. We can subordinate by:
- Letting the non-bottlenecks help the bottleneck or take over some required but low value adding work.
- Everybody works at the pace of the bottleneck, no faster no slower, to avoid overloading the bottleneck with work in progress.
- Those in front of the bottleneck ensure that the buffer of work for the bottleneck is always filled, but not too much.
- Those after the bottleneck ensure that they have some slack to deal with variations in output of the bottleneck.
- Non-bottlenecks ensure that only high quality work in progress handed to the bottleneck.
- As team lead, I subordinated all my work to the team: whenever there was a request for the team, a production issue or an impediment, I dropped all my work and supported the team. I quickly learned that I shouldn’t commit to delivering stories. My contribution to the team was less in contributing to its throughput and more in protecting the throughput of the rest of the team.
- The onsite customer performed acceptance testing on completed stories every week, so that the developers got rapid feedback on the quality and fit of their development.
- The onsite customer and I prepared stories for the next iteration and release, so that the team always had something to work on.
Subordinating is still relatively easy:
- It requires no extra cost or investment.
- Only a few resources, typically those that interact directly with the bottleneck, are involved.
However, there is one aspect of subordinating that won’t be accepted easily: whereas the bottleneck resources should be fully loaded, non-bottleneck resources must have slack time to be able to support the bottleneck and deal with variations. Most managers are evaluated on the efficiency and not the effectiveness of their people. Deliberately making people work below their capacity goes against their goals. We can solve this problem by fully loading the non-bottlenecks but ensuring that a part of their work is ‘discardable’, “nice if it gets done, but not essential or time-critical”. Whenever the non-bottlenecks need to support the bottleneck, they can drop the non-essential work to free up time.
Which decisions do you need to subordinate to the bottleneck?
Step 4: Elevate the bottleneck
This is the step most people will intuitively apply first: add more people, more machines, more training, more tools, more of everything. We only take this step when all the ‘free’ improvements have been performed. We can elevate by:
- Adding more people or machines
- Training and mentoring
- Better tools, faster machines
- Switching to a different technology
My manager asked me about every week if I wanted some more developers, if I “wanted to elevate my team by adding more people”. I declined, because we had plenty of room for improvement left with exploit and subordinate improvements. I asked if we could get faster computers instead. He told me we couldn’t because the hardware budget was exhausted. But there was budget for extra developers if I needed any…
Elevation improvements are more difficult because they require an investment. Elevation improvements are dangerous because most of these improvements take some time to produce results. Results might even worsen until the improvements start to have a positive effect. For example, when we add people to a team we lower the team’s velocity and accept less work while the team members help the newcomer get up to speed.
Don’t elevate yet. I know you can apply so many more exploit and subordinate improvements. 🙂
Step 5: And again!
When we’ve applied one improvement and have seen a positive effect, we go back to the beginning:
- Is our goal still valid? Is our measurement of throughput still correct?
- Where’s the bottleneck? After some improvements we may have solved our worst problem. As there’s always a bottleneck, our second-worst problem gets a promotion. We now need to focus our attention on the new bottleneck.
And the result was…
The team got better and better by repeatedly going through the five focusing steps. After a while, they started to develop stories faster than customers could write them. That’s when we needed to apply focusing steps six and seven.
I want to know more
Read the Goldratt books about the Theory of Constraints.
 
Download and play the “I’m not a Bottleneck! I’m a free man!” simulation from http://www.agilecoach.net
Read more about the Theory of Constraints on this blog.
Software projects are difficult because…
I’ve heard this said several times over the years:
Software projects are difficult because software is so easy to change.
Other engineering and building projects are seemingly easier because they deal with “hard to change” stuff like bricks, mortar and steel.
I heard this statement again at Rob Bowley‘s SPA 2009 “Dealing With the Estimation Fallacy” goldfish bowl discussion about estimation. Rob has written up the outputs of the session, but sadly there’s nothing more about this fascinating statement.
I find it difficult to accept this statement. Surely, the ease of change must be an advantage? It means we can exercise our Real Options later, that we can change our mind when we know more? It means that we can correct mistakes and wrong assumptions more cheaply? It means we can keep our creations relevant in a changing world?
So, to make software projects easier we just need to…
If the previous assertion is true, then making software projects easier is easy.
Make software hard to change so that software projects become (a little bit) easier.
Forget “embrace change”! From now on, I’ll make software hard to change. Not that I have to do much to achieve that, I can create hard to change software without thinking. 🙂
Hm. I still don’t understand why easy to change software makes projects hard. Do you know why?
Some effective ways to make software projects easy
The goldfish bowl came up with some more ideas about change.
1. Use only reliable, thoroughly tested technology that serves your people and process
One remark was that we make things difficult for ourselves through technology churn. Technology and tools change rapidly. Unfortunately, most of these changes are no improvements. But we don’t have to stay on that treadmill. Some time ago a new programmer in the team complained: “Why don’t we use technology X? It’s new!”. My response: “That’s exactly why we don’t use it. Yet.”
When I’m in charge of a project, we will only use a new tool or technology when it’s thoroughly tested and when everyone in the team understands it. We’ll first try the new tool on prototypes or small internal projects so that we can discover the hidden pitfalls. And even then I need to have a backup solution on hand if the new thing doesn’t work. As Weinberg says: “Anything new never works”. As the Toyota Way says: “Only use reliable, thoroughly tested tools that support people and process”.
2. Don’t reuse. Use.
Another remark was that we write too much custom code from scratch. We could reuse more or implement standard packages. My experience with this is mixed.
Standard packages can work IF they deal with a well-known, stable domain where you don’t want to differentiate yourself from your competitors. For example, you probably don’t want a custom accounting package, unless you’re Enron. But is a standard package really the best way to run your manufacturing plant? Maybe when you don’t want continuous improvement. If the slogan is “It’s easier to change your company to match the software than to change the software to match your company”, then you’ve got some really hard to change software. Those projects don’t look easy, though.
I’m no fan of frameworks. The penalty for “colouring outside of the lines” is too steep. Most packages and frameworks are “80% solutions”: great for the first 80% of your project, hell for the last 20%. For example, I like Ruby on Rails, but it takes me so much effort to do something differently than what the Rails developers think is the right way. As a result, I’ve seen mostly productivity decreases when using frameworks, even the ones I wrote 😉
What does work:
- Writing software for use, not for reuse. When I write TDD code, refactor and remove duplication I end up with useable code. A surprisingly large amount of it ends up to be useable in other projects. Three or more instances of this happening and it becomes worth spending time extracting the common code and maintaining it in a library.
- Using libraries, especially small, focused libraries with technical components. It’s a pleasure to be able to download a Ruby Gem, have a quick look at the source, write some unit tests against it, use it. For example, it took me about an hour to integrate the Akismet anti-spam service into my wiki. Since then it has stopped lots of spamming attempts on the many wikis I maintain. Great value for money!
These are just two small measures. They don’t make software projects easy. But they do avoid unnecessary difficulty.
|
|
