“Estimating is often helpful. Estimates are often not,” I said in a Tweet.
Several people asked, “How can this be?” Let me say more about estimation, in more than 140 characters.
Estimating is often helpful.
Estimating helps when the process itself builds shared understanding among the people who want the work done and the people doing the work. This is especially helpful when looking to strengthen or build a team.
Collaborative estimating gives the best results. Diverse experience yields a broader range of perspectives and questions. Questions and perspectives build understanding of the what, why, and who related to the request. That’s helpful.
Group discussions can reveal differences in knowledge and understanding. Finding those gaps early is helpful, and team communication improves as a result.
Group estimating surfaces assumptions. When we are aware of our assumptions, we can verify–or debunk– them.
When the group knows enough about the “what” to think about the “how,” they can analyze implementation. Working out implementation details reveals more assumptions and generates more questions.
Sometimes estimating reveals you only know enough to reason by analogy. The best you can do is posit that the desired functionality is about the same size as X.
But sometimes, agile estimation helps teams realize that they don’t know enough to think about size or effort in any meaningful way.
This situation calls for discipline. Discipline to resist guessing and speculation. Estimates born of ungrounded guesses are worse than useless. Rather than guess, experiment, interview, model, sketch designs or do some activity to gain more clarity about needs and context. Then try again.
Estimates are often not helpful.
Estimation is not without its downsides. People turn informed forecasts into targets. Meeting the target becomes the de facto goal and the de facto method. Non-value added but necessary work gets pushed aside. Meeting needs fades in priority.
People construe estimates as promises. No one can predict the future, but many people treat them as guarantees. Failed predictions fan blame. Trust and openness suffer.
People construe estimates as bids. Bidding usually involves some calculation of profit. That implies a margin. However, managers discourage margins. Some view it as “padding.” And view padding as moral failing. It’s really a contingency for the unknown (or compensation for bosses who are known to cut estimates to fit wishes. See below).
Inappropriate precision implies that people know more than they do. When expectations and reality meet, people may feel disappointed. More likely, they feel deceived. Trust and openness suffer.
People game estimates. I’ve had the experience of thinking long and hard to provide some information about how long something might take, only to have a managers say, “That is too high. The estimate should be X.” Fudging sets off another round of deceit and disappointed expectations. Trust and openness suffer.
So please, estimate. But don’t get caught up in estimates. And grab a copy of George Dinwiddie‘s excellent new book, Software Estimation without Guessing.
Yes, I’ve seen teams and their managers use estimates as targets – not estimating as a team tool – and it ruins team spirit and starts blame gaming.
But the team should also be able to give an estimate to external people outside the team, like the backlog owner and stakeholders. If we’re talkning Scrum here there should be a promise somewhere around sprint planning 2. But I’ve seen the estimating stop there, and I believe that has to continue throughout the whole sprint.
Nice article.
Peter: How can one make a promise to anyone based on an estimate, which is essentially an educated guess?
Around sprint planning 2 the team takes on as many tasks as there is available hours for one sprint, and makes a commitment to that. I called that “a promise”, maybe that was wrong, but still, the team hold themselves accountable for the commitment. Based on estimates, right?
Peter, tell me how you are thinking about tasks and available hours.
It’s all about predictability; I make a few assumptions that there is
* Limited amount of time for the team for one sprint, ie 80 hrs in total for the team.
* Tasks are the details that team thinks has to be performed to meet sprint goals, ie 110 hrs
Then the team takes on tasks for an amount of 80 hrs, leaving tasks for an amount of 30 hrs. This way there’s a guess of what the nearest future will bring. There are some people who is interested in that kind of predictability, at least are my experiences pointing in that direction =)
Hmmm. So the team is pulling /tasks/ not stories (or some other expression of customer value)? How does that work with demonstrating valuable working software at the end of the sprint?
e
Oh, but the tasks belongs to a story which represents the feature aka business value. We use to give story points to the stories and hours to the tasks.
So do you slice the story when it can’t be completed in one iteration? Drive towards done, rather than having a story slop over into another iteration.
Yes, if scope cannot be reduced the backlog owner can decide that the story should continue in the next sprint, and hopefully the first part is something that can be deployed.
In some circumstances /estimates/ are valuable too, not only the process that produces them (/estimating/).
a) if estimates are not used, why do I have to estimate? Isn’t seen as a time-waste activity?
b) improving on my estimates (personal or in a team) helps me to better understand my capabilities on doing things. Next time I’ll be more accurate to predict time/cost/money on what I’m doing
c) stick to an estimates is dangerous, of course. But we don’t have to forget what it really is: a calculated guess.
Estimates are a tool: if we use themm, and do it “correctly”, we’ll see benefits. Using them in wrong ways can lead to a (project) disaster, like any other tools.
(a) and (b) are two motivation to continues to produce estimates. (c) is one – and only one – example of bad usage of estimates.
What do you suppose is the proportion of organizations using estimates /correctly/ vs. those who are not? (Using your definition of /correctly/)
e
Of course I can only suppose that *few* organization use estimates “correctly” (AFAIK). After all human brains are bugged with cognitive biases (eg: Anchoring, Bandwagon effect, Confirmation, etc) that treat estimates for what they really are require *discipline*. But I insist: estimates are only a tool. And like all tools it requires proper usage.
Or, we could think about what we are trying to accomplish, and see if there are other tools that would serve that purpose with fewer detrimental side-effects.
Yes!
Define the objective and move to accomplish it using the best tools and processes available to you.
BTW… I would expect most organization hack away at estimating in a similar fashion to how programmers hack away at coding instead of using proper and professional programming techniques.
But sometimes, estimators realize that they don’t know enough to think about size or effort in any meaningful way.
This situation calls for discipline. Discipline to resist guessing and speculation.
Thanks for this! A very timely reinforcement for my exhausting interviewing, modeling, and sketching. Clearly time for experimentation!
Hold on Claire.
Estimating is all about guessing and speculating. What you want to have is the discipline to know that estimates aren’t a record of history. They are a guess about what will happen in the future.
Also, you can experiment while estimating if need be.
errr. Claire is quoting me.
Estimating is an analysis activity. You have to know something about what you are analyzing. When there are many unknowns, it may be more fruitful to gather information or experiment rather than piling assumptions on top of assumptions.
e
Here’s the problem. Let’s say that you want to have some work done on your house. You call three contractors that specialize in the type of work you need. You get three /estimates/. What is your expectation?
My expectation would be that I will pay the amount /estimated/ … exactly … unless … there is a major complication that the contractor can show and justify.
While software development is very different from home remodeling, the mindset is largely the same. The business wants an /estimate/ and it expects that /estimate/ to be held firm. Realistic? Perhaps not, but we are where we are.
We need to provide /estimates/ and we need to do a better job of explaining how we derived them, what assumptions were made, what risks exist, and what work efforts are not included. We need to do better at educating business people about software /estimates/.
Vin,
You’ve described a bid. A bid is a legally binding price. There may be language in the contract that allows for extenuating circumstances and change orders. Sometimes bids are written +/- 5 or 10 per cent to account for unknowns.
An estimate is a guess. Holding people to a guess is not reasonable.
e
I think if people wanted to know how much something cost, they would look at an estimate range. Or they could look at an estimate and compare it to a past project that shared similar characteristics–considering the early estimate for that project, and the actual cost.
I worked in an organization that expected project managers to deliver within +/- 5 per cent of budgets, which were derived from early estimates. This company had years of data–which they apparently didn’t consult. If they had, they would have seen that only 1 or 2 projects had met that criteria, and those projects were very small.
I suspect that sometimes, people don’t want to know how much something will cost. They want something that sounds objective (as numbers do) that allows them to play the political game of getting what they want to get done funded.
To paraphrase on senior executive, “They never would have funded this project if they knew the cost. But we knew we could go back to the well.” People don’t handle sunk costs very well, either.
Plans are nothing; planning is everything.
Dwight D. Eisenhower
Alan,
I have to say that Dwight wasn’t a software developer.
Plans are part of the record of the planning process. Planning process records are used as inputs into planning process improvement efforts. A software development organization needs to improve process to improve output.
Non-rhetorical question:
In the absence of estimates, how does one plan a release? How does one decide whether it will be worthwhile to undertake a project in the first place?
Look at relative sizing (generally a more appropriate level of precision), business value, capacity.
Look at what we are trying to accomplish, and see if there are other tools that will accomplish that purpose with fewer detrimental side-effects.
My assumption is that organizations want to prioritize spending decisions, and assess whether a project will be a good investment. (I may be dreaming.)
For assessing priority, you could use business value ranking, apply criteria, or use some form of elimination.
Use incremental funding. That gives them a chance to choose again with empirical data rather than guesses.
What other ways can you think of to meet the goal of prioritizing spending?
Aha. When I read “estimates” (in an Agile context anyway) I think of relative sizing (e.g., fibonacci story points) with projections based on empirical data. Now it appears that you meant it differently from how I read it :-/.
Another great post. This one on the psychology of estimates. It also has my mind racing with thoughts. I’ll have a whole series of posts on my blog about this.
It’s too bad many people don’t understand that estimating is great and estimates are awesome. I guess they forget what the word estimate means.
Cost estimates, time estimates, quality estimates… Those could all be live gauges on a software development dashboard for company leadership.
Free cost estimates are what everyone wants and they somehow want them right away. Everyone wants free stuff until you start offering “free fast surgery” or “free house painting, done in one hour”.
Esther, you don’t answer your phone but you sure get people thinking.
As you point out we ask the thing we call “estimates” to fill several roles. These roles are frequently at odds: an accurate estimate of when work will be done, a deadline, a bidding tool, a bargaining chip.
One of the reason I like XP style velocity measurement is that by benchmarking against yourself the whole issue of over or under estimating goes away, its self adjusting.
I have clients who whom the combination of point based estimates, velocity and burn-down charts results in very accurate forecasts for planning dates.
I have other clients were the situation is less clear cut. However, I believe even if estimates give nothing to the scheduling process they serve two purposes: a) as you state the process reveals different interpretations, gaps in knowledge, etc.; b) it satisfies the need many feel to engage in a planning/scheduling activity.
I sometimes think that estimating is simply a placebo, even if it is I think it has a role, albiet a transient role.
BTW You might like to read “The Planning Fallacy” from Kahneman and Tversky, I blogged about this (and other studies) last year
http://allankelly.blogspot.com/2011/03/humans-can-estimate-tasks.html
Estimates drive bids, don’t they?
And, bids drive contracts, don’t they?
So, if you can’t meet the terms of the contract (time, schedules, dollars, features, quality, etc.), you can drag your company into a breach of contract condition.
And, then you get to update your resume.
Kind of makes me want to treat estimates as slightly more important than mere guesses.
Sure, from a project point of view, it’s the process of estimating that’s important.
But, from the business point of view (which ultimately what it’s all about), it’s the estimate itself.
Yes, estimates drive bids.
Yes, bids drive contracts.
The thing is, most people who are making bids are clear that they are making a bid. People who are writing a contract are clear that they are writing a contract.
And within many organizations, there is neither a bidding process nor a contracting process for software projects.
In those organizations, estimates drive prioritization and planning, not bids and contracts.
These problems with estimates can be mostly avoided by using an estimated range rather than a point estimate.
So (for example) choose 2 times such that the project is 10% likely to finish under the minimum time, but 90% likely to finish before the maximum time.
That is assuming all understand that the uncertainty is inherent to the problem and can only be reduced by getting real work done on the problem. The worst case scenario is giving a ranged estimate and management hearing only the lower number.
Yes, it would be very nice if people used range estimates and probabilities.
However, what I usually observe is this: when presented with range estimates, people retain the low estimate and forget the high estimate. And they hear any non-zero probability as “possible,” and make the bet. I am sure someone has done research on the cognitive factors that contribute to this widespread behavior. It’s too common for it to be a fluke.
e
Couldn’t disagree with you more.
One of the core things about agile and scrum is measurement. Measurement for velocity and measurement for improvement. Over the long run business has every expectation to reply on your estimate of stories thus sprint velocity. And business has every right to make intelligent business decisions based on your estimates and team velocity. They have every right to hold you to it.
Estimate isn’t a tool for engineers guys, the whole agile thing starts with the premise that it is business driven. A process or a tool has very little meaning unless it helps with a business goal.
Well, you can disagree with me less, then! 🙂
I agree that velocity can be a very useful tool for planning, predicting, and building trust. I agree that business have every reason to make intelligent business decisions. Best when they can do that on empirical data, which is what agile and scrum allows them to do.
I’ve been guilty of “negotiating” with devs over estimates in order to get more work in a sprint/queue. But no more. Now I understand agile methodology more, the business is told that they should prioritise tasks but there are no promises how much will get done in a specific timescale. And the business understand now. There is no such thing as a guaranteed delivery time in web dev; get over it.
However, from a standing start it is difficult to get people to understand that estimate does NOT mean the same as guarantee. And to get over estmates been considered as such, we removed all numerical representations & went to small, medium & large tasks.
Also, estimates are much more valuable if you’re doing stuff you’ve done before; if you’re working on things with no experience, you need to understand that an estimate is a lot less accurate.
It is also about using a process that fitoolkit circumstances too; what’s best for your business isn’t necessarily going to work for another. That’s the value of a good PM.
Nicely put. I especially liked “People construe estimates as bids.” This alone is likely to take the whole effort away from meaningful, collaborative dialog into competitive antagonism.
Once again, nicely done.
I humbly submit a corollary:
Estimates aren’t useful during your iteration, but they are afterwards.
/Reviewing/ those estimates in your retrospective is an excellent way to find opportunities for improvement. Presumably, one type of process improvement task from a retrospective is “how could we have estimated better?”. Better estimating will yield a narrower ‘cone of uncertainty’, and more reliable iterations. Perhaps most importantly, the team will develop higher self-confidence, which is essential when dealing with difficulties, both internal and external.
Although I’m always skeptical about chances to completely overcome well known Planning Fallacy problem, I second your point about the usefulness of a feedback driven process. Good corollary!
Agree – the conversation that occurs when trying to come up with the estimate is more valuable than the estimate itself.
However, I still find estimates useful. The agile techniques of Story points/T-Shirt sizing and velocity helps with long term planning. And if collect cycle time data, you can come up with some very useful data to allow the business to plan and get some predictability.
I agree with you that collaborative estimation is useful process and is an effective and efficient way for the team to surface hidden assumptions concerning the implementation. However, the estimates are just as useful, but only when there is trust and openness. The team should strive to improve the accuracy of their estimates. At the end of a cycle, the team reviews their estimates and compares them to what actually happened. The discussion that ensues can reveal ways to improve future estimates. It can also lead to better implementation processes for upcoming tasks, which makes the team more efficient. This works best in an agile process like scrum, where the feedback cycle is a few weeks.
Good article!