rob bowley

I ran my session, “Dealing with the Estimation Fallacy” at SPA2009.The turnout was better than I’d expected and we had some great discussions. Interestingly there is a pattern emerging from each time I’ve run this which seems to revolve around trust.

Here is a zip file containing the slides, some presentation notes and photos of the flip chart sheets:

dealingwiththeestimationfallacyresources.zip

And here is the output transcribed (you can view output from the other times I’ve ran it here and here):

Output

Not when but what useful decision we can make given what we know.
Not us to say when the project will be ready.
What if the customer has to decide not when it’s going to be ready but what has to be done?

• What about fixed deadlines?

Do estimates based on % uncertainty over the estimate.
Don’t estimate anything over 3 days work.
Estimates are due to the lack of trust between the business and development team.

• Build up trust through delivering

But then how do the customer to decide whether to go ahead or not?
The process of estimating has the effect of reducing the amount of time it will take.

• Not according to De Marco in Peopleware – quotes a study which found a team not doing estimates was more productive than a team that did.

Why do we think we’re different from any other industry?
Estimates confused with commitments.
We can probably rely estimate using the Gaussian model 90% of the time.

• Lots of estimates work out but sometimes Black Swans stuff it up

“Planning is essential. Plans are useless”
People give numbers too much credibility due to Cognitive Bias (Confirmation Bias).
Be clear it’s an estimate not a commitment.

• Often not satisfactory

Keep your “Cone” short.
Focus on managing the risk.
John Daniels always gives the customer 2 numbers, one high, one low and accepts the risk.
Toyota model -> share the risk in the agreement/contract.
Fixed price buys the commitment & if you deliver early offer to do more -> builds up trust.
Estimation is often used as a political tool, but also entirely reasonable for people to want to know when it will be done.
Use throughput and cycle time instead

• only works on a fixed team otherwise too many variables

“Realities” of the world -> “we need this done in x months, can you do it?”

• Karl Scotland -> ask what they need instead
  • needs trust and this has to be built. how do you get started?
  • Karl: do enough estimation to show it is worthless and then show how wasteful it was = TRUST!
  • Maybe this worked but it’s possible confirmation bias is in play here

Any examples fro other industries who’ve got it right?

• Movie industry is unpredictable which they seem to accept and are better at managing cost

“We’re all Channel Tunnel rather than line-built semis”
We’ve done a poor job of educating our customers that we don’t build line-built semis.

• Such an immature industry
  • we need to find tools to make us more predictable, but tools change too quickly
  • bricks have not changed much in 100 years
  • we’re always changing everything behind the scenes
  • Need to share problem solving with our Customers -> Agile

Why are we so bad at managing our customer’s expectations?

• Customers can’t relate to what we’re building for them unlike a house or skyscraper

Most successful with a shared risk model.
We have a responsibility to say NO!

• often too happy to accept unrealistic expectations

Customer must accept they need to know better how IT works

• We don’t do magic

There are rules in the building industry, maybe there should be rules in software e.g. a minimum level of quality.
We’re (in the session) focusing to much on contractual relationships.
We need estimation for our own internal team planning – we need to match that up somehow.
To estimate we need design and analysis. We fall done when we don’t do this.

• Breaking down into user stories is enough estimation

We’re very focused on delivering custom pieces. If we offered less customisation we’d be more predictable.

• Is it in too many people’s interest to make something custom?
• Not sure using SAP is any cheaper!