After a successful run of Retrospective Surgery at SPA2009 we got some really good output so thank you to all who attended. As a result I’ve updated the Agile Retrospective Resource Wiki with new tools, ailments and cures and a cool new plan I bumped into the other day. It’s beginning to come together really nicely

Here is a zip file containing the slides, some presentation notes and photos of the flip chart sheets:

dealingwiththeestimationfallacyresources.zip

And here is the output transcribed (you can view output from the other times I’ve ran it here and here):

Output

Not when but what useful decision we can make given what we know.
Not us to say when the project will be ready.
What if the customer has to decide not when it’s going to be ready but what has to be done?

• What about fixed deadlines?

Do estimates based on % uncertainty over the estimate.
Don’t estimate anything over 3 days work.
Estimates are due to the lack of trust between the business and development team.

• Build up trust through delivering

But then how do the customer to decide whether to go ahead or not?
The process of estimating has the effect of reducing the amount of time it will take.

• Not according to De Marco in Peopleware – quotes a study which found a team not doing estimates was more productive than a team that did.

Why do we think we’re different from any other industry?
Estimates confused with commitments.
We can probably rely estimate using the Gaussian model 90% of the time.

• Lots of estimates work out but sometimes Black Swans stuff it up

“Planning is essential. Plans are useless”
People give numbers too much credibility due to Cognitive Bias (Confirmation Bias).
Be clear it’s an estimate not a commitment.

• Often not satisfactory

Keep your “Cone” short.
Focus on managing the risk.
John Daniels always gives the customer 2 numbers, one high, one low and accepts the risk.
Toyota model -> share the risk in the agreement/contract.
Fixed price buys the commitment & if you deliver early offer to do more -> builds up trust.
Estimation is often used as a political tool, but also entirely reasonable for people to want to know when it will be done.
Use throughput and cycle time instead

• only works on a fixed team otherwise too many variables

“Realities” of the world -> “we need this done in x months, can you do it?”

• Karl Scotland -> ask what they need instead
  • needs trust and this has to be built. how do you get started?
  • Karl: do enough estimation to show it is worthless and then show how wasteful it was = TRUST!
  • Maybe this worked but it’s possible confirmation bias is in play here

Any examples fro other industries who’ve got it right?

• Movie industry is unpredictable which they seem to accept and are better at managing cost

“We’re all Channel Tunnel rather than line-built semis”
We’ve done a poor job of educating our customers that we don’t build line-built semis.

• Such an immature industry
  • we need to find tools to make us more predictable, but tools change too quickly
  • bricks have not changed much in 100 years
  • we’re always changing everything behind the scenes
  • Need to share problem solving with our Customers -> Agile

Why are we so bad at managing our customer’s expectations?

• Customers can’t relate to what we’re building for them unlike a house or skyscraper

Most successful with a shared risk model.
We have a responsibility to say NO!

• often too happy to accept unrealistic expectations

Customer must accept they need to know better how IT works

• We don’t do magic

There are rules in the building industry, maybe there should be rules in software e.g. a minimum level of quality.
We’re (in the session) focusing to much on contractual relationships.
We need estimation for our own internal team planning – we need to match that up somehow.
To estimate we need design and analysis. We fall done when we don’t do this.

• Breaking down into user stories is enough estimation

We’re very focused on delivering custom pieces. If we offered less customisation we’d be more predictable.

• Is it in too many people’s interest to make something custom?
• Not sure using SAP is any cheaper!

It followed the same format as the previous time I ran it so you can go there to find the slides etc.

Here’s the output from the discussion. I’m running this session again at SPA 2009 and after that I will collate the output from all three into something coherent and useful:

How do you keep the context stable?
The problem is managing the context. If the business says you need to do it sooner that changes the context.
What about if the business doesn’t have the full context? What about issues you don’t know about?
Why are we different to any other type of organisation?
It’s too difficult to estimate more than 2 months ahead
Familiar work is easy to estimate, the unknown is not (which is mainly what we do)
If we report back to the business regularly they can’t deny hard facts
Manage the risk by working iteratively
It’s quite common for people to start building skyscrapers before the architects finish designing the top

• but their customer’s not going to ask for another 20 floors and more space halfway through

We need to help the customer understand what they need
People need to know whether it’s a viable business case
Projects are never agile if you have requirements up front (fine if nothing changes though)
You should only commit to work three months ahead
“2 years” for a software project is too long – will not be competitive
Movies spend a long time in development, then they get the budget and the scope gets cut to fit. But, they are more predictable than software.
Too much money (e.g. banks) will never produce good software – scope to big.
Are big estimates/commitments ever believed by anyone? Often they’re only used to make projects viable
IT departments are not pushing back enough – should never suggest you can plan more than 3 months in advance
2 types of estimation:

1. The real business of estimation (e.g. stories, iteration scope, tasks ect)
2. The game of estimating within the business context

Agile projects are rarely ever really agile as they’re operating withing the context of the business/traditional practices.

Not only that but I will also be presenting at XPDay in December! The session is called “Using lean to evolve out of Scrum” which was the brainchild of Matt Wynne and based around our experiences working together on a project.

Gobsmacked I am.

Firstly I asked the audience why we estimate?

• Predictability
• To be able to choose between options
• Planning
• So we can manage budgets
• To justify development
• Manage expectations
• Learn more about the problem
• Synchronise with other projects
• Get a cost benefit ratio
• Prioritisation
• Customers like to know when it’ll be ready

I then did a short presentation (slides below) explaining why we’re no good at estimating.

Finally we had a group discussion where we discussed my findings and what we can do. Here’s the output:

People often care more about the deadline as they only use 5% of the features

We need to create a better contract with the customer

When we need to synchronise with other projects what choice do we have?

We rarely measure by what we’ve achieved.

• Should be more like “we need this much business value for this much money”

Trust doesn’t exist – contracts are defense against the risk

That’s life! E.G. 2012 Olympics site build – people need to come together on a deadline

Agile enables us to react and focus on what’s important – is it scope? is it cost? is it deadline?

Also, iterative development means at least we deliver every two weeks to at least have something.

Measurement should be based on what’s been done.

However, estimation is used to measure productivity – we need better forms of measurement.

Estimation is not risk management but it’s what it’s used for.

Should the customer estimate value?

The closer you get to the end of a project the greater the pressure to estimate accurately.

Also is the impact of pressure from customer to estimate more accurately.

When integrating with 3rd parties need to estimate.

Need to recognise risk when estimating.

• Sponsors need to understand risk.

Time/duration of estimate = ^risk.

It’s about how we manage the estimate not the way we estimate where the answer lies.

• Managing risk/quantifying risk

If customers are more aware of risk they’ll make better decisions.

A good model would be to recognise the risk in the contract e.g.

x% risk = x cost

xx% risk = xx cost

Resources

Slides

Click here for ppt >>

Click here for pdf >>

Links

The Black Swan: The Impact of the Highly Improbable

Oliver Burkeman on why everything takes longer than you think

Planning Fallacy

Overcoming Bias: Planning Fallacy

Optimism Bias

Dept. of Helth Guidelines for Optimism Bias

Cone of Uncertainty Controversy

Predicting Velocity When Team Membership Or Size Changes Frequently | Mike Cohn’s Blog

Use Risk Management to Make Solid Commitments

There’s a lot more on my delicious bookmarks tagged for estimation

I recently ran a workshop at work on retrospectives. The intention was to find out the biggest problems teams face and come up with “cures” for them. However we also looked at symptoms of good retrospectives and spent some time sharing tools and techniques that can be applied to add some zing and stop them becoming tired and repetitive. Here’s the output of the session:

In the first exercise we collected everyone’s problems. Groups were asked to choose the biggest problems and these were passed on to the next table who suggested solutions. Below is a list of these “ailments” and their “cures”. I have grouped them where they overlapped between teams.

Ailments and cures

Biased chair / Agenda hijacking

• Feedback to chair and escalate if necessary
• Rotate chair
• Coach chair on “Agile” principles
• Let team choose an un-biased chair

Lack of preparation / Forgetting what’s happened

• Compensate in the meeting by having a good time line (Ed. help everyone remember what happened?)
• Prepare – personal/team log
• Remind participants to think of good and bad points
• Reminder before the meeting

Actions not captured / No obvious record or review of previous retrospective findings

• Next retrospective review actions from last one
• Capture/Document actions & follow up by Scrum Master
• Maintain backlog
• Focus on last sprint only
• Reduce actions to a manageable number

People not speaking up/shy

• Chair/facilitator needs to create the right environment
• Suggest box/amnesty
• Try different games which are more suited to retiring types

Retrospective points not shared with other teams (”Highlight points to share with other teams” on card)

• Rotating facilitators
• Shared retrospective blog
• Retrospective “lurking”
• Cross team collaboration needed

Voting system may result in valid issues not being addressed

• Non-addressed issues get rolled over (& keep votes?)
• Themed retrospectives
• Encourage team to get on well so they empathise more with issues affecting minority
• Vary the retro format (e.g. no voting)

Lack of engagement

• Book samples – try new things

Symptoms of effective retrospectives

The teams were then asked to explain how you know your having good retrospectives:

• Achievable actions
• Reference to past retrospectives during sprint
• Everyone had a chance to give their views
• Actions are carried out
• Positive team vibe
• Lower absence and higher team moral
• Lower recurring problems
• Increased velocity

Tips, tricks and tools

Lastly everyone shared techniques they’d used successfully in retrospectives.

Tips

• Split into small groups to narrow down actions (helps with large teams or with quiet members)
• Use a space without a table
• Have a a backlog of retrospective actions with done / not done next to them
• Write the output on a flip chart and stick it up in the workspace where all can see
• Location, location, location – find a good spaces and mix it up so not always in same place
• Write up the retrospective output including actions and put on a blog/wiki or send round in an email
• Forward-specting – what can we start doing now

Tricks

• Food and treats!

Tools

• Happiness Histogram – get team to rate sprint from 1-5 and plot in a histogram to get a general feel for the mood.
• Use coloured post it notes for mood then group by area (Ed. not sure I’ve got this right)
• XP Radar
• Trade Off Triangle
• Plan of Action Retrospective
• Agile Questionnaire
• Timelines
  • Called an energy seismograph here
  • Another format here
• Draw a big picture of a ship. Positive events stuck up as wind in it’s sales. Negative events as weight on the anchor