Category Archives: Uncategorized

Estimation is at the root of most software project failures

Synopsis

I believe estimation, and the way it’s regularly misused, is at the root of the majority of software project failures. In this article I will try to explain why we’re so bad at it and also why, once you’ve fallen into the trap of trying to do anything but near future estimation, no amount of TDD, Continuous Delivery or *insert your latest favourite practice here* will help. The simple truth is we’ll never be very good at estimation, mostly for reasons outside of our control. However many prominent people in the Agile community still talk about estimation as if it’s a problem we can solve and whilst some of us are fortunate to work in organisations who appreciate the unpredictable nature of software development, most are not.

Some Background

I recently tweeted something which clearly resonated with a lot of people:

"With software estimation you've only realistically got a choice of 5 mins, 1 hour, 1-2 days, about a week, and then all bets are off."

It was in response to this article Mike Cohn posted proposing a technique for providing estimates for a full backlog of 300(!) stories when you haven’t yet gone any historical data on the team’s performance. The article does provide caveats and say it isn’t ideal, but I think it’s extremely harmful to even suggest that this is something you could do.

I’ve long had a fascination with estimation and particularly why we’re so bad at it. It started when I was involved in a software project which, whilst in many respects was a success, was also a miserable failure resulting in a nasty witch hunt and blame being put at the foot of those who provided the estimates.

I’ve previously written about the estimation fallacy, that formative experience, and ran my “Dealing with the Estimation Fallacy” session at 2/3 conferences including SPA 2009. There are also plenty more estimation related articles in the archive.

 

Why we’ll always be crap at estimating

“It always takes longer than you expect, even when you take into account Hofstadter’s Law.”
– Douglas Hofstadter, Godel, Escher, Bach: An Eternal Golden Braid

Since my rocky experiences with estimation and my subsequent research into the area I’ve gone to great lengths to try to improve the way estimations occurs. At my current organisation we collect data from all our teams on how long work items take (cycle time) and have this going back over two years. It’s a veritable mountain of data from which we’re able to ascertain the average and standard deviation for cycle times.

An example of some cycle times from a team's work data
We can then take a list of feature requests, do some high level t-shirt size estimation on them and should comfortably be able to give a range of dates within which the team is most likely to reach particular milestones (the range represents the uncertainty – low, medium and high based on the average +/- the standard deviation, smart eh?).

Right?

Wrong.

We were finding that unless we were talking about stuff coming up in the next few weeks – by which point we’d generally done more detailed analysis – we were either at the far end of the estimation range or missing the predicted milestones dates altogether. It was almost always taking longer than we expected. You could argue that it was poor productivity on the behalf of the developers, but the projections were based on their previous performance so it’s a difficult one to argue, especially as our data was also suggesting that the cycle times for work items had actually been going down!

There are two main reasons things were taking longer than expected:

1. Cognitive Bias

From Wikipedia“Cognitive bias is a general term that is used to describe many observer effects in the human mind, some of which can lead to perceptual distortion, inaccurate judgment, or illogical interpretation”.

In terms of estimation, cognitive bias manifests itself in the form of Optimism Bias“the demonstrated systematic tendency for people to be overly optimistic about the outcome of planned actions”.

There is also Planning Fallacy“a tendency for people and organizations to underestimate how long they will need to complete a task, even when they have past experience of similar tasks over-running”.

It’s a huge problem, especially when we’re trying to do long range high level estimation. Our brains are hard-wired, not only to be optimistic about our estimates, but also to tend towards only thinking of the best cases scenarios. There is very little we can do about this apart from be aware it happens.

It’s only when you get round to doing the proper analysis of the work that you start seeing other things you need to do. I remember Dan North once saying in a presentation that estimation is like “measuring the coast of Britain”, the closer you look the more edges you’ll see and the longer it gets.

2. “Known unknowns and unknown unknowns”

Donald Rumsfeld got soundly mocked for his infamous “unknown unknowns” quote, but he was right. With software development, just like any Complex Adaptive System, there are many things you simply cannot plan for. We can at best be aware there will be pieces of work we’ve not accounted for in our estimations (known unknowns), but we’ve got no way of knowing how much work they’ll represent. The best we can do is know they’re there. The unknown unknowns? Well they’re even harder to predict 🙂

Why estimation can be so harmful

Estimates are rarely ever just estimates

There is always a huge amount of pressure to know when things will be done and with good reason. People want to be able to do their jobs well and it’s hard if they can’t plan for when things they’re dependent upon will be ready. This is why an estimate is rarely ever “just” an estimate. People use this information, otherwise they wouldn’t be so keen to know in the first place. You can tell them all you like that they can’t be relied upon, but they will be. As Dan North recently tweeted, “people asking for control or visibility really want certainty. Which doesn’t exist”.

Some of us are fortunate to breath the rarefied air of an enlightened organisation where the unpredictable nature of software development is generally accepted, but lets face it, most are not.

Re-framing the definition of success (or highlighting the failure of estimation)

The most significant impact of providing estimates is that as soon as they’re in the public domain they have a habit of changing the focus of the project, especially when dates start slipping. Invariably a project of any reasonable duration, which started with the honourable objective of solving a business problem, quickly changes focus to when it will be completed.

Well, there’s actually a bigger problem here already. For most organisations the definition of success remains “on time and on budget”, not whether the project has made money, or improved productivity, or brought in new customers. It is often said the vast majority of software projects are considered failures and the Standish Choas Report is usually cited here. The problem is even they define success as meeting cost and time expectations. All this report is highlighting is the continued failure of estimation.

At best providing long range estimates just supports the “on time, on budget” mentality. At worst it takes projects started with the best intentions and drags them back down to this pit of almost inevitable failure and you – as the provider of the estimate – along with it.

The consequences…

All the typical issues dogging software development raise their heads when estimation lets us down. It’s the primary cause of death marches. Corners get cut, internal quality compromised (at the supposed sake of speed), people start focusing more on how the team works, pressure mounts to “work faster”, moral drops and so on – all resulting in a detrimental impact to the original objective (unless it was to be on time and on budget of course, but as we know even then…).

It’s a vicious cycle repeated in organisations all over world, over and over again. It’s my strong belief that the main reason it keeps occurring is because we can’t escape our obsession with estimation.

“But we can’t just not provide estimates”

This is the typical response when I say we should start being brave and just saying no to requests to provide estimates on anything more than 1 or 2 months (at most) work. My answer to that is if there’s as much chance of you coming up with something meaningful by rolling some dice or rubbing the estimate goat then what purpose are you satisfying by doing so?

What’s the least worst situation to be in? Having an uncomfortable situation with your boss now or when – based on your estimates – the project’s half a million over budget and 6 months late?

Conclusion

Some people still seem to want to cling on to the idea that estimation is still a valuable activity and particularly that we are able to provide meaningful long range estimates, but as I’ve explained here it’s a fools errand. My advice to anyone who’s put in the situation of providing long range estimates is to be brave and simply say it’s not something we can do, and then do your best to explain why. It’s against the spirit of Agile software development, but that’s not even the point, mostly it’s just in no one’s best interests – well, unless you’re a software agency who rely on expensive change requests to keep the purse full, but that’s a whole different story.

Epilogue – what do we do instead?

It’s obviously no good to just say “no” and walk away and I’m certainly not advocating that, but it’s also not the point of this article to try and explain what the alternatives are. There are a plethora of options out there, especially in Agile & Lean related literature and from well known Agile and Lean proponents, but crikey, if there’s one thing that everyone within our passionate communities can agree it’s that the best software comes from iterative, feedback driven development, something which is completely at odds with the way estimation is currently being used by most organisations.

 

 

Self-organisation and team size

In my last post I talked about how self-organisation occurs and how we can try to make the most of it. This time I want to talk more specifically about team size and how I think it impacts self-organisation.

I would argue that the difference between a group of individuals and a team is the ability of the group to adapt to maintain its interests. For example, an amateur football team needs to adapt to the opposition’s tactics to try and win games and stay competitive. If they couldn’t do this effectively they’d probably lose all their games, get relegated and eventually get so fed up they’d disband and spend their weekends playing golf instead.

As we know from my last article, that ability to adapt is when self-organisation occurs and therefore we can say self-organisation is a key component of effective team dynamics.

Too big?

A significant factor in being able to self-organise is for the people who make up a group to be able to communicate & collaborate effectively and form the shared identity and shared goals which determine the group’s boundaries. This is one reason why agile proponents favour co-located teams and open plan offices, but there’s only so much that will help – 100 people working in the same office on the same project are going to struggle to maintain the types of quite intimate relationships which are required for self-organisation to emerge.

Too small?

Most attention seems to be focused on how large a group can become be before it starts to lose its ability to self-organise and work as a team, but I think it’s just as important to consider how small an effective team can be. If we look to sport we very rarely (if ever) see teams of less than 5 people and when we do it’s in sports such as rowing, where the complexity of the interactions between team members is low. When you only have 3 or 4 people in a team (as opposed to 10) the flexibility of the group to adapt to situations is limited by the number of ways the group can organise itself.

For example, in a game of 3-a-side football (soccer) someone is invariably going to have to be responsible for guarding the goal to prevent the opposition just shooting from long range or provide some defence if one of the opposition breaks away whilst your team is attacking. This leaves the other two players little option but to pass it back and forth between each other, making it rather predictable for the opposition and frankly, not much of a spectator sport.

Adding one more person doubles the number of possible connections between the group. Adding two more almost quadruples it. As this is exponential the possible connections increases greatly with each new addition to the group, but this then comes back to how many people you can realistically maintain a relationship with within the boundaries of the group and the context of its environment, so clearly there’s a balance between the two.

Software teams

From what I can see the main objective of working in teams is for the whole to be greater than the sum of its parts. I’m not saying a “team” of 3 people won’t do a good job, but it will find it hard to behave like a team as it will struggle to reach and maintain the behaviours it needs to self-organise well – the limitations of the group become more acute. For example, when someone is on holiday or off sick it’s much harder to adapt to this and if you have junior or inexperienced people in the team you are very limited about how you can share work, as neither of them can work very well on their own or together.

Scrum proponents suggests the most effective size for software teams is between 5-9 people. Jeff Bezos introduced the concept of Two Pizza Teams – team’s are at the best size when comfortably fed by two large pizzas. In fact, doing a general search for optimum or effective team sizes finds that many people have come to a similar conclusion on the effectiveness of team sizes (see here and here)  – that the team only really becomes a team with around 5 members, but starts to degrade when you reach 9.

If a small team (e.g. 2-3 people) is your only option then it’s certainly no disaster, but if your decision was how to best organise a larger number of people into contextual groups then it may be better to aim for somewhere a bit higher as the overhead of managing all the small teams combined with the increased complexity of the communication networks between the teams and the risk of knowledge silos may well outweigh any perceived benefits.

Self-organisation: it’s not a case of whether to, but making use of the fact it occurs

“The best architectures, requirements, and designs
emerge from self-organizing teams”

Principles Behind the Agile Manifesto

I’ve often had people say to me that we’re not really self-organising because we’re not all taking part in all the decisions made at our organisations, such as how the company is run or what products we should be making. This is a misunderstanding of how self-organisation occurs. The fact is we’re constantly self-organising, it’s just most of the time we’re completely oblivious to the fact it’s happening.

For example, within a group of people (such as a workplace) there are many social sub-groups, such as the early morning runners, the ones that go to lunch together and the after work drinkers. They all naturally self-organise around shared goals & desires. Even within a team where a highly autocratic management approach is in place, members of the team will self-organise around anything they can (such as making the tea or going out to buy some biscuits). The thing with self-organisation is it’s actually harder not to be doing it!

If you define self-organisation as when everyone in a team is able to control everything that affects it where do you stop? Your team? Your department? Organisation? City? Country? See where I’m going?

Self-organisation always occurs within boundaries defined by it’s context. For example, a football team might be a football pitch for 90 minutes, a group of friends on holiday the holiday villa they’re staying in or your department at work by the responsibilities it has. Something or someone always sets these boundaries. If they did not exist self-organisation couldn’t occur.

What causes self-organisation?

A complex system* will always be vulnerable to external events which are outside of it’s control, such as the opposite team, the weather or your customers or stakeholders. Self-organisation emerges when it becomes important to respond to these uncontrollable events in a manner which benefits the system. The ability to do so results in the system being adaptive. We know that software development is complex so we can determine we’re in a complex adaptive system. It’s a huge subject and not the particular focus of this article so I won’t try to go into any more detail, but if it’s a new concept to you I strongly encourage you to investigate further.

How to take advantage of the fact that it occurs

When self-organisation is on your side it can be extremely valuable. If you’re working in a complex adaptive system it’s shown that by far the most effective way to allow that system to operate is to allow it to self-organise, because that’s what it will naturally try to do.

The problem is it’s not always on your side (if you’re an oppressive Arab dictator, for example) and will often work against your goals and objectives. I’ve often seen self-organisation criticised because this happens. I’ve also seen it criticised because a team is unable to self-organise effectively when allowed to do so. In both cases you may be tempted to blame the individuals in the team. This would be unjust because in reality it is the system to blame (and probably you as the controller of the system).

To enable self-organisation to work in your favour you need to understand and ideally define the system – set it’s boundaries and try to organise it in a manner which will encourage self-organisation to occur in a way which supports your objectives. This is often called Systems Thinking. Again this is a large and well documented subject so there’s no need for me to elaborate further (if you’re looking for a good place to start though I thoroughly recommend Jurgen Appelo‘s excellent book, Management 3.0).

 

*It’s widely held that software development in general exhibits the properties of being complex (as described in the Cynefin Model).

A formative experience

It may surprise some, but I’ve been at this agile lark for less than 4 years (or maybe that’s a lot?). I’ve been motivated by Jason Gorman’s recent post and case study (and some subsequent twitterings) to share some of my thoughts from working at BBC Worldwide1 at that time, a particular project I worked on which has influenced me far more than anything I’ve been involved with before or since and how it all changed me (for the better).

The Online Catalogue (OLC) project

Shortly after I started at Worldwide I was moved into a team doing a re-write of the Online Catalogue (now renamed BBC Wordwide Sales – essentially a sales device so broadcasters can find and view details on shows produced by the BBC they could purchase and broadcast on their own channels). It was the company’s first foray into Agile and both a great success and complete disaster.

It was a success because we were a great group of people being given the opportunity to determine how we wanted to work. We made many, many mistakes (inc. classics such as “done” being development complete i.e. not tested or released), but crucially we were learning and improving all the time.2 It was an extremely empowering and motivating experience and created an incredible bond between the people in the team (although I don’t see them often I still consider many of them great friends today). The highly collaborative nature of the way we worked meant I learnt more in the first year than all of my career up to that point. The wonders of TDD, refactoring and pair programming were truly opened up to me. It wasn’t so much a jump as a giant leap. The news of our successes spread quickly and it wasn’t long before Agile software development was the de-facto approach.

No doubt you’re probably far more interested in why it was not a success and you’d be right to be (I learnt far more from what went wrong that what went well).

Crucially before development started a year had already been spent on analysis resulting in a use-case requirements document a good inch thick so we weren’t iterating on functionality, only incrementing in two weeks sprints (and knowing the audience I don’t think I need to explain further). Also the expectations on the project’s delivery were very optimistic. The combination of these two and the political and bureaucratic culture of the organisation meant failure was baked in from the start.

Shortly after development began it was evident it was going to take a lot longer than expected, but by that time the project had gathered too much momentum and too much money had been spent. At each management review the estimated completion date got longer and longer but each time no one was prepared to cut the scope or do the honourable thing and kill it. As is typical with death marches failure isn’t an option until it’s the only one left. The initial estimate was 3 months and the project was eventually completed 18 months (and well over a million pounds) later. The product itself? To it’s credit it was a much better engineered application than it’s predecessor and was a good foundation for future development, but in many ways it had less functionality than the product it was replacing.

The subsequent witch hunt was extremely unpleasant with the blame eventually being laid at the foot of the Lead Developer and Project Manager. Their estimates weren’t good enough. Fortunately our PM (who was actually excellent and by far the best I’ve ever worked with) had already left, which unfortunately meant the Lead Developer bore the brunt of it and his contract wasn’t renewed, which was undoubtedly a political gesture to higher management. There was no interest or barely any investigation into the root causes. When we did eventually do a project review the findings were filed away. It didn’t matter any more as blame had already been assigned and everyone had moved on.

I wouldn’t say we were completely without fault, there were many things within our power we would have done differently if in the same situation again, but I could say that about all the projects I’ve worked on since.

The community

Pete Camfield and Jason Gorman built a brilliant community, among other things introducing lunchtime sessions where they managed to get in many well known faces to do talks, but also encouraged us to do our own. I did my first ever presentation on “Test Smells” inspired by the xUnit book and have since presented at numerous conferences and events, something I can’t imagine I’d ever have had the courage to do without this opportunity. We also started a design patterns study group and a library (which I proudly curated).

Additionally the lead developers, coaches and some others (like myself) who were motivated to change things got together regularly to discuss how to do so (during the time I wrote an article which I think sums up the mood quite well).

Of course it’s no community without brilliant people, of which there were many (I daren’t name check for risking excluding or offending anyone).

Why I left

I think I’ve already alluded enough to the fact there were significant systemic issues with the organisation. I guess it’s somewhat inevitable (and not exclusive to BBCWW) that larger companies suffer somewhat from bureaucracy and the political environment that generates.

Most people I worked with were decent and just trying to do a good job, but the nature of the environment brought out the worst in people (in the same way buying a house in the UK seems turns everyone into cold-hearted monsters)3 and this went all the way up the chain. We were able to optimise the way we worked within teams but efforts to change things beyond this were exhausting and mostly futile. It was ultimately an environment which suited politically minded career types rather than people who wanted to make a difference. I eventually got too frustrated with banging my head against the ceiling.4

How it changed me

The combination of my experiences on the OLC project and the environment created by Pete and Jason changed me unrecognisably. I learnt that in an agile environment being a super brain developer (which I’ll never be) is really not that important. It matters far more if you give a shit and want to make things better. As my confidence in my abilities grew I became more vocal about what was wrong, at the beginning within our team, but soon (as most agile teams quickly realise) about what was wrong with the organisation which was causing many of our problems. Among other subjects my experiences on the OLC project inspired countless articles and conference sessions about (and a fascination with) estimation and predictability.

Most of all it showed me that, unlike my previous belief, most organisations are not ran very well, in fact they’re being run very badly, which means billions upon billions of pounds are increasingly and needlessly being wasted on software projects and thousands upon thousands of people are being stressed, frustrated and depressed by being involved in them. It showed me that there is a better way and that I get huge satisfaction from being involved in trying to improve that situation (well, at least not making it any worse).

Conclusion

Agile environments make people care. It means you’re much more exposed to your customers and stakeholders and spend a lot more time (than most people in your organisation) engaged in retrospecting and discussing problems. I’ve also noticed recently that people are much more critical on agile teams – something I thought was a bad sign but I’ve now realised the opposite is true5

It has it’s downsides of course. More emotional engagement means more emotional attachment. I ride the peaks and troughs a lot and find it difficult to switch off.

It’s not as much the project or organisation it makes me care about as much as the people I work with, it just makes it all more human.

 

 

 

Footnotes

1 BBC Worldwide is the commercial arm of the BBC, a wholly owned subsidiary and quite separate from the publicly funded corporation (although all it’s profits go back to fund the BBC).

2 Matt Wynne and I presented our experiences of evolving from Scrum to Lean at XPDay 2008. Here’s a write up and here’s the slides.

3 This kind of behaviour is well documented by complexity sciences but most famously by the Stanford Prison Experiment.

4 I’d like to think this doesn’t have to be the case. I think at least part of the problem at BBC Worldwide was it suffered from having too many long termers (10 years was quite normal) meaning people were quite protective of their positions or not keen to make any noise/get off their comfy chairs with the consequence the status quo was very difficult to change. Also, within the development team the ratio was weighted too far towards contractors than permanent staff (I was perm) with the result that people either didn’t care or weren’t around long enough to make it stick. I understand the IT department has recently been through a major overhall with many people having left via redundancy and new management at the top, so I doubt my experiences are still relevant, however I’ve not heard anything to say it’s improved either.

5 A recent experience report published by Benjamin Mitchell found that a team involved in a System’s Thinking based approach were the most motivated but also more critical (than other areas of the business) of how well they were satisfying customers.

Experience Report: Pair Programming Interviews

Ivan Moore wrote an article a while back on how he runs pair programming interviews. After introducing them to the recruitment process at my company I have to agree with Ivan that they’re incredibly valuable – among other things they’re a great way for a potential employee to show they have a developer conscience. I wanted to share some of our learnings too.

Real Code

We use real code and real problems we’ve already had to solve. I know others like to use programming exercises, but we like to see how candidates deal with problems which are more relevant to the job they’re applying for such as dealing with legacy code and unit testing.

Code Review

We start by walking the candidate through the code and it’s context, give them 10 minutes to review the code by themselves and then discuss their thoughts. Outside of the actual pairing exercise this is a great opportunity for them to show us how well they understand OO principles and practices by identifying any code smells. We’re also looking out for how well they explain their answers as an ability to communicate well is essential.

The Exercise

Next we roll up our sleeves and start programming. We set the candidate a challenge (in the same code base), work with them as much as possible without making it too easy, explain anything they’re unclear of and help them with any tools they’re not familiar with. Whenever we come to a tricky or challenging situation we ask them what options we have. If they’re totally stuck we will suggest possible solutions, but ultimately they’re the one making the decisions. The exercise continues until the allotted time runs out (1 hour normally).

It’s about what you do not how you do it

We explain to the candidate at the beginning of the interview that there is no finishing line. What we’re looking for most is how they approach a problem and how well they work with others. Often people who get the furthest do so by making risky refactorings and poor design decisions.

Pair pair programming

It’s always good to have more than one opinion so generally there will be two people in the interview aside from the candidate. One takes part in the exercise whilst the other acts as an observer. This allows the navigator to focus on the exercise. After the interview we get together and discuss our thoughts. Often one of us will have forgotten or not noticed something from the interview and this ensures we get the most balanced and fair view possible.

Does it work?

So far everyone who we’ve recruited who has taken part in the pairing exercise has lived up to, if not excelled our expectations of them based on how they performed. One drawback is although we do try to keep the interview as relaxed as possible it can be quite intimidating for some, however having done quite a few now I cannot imagine recruiting a developer without using this technique as part of the interview process.

Not so poor man’s dependency injection

Uncle Bob’s written a post explaining why he tries to be very sparing with the use of IoC containers. I couldn’t agree more. Overuse of IoC containers can end up with a huge amount of indirection and noise in compared to the value they may provide*.

In my mind, the main benefit of dependency injection is testing and is crucial to being able to do TDD. If you do TDD you want to test one thing at a time (one logical assertion per test). If you find you need another piece of logic which is not the responsibility of the object you’re testing you don’t carry on writing reams of code until your test passes, you create a stub for the piece of functionality, make your test pass and only then consider implementing the stubbed out code.

I’ve always been a fan of poor man’s dependency injection. I can’t find any good examples on t’web so I’ll give an example (in C#). Poor man’s basically means having a default parameterless constructor and an overloaded one which takes the dependencies. The default constructor calls the overloaded one with concrete instances of any dependencies:

public interface IUserPaymentRepository
{
     void MakeTransaction(Price amount);
}

public class TrackPaymentActivity
{
     private UserPaymentRepository _userPaymentRepository;

     public TrackPaymentActivity():this(new UserPaymentRepository())
     {
     }

     public TrackPaymentActivity(IUserPaymentRepository userPaymentRepository)
     {
          this._userPaymentRepository = userPaymentRepository;
     }

     public AttemptToPayForTrack()
     {
          ......
          _userPaymentRepository.MakeTransaction(trackPrice);
          ......
     }
}

This allows you to call the parameterless constructor in your production code, but the overloaded one from your tests. I guess this is where it gets it’s bad name as test hooks are pretty scorned upon and there’s no denying that’s what’s going on here.

Here’s the thing – in many of the situations where you need to inject a dependency it’s for one piece of behaviour and there’s no decision going on to determine which object you need for that behaviour – you know there’s only one place in your code base that has that functionality. In other words, the dependency is still there! It’s just that by using an IoC container you’ve made it magically look like it’s decoupled. With poor man’s it’s easy to see these dependencies but if you’ve palmed it all of to an IoC  you’re gonna end up having no idea what’s using what until you fire up the good ol’ debugger. What’s worse is your code metrics won’t pick it up giving you the impression your code is in a better condition than it actually is.

Of course there is a time and a place for IoC containers. It’s just that it’s probably a lot less than you thought. If there’s a decision to be made at runtime about which type that impliments IUserPaymentRepository should be used or there’s more than one member on the interface (suggesting that it is stateful) then an IoC container would be desirable, otherwise I’m often quite happy being poor.

 

 

*I actually don’t have a huge amount of experience with IoC containers, but that’s because like Uncle Bob, I’ve always tried to avoid them until absolutely necessary, however I do have first hand reports (within orgs I’ve worked for) of where IoC abuse has caused very expensive problems.

Understanding chaos and what it means to software development

Tim Ross pointed me in the direction of a truly mind blowing documentary on the BBC iPlayer about the nature of chaos, it’s fundamental role in the universe and how it explains the behaviour of complex systems such as the way birds fly in flocks and how we evolve. If you’re in the UK I’d highly recommend watching it before it gets taken down next Sunday (24th Jan 2010). It’s called The Secret Life of Chaos.

What, you may ask, has it got to do with software development? Everything actually. If you recognise that you’re working within a complex system, then you must accept the results will be totally unpredictable (that’s the chaos element), because the laws of nature say it will be so. Instead of trying to force it to be predicable (e.g long term planning, estimation based planning etc.) you allow it to behave like a complex system, which is very simple:
  • self-organisation e.g. A flock of birds organise themselves into the most appropriate formation, no one tells them how to do it – there’s no “head bird” orchestrating things.
  • simple rules e.g. Animals are impelled to mate with each other, which results in evolution.
  • feedback e.g. Mating results in offspring which, if they are successful within their system also mate and produce offspring, resulting in animals more suited to their system.

There are a lot of people within software development starting to talk about how we can harness complexity science to create better organisations and software. Here are some examples:

 

The roles and responsibilities of an agile Software Team

We’re currently going through a highly transformational period at work introducing many new concepts to people intended to better manage the flow of work and the long term sustainability of the organsation. It became clear there was a need to have some means by which to guide people as to what was expected from them and the previous job descriptions were no longer appropriate. This article contains the new responsibilities and the justification for them which I thought may prove useful to others.

Firstly, credit to Portia Tung and Pascal Van Cauwenberghe as their article on the Roles and Responsibilities of an Agile Team inspired much of what follows.

Roles

Disclaimer: These are quite generic descriptions which I hope may prove useful if you are required to formulate such things, however they are still specific to our needs. We don’t have business analysts, project managers or architects for example (and currently do not feel the need for them either) so if you feel there’s something missing from where you work that’s because it probably is.

Team Member
Developer / Tester
Lead Developer
Team Leader
Principal Developer

Some more details

  • Everyone is a Team Member first and foremost. It was interesting when drawing up these job descriptions how almost all the responsibilities are applicable to a Team Member with the other roles (in most cases) adding little more than extensions to the same responsibilities.
  • Some of the roles aren’t positions in themselves. For example, the Team Leader role does not map directly to a project manager (we don’t have any) – it is held by whoever is most appropriate (it could be the Lead Developer but it could be a tester or developer).
  • Each responsibility follows the format of the explanation of the responsibility (in bold) followed by the justification e.g.

To ensure deliverables meet the acceptance criteria given so that we do not waste time reworking them.

I think this is really important. I’ve rarely ever seen a job description which justified itself. It’s like a User story with the “so that…” part missing.

Obective

The overriding objective was to provide a description of the desired environment within which self-organisation and self-empowerment are the preferred management approach. This can be directly related back to two principles in the Agile Manifesto:

The best architectures, requirements, and designs
emerge from self-organizing teams.

Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.

“Respect for People” is also a one of the two main pillars of the Toyota Way (the other being continuous improvement). In my mind there is no better way to grow people within your organisation than empowering them and respecting and trusting them to do their job (interestingly it appears this pillar is commonly overlooked resulting in the Toyota Half-Way).

The Estimation Fallacy

I’ve had a lot of reasons to think about estimation recently and I’ve come to a firm conclusion – it’s a complete waste of time. There are so many things you could be doing that will add value to your project – estimating adds nothing. In fact it has has the adverse effect of making your project less likely to succeed. I will explain why:

We cannot predict the unpredictable

More often than not, the factors that have the biggest impact on the duration of a project we simply did not see coming. Afterwards we look back and say “ah, well we know that for next time so we won’t make the same mistake again”. We investigate the reasons things went wrong, blame people or processes and move on to the next challenge confident this time it will be a success. This is the fatal flaw. What we don’t recognise is that the problem was not the particular event that delayed the project,but that something unexpected happened at all. This is an extremely costly mistake which eventually ends with people losing their jobs and a lot of money being thrown away. Some people may argue that when they estimate they allow for this by applying a “margin of error”. How much then? 5, 10, 20 percent? The problem with these unpredictable events or Black Swans is that no margin of error could possibly account for them, especially so if the object of your estimate is to win business or commit your organisation’s finances for the next X months. Unfortunately its in the nature of our business that we will constantly be presented with “unknown unknowns” and the sooner we realise this the better.

Even without these “unpredictable” events. We are useless at predicting the future

Until recently, I was a believer in McConnell’sCone of Uncertainty which argues that the further away you are from a project deadline the more exponentially unreliable your estimates will be (this is not improved by putting more effort into the estimation process). Well I now think this is invalid. For one thing the graph is symmetrical. If this was based on reality it would mean we overestimate as much as we underestimate. If that was the case we would deliver early on as many projects as we deliver late (stop laughing). Also, it suggests that our estimates get better as the project progresses. Even with iterative development and when we estimate at the last responsible moment (e.g. the week before) and assuming no big surprises came our way (which always do), I have found we are mostly way out (I would consider anything above a 10% error margin to be enough to make it a worthless exercise). On the project I’ve been working on for over a year now, with roughly the same team (a really good team, the best I’ve ever worked with), the accuracy of our estimation has not improved in the slightest.* All we can say is (assuming no Black Swans come our way which as I’ve stressed, always do) the closer we get to the finish line (i.e. the less work there is in the backlog) the less there is to go wrong.

It is not in the interests of the customer

If the idea is to give our customers something they can use to forecast budgets then we’re not doing it. As we cannot predict the future, what we end up giving them is next to useless, in fact its likely to have a detrimental effect by lulling them into a false sense of security and dissuading them from allowing for uncertainty in their budgeting.

Dr Dobbs’ Journal did a survey on how we define success. They found:

61.3 percent of respondents said that it is more important to deliver a system when it is ready to be shipped than to deliver it on time.87.3 percent said that meeting the actual needs of stakeholders is more important than building the system to specification.79.6 percent said that providing the best return on investment (ROI) is more important than delivering a system under budget.87.3 percent said that delivering high quality is more important than delivering on time and on budget.

So why are we so obsessed with it? The most common criticism I hear of agile methodologies is if a customer is given the choice between a company that says they’ll deliver in X months and cost £X and one that will not promise anything (sic) they’re bound to go with the former. Well, the survey above would suggest otherwise, as would I. In my last job I was in the position of choosing an agency to build a website and can assure you the last thing on our mind was how good they were at meeting deadlines. We were most impressed by the agency (sadly now defunct) who, for their pitch, did research into our customers and actually started building the site rather than knocking up any estimates.

What about when projects deliver on time and on budget?

Whilst some projects do deliver on time and on budget much of this can accounted for by chance rather than excellent estimation skills. These projects get scrutinised for what went so well (at least they should if your organisation is in any way decent) and the lessons are taken away to the next project. However whilst some of the lessons learnt may well be valid, no consideration is given to the enormous impact of blind luck! Adversely to when projects go bad, people and processes are given too much credit for success. This all results in aconfirmation bias. Every time you do this is like looking for a higher piece of cliff top to balance on the edge of.

Conclusion

Estimates are good for one thing – showing how pointless estimating is. We are able to use them track a project’s progress and show where events took it on a different course that no one had expected.

Only working in an iterative process where you’re presenting your productivity to the customer on a regular basis are they going to be in a position where they can make informed decisions on the effectiveness and ongoing viability of the work being undertaken. Fail faster, fail better.

* Instead of trying to improve our estimates (again) we decided to spend less time doing it. In our sprint planning meeting we no longer break our stories down into tasks. Therefore we do not measure our progress during the sprint in such detail. So far this has had no adverse effect, but has had the effect of freeing up many hours of development time.