It may surprise some, but I’ve been at this agile lark for less than 4 years (or maybe that’s a lot?). I’ve been motivated by Jason Gorman’s recent post and case study (and some subsequent twitterings) to share some of my thoughts from working at BBC Worldwide¹ at that time, a particular project I worked on which has influenced me far more than anything I’ve been involved with before or since and how it all changed me (for the better).

The Online Catalogue (OLC) project

Shortly after I started at Worldwide I was moved into a team doing a re-write of the Online Catalogue (now renamed BBC Wordwide Sales – essentially a sales device so broadcasters can find and view details on shows produced by the BBC they could purchase and broadcast on their own channels). It was the company’s first foray into Agile and both a great success and complete disaster.

It was a success because we were a great group of people being given the opportunity to determine how we wanted to work. We made many, many mistakes (inc. classics such as “done” being development complete i.e. not tested or released), but crucially we were learning and improving all the time.² It was an extremely empowering and motivating experience and created an incredible bond between the people in the team (although I don’t see them often I still consider many of them great friends today). The highly collaborative nature of the way we worked meant I learnt more in the first year than all of my career up to that point. The wonders of TDD, refactoring and pair programming were truly opened up to me. It wasn’t so much a jump as a giant leap. The news of our successes spread quickly and it wasn’t long before Agile software development was the de-facto approach.

No doubt you’re probably far more interested in why it was not a success and you’d be right to be (I learnt far more from what went wrong that what went well).

Crucially before development started a year had already been spent on analysis resulting in a use-case requirements document a good inch thick so we weren’t iterating on functionality, only incrementing in two weeks sprints (and knowing the audience I don’t think I need to explain further). Also the expectations on the project’s delivery were very optimistic. The combination of these two and the political and bureaucratic culture of the organisation meant failure was baked in from the start.

Shortly after development began it was evident it was going to take a lot longer than expected, but by that time the project had gathered too much momentum and too much money had been spent. At each management review the estimated completion date got longer and longer but each time no one was prepared to cut the scope or do the honourable thing and kill it. As is typical with death marches failure isn’t an option until it’s the only one left. The initial estimate was 3 months and the project was eventually completed 18 months (and well over a million pounds) later. The product itself? To it’s credit it was a much better engineered application than it’s predecessor and was a good foundation for future development, but in many ways it had less functionality than the product it was replacing.

The subsequent witch hunt was extremely unpleasant with the blame eventually being laid at the foot of the Lead Developer and Project Manager. Their estimates weren’t good enough. Fortunately our PM (who was actually excellent and by far the best I’ve ever worked with) had already left, which unfortunately meant the Lead Developer bore the brunt of it and his contract wasn’t renewed, which was undoubtedly a political gesture to higher management. There was no interest or barely any investigation into the root causes. When we did eventually do a project review the findings were filed away. It didn’t matter any more as blame had already been assigned and everyone had moved on.

I wouldn’t say we were completely without fault, there were many things within our power we would have done differently if in the same situation again, but I could say that about all the projects I’ve worked on since.

The community

Pete Camfield and Jason Gorman built a brilliant community, among other things introducing lunchtime sessions where they managed to get in many well known faces to do talks, but also encouraged us to do our own. I did my first ever presentation on “Test Smells” inspired by the xUnit book and have since presented at numerous conferences and events, something I can’t imagine I’d ever have had the courage to do without this opportunity. We also started a design patterns study group and a library (which I proudly curated).

Additionally the lead developers, coaches and some others (like myself) who were motivated to change things got together regularly to discuss how to do so (during the time I wrote an article which I think sums up the mood quite well).

Of course it’s no community without brilliant people, of which there were many (I daren’t name check for risking excluding or offending anyone).

Why I left

I think I’ve already alluded enough to the fact there were significant systemic issues with the organisation. I guess it’s somewhat inevitable (and not exclusive to BBCWW) that larger companies suffer somewhat from bureaucracy and the political environment that generates.

Most people I worked with were decent and just trying to do a good job, but the nature of the environment brought out the worst in people (in the same way buying a house in the UK seems turns everyone into cold-hearted monsters)³ and this went all the way up the chain. We were able to optimise the way we worked within teams but efforts to change things beyond this were exhausting and mostly futile. It was ultimately an environment which suited politically minded career types rather than people who wanted to make a difference. I eventually got too frustrated with banging my head against the ceiling.⁴

How it changed me

The combination of my experiences on the OLC project and the environment created by Pete and Jason changed me unrecognisably. I learnt that in an agile environment being a super brain developer (which I’ll never be) is really not that important. It matters far more if you give a shit and want to make things better. As my confidence in my abilities grew I became more vocal about what was wrong, at the beginning within our team, but soon (as most agile teams quickly realise) about what was wrong with the organisation which was causing many of our problems. Among other subjects my experiences on the OLC project inspired countless articles and conference sessions about (and a fascination with) estimation and predictability.

Most of all it showed me that, unlike my previous belief, most organisations are not ran very well, in fact they’re being run very badly, which means billions upon billions of pounds are increasingly and needlessly being wasted on software projects and thousands upon thousands of people are being stressed, frustrated and depressed by being involved in them. It showed me that there is a better way and that I get huge satisfaction from being involved in trying to improve that situation (well, at least not making it any worse).

Conclusion

Agile environments make people care. It means you’re much more exposed to your customers and stakeholders and spend a lot more time (than most people in your organisation) engaged in retrospecting and discussing problems. I’ve also noticed recently that people are much more critical on agile teams – something I thought was a bad sign but I’ve now realised the opposite is true⁵

It has it’s downsides of course. More emotional engagement means more emotional attachment. I ride the peaks and troughs a lot and find it difficult to switch off.

It’s not as much the project or organisation it makes me care about as much as the people I work with, it just makes it all more human.

 

 

 

Footnotes

¹ BBC Worldwide is the commercial arm of the BBC, a wholly owned subsidiary and quite separate from the publicly funded corporation (although all it’s profits go back to fund the BBC).

² Matt Wynne and I presented our experiences of evolving from Scrum to Lean at XPDay 2008. Here’s a write up and here’s the slides.

³ This kind of behaviour is well documented by complexity sciences but most famously by the Stanford Prison Experiment.

⁴ I’d like to think this doesn’t have to be the case. I think at least part of the problem at BBC Worldwide was it suffered from having too many long termers (10 years was quite normal) meaning people were quite protective of their positions or not keen to make any noise/get off their comfy chairs with the consequence the status quo was very difficult to change. Also, within the development team the ratio was weighted too far towards contractors than permanent staff (I was perm) with the result that people either didn’t care or weren’t around long enough to make it stick. I understand the IT department has recently been through a major overhall with many people having left via redundancy and new management at the top, so I doubt my experiences are still relevant, however I’ve not heard anything to say it’s improved either.

⁵ A recent experience report published by Benjamin Mitchell found that a team involved in a System’s Thinking based approach were the most motivated but also more critical (than other areas of the business) of how well they were satisfying customers.