I’m a big fan of retrospectives having found them incredibly useful in ensuring teams can focus on continously improving their process. However they are difficult to get right and I know many teams struggle to get much out of them and often give up altogether. I think this is a big mistake.
Firstly, if you haven’t read it, I’d highly recommend Agile Retrospectives: Making Good Teams Great by Darby, Larsen & Schwaber. A great introduction to retrospectives with a lot of plans and ideas to help you out (although I have to say I find many of them too long for my liking).
However, beyond that book there really isn’t much else available so I’ve started a wiki to collect retrospective resources. Without further ado I’m delighted to announce:
The Agile Retrospective Resource Wiki or http://retrospectivewiki.org
A plea for contributors
Everything else you need to know is on the wiki, but I would like to take this opportunity make a plea for contributors. The content is currently a bit limited as I’m the only contributor so far so if you’ve got any cool retrospective plans, tips, tricks or anything else please add them.
Along with Matt Wynne I’ll be running the Retrospective Surgery session mentioned on the wiki at SPA2009 next week so I expect we’ll be getting some juicy new content imminently.
James Shore recently wrote an article on agile mediocrity:
“In sales, they teach that the first objection is often a false objection–misdirection rather than a real concern. “Does Agile scale?” can be one of these false objections.
I wasn’t really being asked “Does Agile scale?” (By now, though, we know the answer: yes.) What I was really being asked was, “Does Agile work in large, dysfunctional organizations? Can I keep doing all of the ineffective things I’m required to do and still say I’m Agile? I can’t have a co-located team–it’s out of my control. I can’t have active, involved product managers–they’re too busy. I can’t create cross-functional teams–it would disrupt our reporting hierarchy. How does Agile scale to mysituation?”
It doesn’t… at least, not well. Welcome to Mediocrity.”
I’m slightly more sympathetic than James (who has clearly become frustrated seeing the same pattern over and over again) having worked in a large organisation adopting agile and see how we can quickly get to agile mediocrity, but struggle to go any further. Typically the catalyst for change is some enthusiastic developers and a forward thinking dev manager bold enough to take a chance that the Agile gumpf these guys keep going on about might just solve a few problems. This has some success, which gets jumped on by higher management who are lured in by the buzz and enthusiasm and it gets rolled out to other teams, however then it starts hitting walls. Firstly is the resistance within the department (and this is relatively easy to get over), but then it becomes clear (to the agile teams) that to achieve real results change needs to occur further afield and here’s where it all starts to seize up.
Agile is a bit like dropping a bomb in an organisation in that it creates shock waves which spread outside the IT department and have the potential to change the entire organisation (after all agile is the child of lean production which can be applied everywhere ). The problem is the wider the shock waves spread the more difficult the change is. Large bureaucratic organisations will always suffer from more inertia and change will be slower. I guess here is where a lot of people might get frustrated and start saying things like “it doesn’t scale”. The big mistake is blaming agile for your organisational inefficiencies. It’s not the fault of agile that your company is a lumbering old dinosaur that shares the characteristics of a supertanker when trying to change direction ;)
In my mind this is largely a management problem. They’re often very keen on Agile when it doesn’t mean they actually have to do anything. A self-motivated software team trying to make things better? Great! Restructure the organisation so the delivery team is sitting and speaking with the customer, employ in-house testers and allow time for learning in working hours? Mmm not so attractive.
It’s not sufficient just to roll out Scrum or install a Kanban board. As I’ve said similarly before to succeed with agile you need to look past the latest fashionable methodology and really understand the underlying principles. The Agile Manifesto and the principles behind it would be a good place to start.
One of the keynotes at XPDay 2008 was from Dan Jones, the author of the books The Machine That Changed the World and Lean Thinking and one of the team of people who came up with the term “Lean Production”. It was quite humbling to be in the same room as the living person who has probably been more influential than anyone else to modern business processes in every industry and all over the world.
One of the most interesting things he said was that when they were coming up with the name for what they were trying to promote they considered “agile”, but thought it would be too difficult to sell and so decided on Lean instead. Also, he didn’t have much understanding of modern software development, but at a glance saw very little difference between what we called Agile and he called Lean.
So when we talk about Lean Software Development being an Agile methodology we’ve got it the wrong way around. Agile is Lean, Scrum is Lean, XP is Lean. You were already doing Lean Software Development, you just didn’t know it!
In my mind it’s time to drop the titles (which all carry too much stigma) and simply start referring to it as professional software development.
XPDay 2008 took place last week and I felt it was a great success. The open space format really excelled and there was very little I saw/took part in that wasn’t interesting or generating new ideas. The general mood seemed to be that the understanding of agile practices and principles was quite mature now and most of people’s problems were around where we are coming up against and conflicting with the business context (5 year plan’s, understanding the need for slack and long term benefits against short-termism).
We were also honoured with a keynote from the inventor of Lean, Jon Daniels and his colleague Marc Baker who gave us a frightening insight in to the state of the NHS and how Lean practices are having dramatic effects, but I want to post separately about them.
Other sessions:
Software Craftsmanship
With Jason Gorman’s software craftsmanship conference coming up in the new year, John Daniels wanted to talk about what craftsmanship meant to us. It was a really interesting discussion and the general feeling was that people were uncomfortable with the idea that we could define what it meant to be a software craftsman as there are far to many skills and too much diversity in the types of these skills. This was summed up well by the following: To explain what a craftsman is, Keith Braithwaite used the example of his uncle(?) who has been a farrier for most of his life with people coming to him to learn the skills, yet he only considers himself a journeyman and we explored the similarities/differences between what he did and what we do. In the end someone came up with the excellent point that we don’t make and fit horseshoes, we build cathedrals and there are so many different skills needed (which change all the time) we could never be expected to “master” them. I’m going to Software Craftsmanship 2009 and I look forward to seeing how it pans out.
Why aren’t they typing more?
Douglas Squirrel asked us how we deal with people asking unanswerable questions like “why aren’t they typing more?” and “you need to focus on productivity rather than agility”. There were lots of interesting ideas, but the most revealing one for me was that we need to be able to deal with people rationalising rather than being rational (e.g. the idea that being busy = productivity). Someone mentioned a book called Predicably Irrational which investigates this phenomenon (and I shall add to my reading list). Most of what we do within an agile context flies in the face of conventional wisdom and is very radical compared to the most of the organisations we work with/within. I think this is a problem as a community we need to spend more time looking into.
Scrum is getting a real bashing at the moment as you can see here and here and I think it could do with some defending.
I was on a team that adopted Scrum and it really empowered us. After a while, pair programming, TDD and refactoring became common-place because we learnt through the iterative process that they helped us write better software. People saw the positivity in the team and the productivity improvements and it lead to a fundamental shift in the department’s culture. Now all our teams are doing some form of iterative development, BDUF has gone out of the window and practices such as TDD and pair programming are actively encouraged by the management! People at all levels can see the difference it’s made to our productivity and the reputation of our department.
So Scrum can be very successful. It would never have got this far if people weren’t doing Scrum and getting positive results, in fact agile would not have become so big if it wasn’t for the success of Scrum so we’ve got a lot to thank it for.
However, here is where I think some of the criticism sticks:
Three people on the team above (including myself) went on the Certified Scrum Master course ran by Mike Cohn who is an excellent teacher, but I’ve since been on a “Scrum Estimating and Planning” course which seemed more geared towards telling people what they wanted to hear rather than any fundamentals of agile and the problems it’s trying to solve. As James Shore suggests in his article, the quality of the teachers on the CSM courses has a huge impact.
As company’s will always prefer to send people on a course rather than create a learning culture within (especially if they’re very small, which is more forgivable) the 2 day Certified Scrum Master course will be with us for a while yet and most significantly, will continue to be the main entry point to the agile world. People will continue to adopt Scrum because they’re failing with whatever they’re currently doing and continue to fail because the Scrum trainer will teach them how to write a story or run a stand up but not address the root problems, partly because you could never go into such detail in such a small amount of time and partly because that’s not the point of the course.
“Certified Scrum Master” suggests that you’re some kind of guru (it certainly sounds a lot more impressive then my BA(h) in Business and Quality Management which took me 4 years to get a mediocre grade in). Perhaps a name change would make it more credible. If organisations were not led to believe this 2 day course is going to solve all their problems then they wouldn’t be in the situation they’re in.
Edit: Since writing this article (a few hours ago) I’ve had a significant change of heart and feel I’ve incorrectly laid blame at the foot of the Scrum Alliance. It’s not their fault that their Certified Scrum Master training (which is designed to teach people how to become Scrum Masters, not solve all their problems) has become so popular. However, it is a bit unfortunate that the name could so easily be misinterpreted to mean more that it is and it’s also unfortunate that organisations hook on to the most simple looking solution to their problems (and no, I haven’t been threatened by Scrum mafiosa or bottled out I’ve just changed my mind).
It’s startling to see how much momentum has been gathering around Lean Software Development and I don’t think it’s a coincidence that at the same time many people seem to be falling out of love with Scrum.
At last year’s XPDay there were no sessions on Lean methodologies, this year there are three (the one I’m doing with Matt Wynne, Karl Scotland’s on Kanban, Flow and Cadence and a keynote from the Lean Enterprise Academy).
There’s something about this that makes me feel slightly uneasy. What’s so wrong with Scrum? Well, I’d be the first to stand up and say that Scrum has it’s failings – they’re the same reasons it’s become so popular. The simplicity, clear definition and business-friendliness of Scrum make it easy to sell (arguably, unlike XP) and waterfall or traditional development dynamics (project manager, use-case, Gantt chart, meeting) can be easily translated into Scrum’s characteristics (scrum master, story, burn down, stand up). Unfortunately all too often this is what happens resulting in what’s become know as Cargo Cult Agile, or WAgile, the underlying concepts and belief systems being mostly ignored. When I did Mike Cohn’s Certified Scum Master Course his principle lesson was “Inspect and Adapt” which he repeated endlessly throughout the course, but when I look on the Wikipedia entry for Scrum this doesn’t even get a mention so it’s easy to see how this happens.
So what’s so different about Lean? Well for a start there aren’t any clearly defined rules, but instead principles such as Eliminate Waste and Build Integrity In. However it strikes me that there’s just as much room for abuse with concepts such as Kanban, Minimally Marketable Features, Cumulative Flow Diagrams and so on. Consider this exert from The Toyota Way (which I’ve shamelessly stolen from James Shore’s article on Kanban Systems – I am actually reading this book as we speak, but haven’t got very far yet):
“…TPS experts get very impatient and even irritated when they hear people rave and focus on kanban as if it is the Toyota Production System. Kanban is a fascinating concept and it is fun to watch… When is the kanban triggered? How are the quantities calculated? What do you do if a kanban gets lost? But that is not the point… The challenge is to develop a learning organization that will find ways to reduce the number of and thereby reduce and finally eliminate the inventory buffer… So kanban is something you strive to get rid of, not to be proud of.”
Many of the reasons people aren’t being as succesful as they’d like with Scrum are exactly the same reasons they won’t be any more successful with any other methodology. People tend to focus on tools because it’s a lot easier than trying to tackle the often very difficult, challenging and more fundamental problems they grew from. Real change is hard and takes time, a very long time in some cases.
If you’re failing with Scrum don’t think lean, kanban, extreme programming or any other colour of agile will save you. Essentially if you’re failing it’s because you’re doing it wrong. However if you’ve found Scrum is working really well for you and has brought enormous benefits maybe you should come to the talk Matt and I are doing at XPDay and see how we evolved to a more lean process.
Update: since writing this James Shore has written a frighteningly similar post on The Decline and Fall of Agile, but I guess great minds think alike huh? ;-P
The Plan Do Study Act Cycle
http://www.tin.nhs.uk/index.asp?pgid=1130
Clearly this has been in use by the NHS for quite some time.
It’s also known as the Deming or Shewhart Cycle: http://en.wikipedia.org/wiki/PDCA
W. Edwards Deming is the godfather of the Quality Management movement and a man I studied in great detail (but have since totally forgotten) in my Business and Quality Management degree, which I hated. I find it very amusing that half the reason I got into software development was because I was so disillusioned by my degree course and now I can’t go anywhere without hearing about the likes of Shigeo Shingo, TQM and JIT. The difference is now I understand exactly what they where talking about.
Why do we need roles in an agile team?
Self-organisation is the goal of an agile team, but this does not happen unless the environment is provided. It is the responsibility of the leaders in a team to create and maintain this environment. However, we need to avoid creating bureaucracy and dogma so should try and steer clear of tightly defined roles and responsibilities. The boundaries need to be loose enough so we can get the most out of individuals and customise the roles to the requirements of the specific team.
Why add more roles at all then?
In my organisation either you’re a Developer or a Team Lead. This is not a fair reflection of the value our colleagues bring to the organisation. A great developer is worth exponentially more than an average developer and excellence needs to be appreciated. Also, what makes a developer good are not necessarily the same skills we’re looking for in a team lead.
The responsibilities an agile team needs to be successful
Below I’ve listed responsibilities desired in an agile team. Whilst I have grouped them by the roles of Team Leader and Lead Programmer they are by no means mutually exclusive and do not even have to be the responsibilities of either. We want to be creating teams that have senior people capable of taking on as many of these responsibilities as possible and then allowing them to decide who is best suited to do what. This will have the dual effect of empowering our teams (therefore reducing their dependency on management) and allowing people the flexibility to develop themselves in the areas they enjoy the most rather than feeling they are competing for a single position or even worse, not seeing one that’s suited to them at all.
Team Leader
Communication champion.
Gets the most out of the team.
Empowers the team.
Provides direction (can be technical and process).
Resolves conflict.
Removes blockages.
Makes sure everyone is happy.
Ensures standards and processes agreed within the team are maintained.
Ensure all members of the team have a proper understanding of the customer’s requirements.
Lead Programmer
Code champion.
Responsible for the quality of the code base.
Responsible for the underlying architecture for the software.
Responsible for technical decisions.
Ensure all members of the team have a proper understanding of the technical vision.
Train and informs developers on coding best practices.
