At the recent XPDay 2009 conference in London I organised an open space session under the title “Agile isn’t solving our customers problems because they’re not here”. It was driven by my feeling that whilst when Agile is being done well it is improving the reputation of software development, the impact it’s having is relatively minor. My biggest takeaway from the session is that nothing in our Agile toolkit really addresses the needs of our customers*.
In the short time I’ve been involved with the community I’ve seen almost no discussion or articles which involve any contribution from people outside of the IT department and none of the methodologies/name clouds I can think of appear to have been developed or evolved with any collaboration from the people driving the work.
I don’t mean to disparage all the hard work and enthusiasm people (I include myself) have put in to trying to make things better, I just think the fact that we’ve left out the most important people from our discussions has caused us a lot unecessary of pain. Agile adoption is never a smooth affair, especially the process of “convincing” people outside of the development department. If we involved our customers more in the process of forming our principles, tools and methodologies we would surely get where we all want to be a lot more quickly. Remember, people don’t resist change, they resist being changed.
Here are some examples of what I mean:
- Speaking to my CEO in the pub the other day he said he often finds it “patronising” when we try and impress on him the importance of the the things we’re trying to do.
- When I first started with Scrum I countlessly came up against resistance and cynicism when trying to encourage customers to participate in traditional Scrum meetings such as retrospectives, planning and stand ups. Nothing Scrum teaches prepares you for this (I’ve done both CSM and Estimating and Planning courses and read a lot otherwise).
- The terminology we use (e.g. Scrum, Sprints, Stories, Kanban, eXtreme Programming) is totally immature in some people’s eyes. One useful takeaway from the session was to stop using inappropriate terminology in front of customers.
I have an enormous amount of respect for our CEO and others within our organisation for putting the trust they do in us even though they (understandably) find so much of what we do beguiling and irritating. In my experience you would be very lucky to be able to work with someone who is prepared to take that kind of risk – in most places I’ve worked you just don’t stand a chance.
For me, the flaw lies deep within Agile. It was never designed to address the needs of customers. XP and Scrum were designed for fixing dysfunctional environments. The terminology was designed to appeal to developers. When most of the Agile principles and methodologies were developed the need was different and they don’t appear to have evolved. New methodologies suffer the same – why does there have to be an introductory guide to Kanban for Managers? If we’re having to try and sell this stuff to people I think we’ve already lost most of the battle.
In the last 15 years or so most of the “microeconomic” problems with software development have been solved. The majority of people writing software may still not be doing it well, but the answers are there if you care to look. The big problems are still out there though and as a community we need to start addressing them and I think the only way we can do this is by getting our customers more involved in the debate.
This post is really a rallying call to all those that feel the same as I do. I’m keen to start doing something about this, but where do we start?
*I use the term customer here to mean anyone who is a customer of a development team.
As an “Agile translator,” I find the established terminology a bit complicated, but I’m going to err on the side of using it rather than attempting to create yet another vocabulary.
To me, “Agile” has pretty useful contours, but several cliques are fractured around turf (money?), so that instead of going deeply, there is a lot of brand creation and marketing. This has lead to practices defined so tightly that no one else can play,–the antithesis of extensibility.
Certainly, the core,–complex adaptive system theory, was never the property of software development. And, the open nature of the cluster of documents around the Agile Manifesto means “Agile” can be translated without too much adjustment and broadly adapted.
Agile is young yet and it’s OK that it’s not perfectly aligned with needs outside of software development.
Great article and some very interesting points that I’ve not seen vocalised elsewhere.
I’ve had recent experience in the use of Scrum in two very different organisations. One with a turnover approaching £4bn, and thousands of staff, one a much smaller concern and what you say applies, in spades, to the former. Scrum worked like a dream and delivered real value to the business in terms of the ability of the team to adapt to a constantly shifting business landscape. The executive needed no persuading that Scrum was the tool that gave them this ability. However, would they ever wish to sit in on a sprint planning meeting – no chance!
The way this was tackled was by applying a veneer of “classical” project management to the exercise. The executives (a heck of a lot of the active board!) attended a “Steering Committee” meeting once a month (sounds much more impressive than a sprint planning session, doesn’t it?) where they were able to set business priorities for the “Project Manager” (read proxy “product owner”) who was then responsible for setting priorities at the next sprint planning session.
The exec knew it was Scrum in the background (although the customers often didn’t) but I think the overwhelming view was that Scrum was a silly little roleplay that developers liked to play, but if it made them productive, so what?
In the smaller organisation, we started Scrum from the ground up and I’m always a little ashamed, when trying to sell an idea to decision makers, to have to talk about burndowns and srum masters…
The Scrum implementation was a great sucess in the end but that was in spite of the terminology, not because of it. A ScrumSpeak>ManagementSpeak translator anyone?
Maybe.
Rob,
Thanks for this article. You are very right, and I think the place to begin is, as you pinpoint one main problem there, names and terms.
I like the “people resist being changed” bit. One thing we can do is speak their language. This has been heard to improve understanding.
Your point reminds me of the way Dan North came to coin the term BDD because he couldn’t gain understanding for TDD.
Funny, that so many of us are searching for all sorts of “smell”, but no one noticed the big one;-)
Thanks again,
Olaf Lewitz
I think it is a mistake to talk about agile details with the CEO. Agile is a great way to develop software. Much of it is too detailed to be interesting for a non technical CEO.
The thing you should be talking to a CEO about is how they decided on what their goals were and how that was translated into a need for a new system. Take a look at systems thinking. That is much more relevant for a CEO and will get them into the same mind set as agile.
Great article Rob. I guess I have two observations:
1. There *are* people who talk about how to integrate agile development processes into the wider organisation
2. Those people are not part of the ‘Agile community’
The ‘Agile community’ has always been driven from the development/IT side. That’s fair enough as that’s where the impetus started, but its not a sustainable position. Sadly the ‘Agile community’, as embodied by the promoters of Scrum and DSDM particularly (and now Kanban), has never embraced those who want to fit these processes into large and unwieldy organisations for whom yearly planning and budgets are a fact of life. These ‘outsiders’ talk about things like ‘putting the system into maintenance’, ‘return on investment’, ‘hard deadlines’ and ‘resources’ terms which sit uncomfortably with a community that still, 15 years on, consists mostly of developers and ‘coaches’ … and salesmen who want to sell into that community.
Some of the terminology is infantile but that’s not the root cause of the issues IMO. Most CEO/CIOs I know couldn’t care less whether you call something TDD or BDD, a Sprint an Iteration or an Episode, if they understand the *benefits* that such an approach gives their business. And that’s the problem: the Agile community’s message is driven from the bottom up and too many members of that community are unable to express business benefits in business terms. When we sit down and explain to a CEO/CIO that we will deliver a system that will have them trading with consumer customers within 3 months, executing B2B transactions within 6 and providing a partner marketplace within 12, that the costs will be 60% of the current budget, that we believe we can shorten the time to ROI form 18 to 14 months, and that we have an approach that allows them to choose whether they sustain investment and deliver many more features or reduce the ongoing cost whilst still maintaining the current feature set, they will listen further. If we start out by explaining how by handing over a manual on managing requirements and prioritisation they wont.
<>
I find this surprising. I’ve found that the frequent delivery of business value does address the needs of my customers. It’s exactly what they want.
<>
I’ve always found it really easy to convince people outside of the development department. Most of the problems I encounter are within the IT department. I often find that the goals of the CTO and CEO are not aligned (but I guess that’s more of a large organisation issue). There are Business Analysts and Customer Proxies who sit between the customer and the developers preventing good collaboration.
I have found many agile teams that do not engage with the business and some appear to treat agile as the end goal… For me agile is a collection of useful tools and customer collaboration is key to a successful project.
I don’t think it’s Agile’s fault that the CEO sometimes find the development team patronising.
Rob, very good points. Unlike Paul, I do think the language is a problem: one of the big issues in bringing software into the organisational fold is the “speaking the same language” thing: I think it supports many software folks’ sense of difference, of superiority, to have their own language. After all, we live in the world of engineering and logic: what do “mere business people” know (the answer, difficult for some to swallow, is – actually, quite a bit).
I’m sure this is at the root of your CEO’s sense of being “patronised” by IT people. What’s more (and this is something the “Agile Enterprise” people need to understand) it’s a bit much, now, for people in the software world to be telling senior people in the business how they should run their organisations.
The language of agile and lean in software (and the language of Software Craftsmanship, come to that) can make us sound like children, idiots or cult members.
Thank you for an interesting post Rob.
I disagree with you when you state, “For me, the flaw lies deep within Agile. It was never designed to address the needs of customers.” The first principle in the Agile Manifesto is, “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” To me that reads the point of Agile is to satisfy the needs of the customer, specifically through the delivery of software *they* deem valuable. I do agree though that both Scrum and XP can be used highly effectively to find and address the dysfunction in organizations.
“If we’re having to try and sell this stuff to people I think we’ve already lost most of the battle.” I agree with you here, and somewhat on the terminology point. When I speak with CEOs they don’t care what methodology the IT department or developers use, as long as they see value coming out the other end. Where I see people trying to “sell” Agile is when the corporate culture doesn’t necessarily support the values of Agile – collaboration, transparency, involvement. They then seek permission to adopt the principles through the selling of Agile.
Ultimately we can all have all the process we want, but if we don’t address the root cause issues that Agile addresses, we won’t be effective. And that’s all about the people involved.
We start by changing the tone of the conversations we have each and every single day.
As a community, we need to evolve beyond terms like “agile”, “scrum master”, “chickens & pigs” and “single wringable neck”.
What we are doing here should become plain ole good practices, instead of carving up endless niches within our community and alienating newcomers.
Interesting post, Rob.
I would agree that, in the grand scheme of things, how fast we climb the wrong mountains is a moot point.
How we collaborate with customers is paid a massive amount of lipservice in IT. We talk about it a lot. Just like we talk about this mysterious fairy dust called “value”. In reality, though, we’re usually pedalling a solution that’s looking for a problem. And I include our working practices in that where the rubber meets the road.
There is work out there – my own included in years gone by when I wore the “architect” hat – to genuinely try to bridge the gap. My own take on it is that information systems aren’t constrained to computers, and tests don’t have to be just for software. A test script could be written for a business scenario in, say, a restaurant and used to build and validate our understanding of that business.
I did some work with Duncan Pierce on a project for a newspaper delivery company and we wasted months trying to understand the business by sitting in meetings and talking to customers. Eventually, in desperation, I insisted we actually spend a day at a delivery depot and watch all the processes in which our software was going to be used. It was a revelation. All sorts of things we just couldn’t make sense of suddenly came into focus, and we began to establish a vocabulary that was based on what the business did rather than what their software systems thought they did – and, boy, were we off target up until then!
As many SAP customers may know, it’s not unheard of for businesses to change the way they work to suit their software systems. I have seen this many times. You hear people using software terminology to describe their work, when it really ought to be driven the other way around.
I’m not going to say the “A” word, but I know from experience that it is perfectly possible to build a much stronger shared understanding by seeing it and trying it for ourselves. After all, that’s how our customers learned their business. Not by sitting in meeting rooms with programmers yapping about it 🙂
So, I guess I’m saying that some people ARE trying to address this, and there are some practices and techniques that can help bridge the gap that don’t involve nonsense like “enterprise architecture” and “business process analysis”.
But they are not in widespread use because most programmers are frankly not past the “cool technology solution looking for a problem” stage.
As others have hinted, this seems to be a contradiction “improving software development” but “impact is relatively minor.” What then is improving or changing that is helping our reputation?
The places where I’ve seen reputations improve without improving the business was usually associated with a lot of talk. Lots of talk, lots of optimism, lots of “everyone is doing it and it is working so well!’ eventually followed by “nothing seemed to have improved much.”
I usually first determine if the company is delivering on its promises (on-time, quality). It is always interesting to hear about “the great improvements we’ve made” but see no evidence of an impact.
I’ve also seen companies try to fix poor performance by using a shiny new techniques (e.g. Scrum) but still “wing it” when promising customers when a product can be delivered. Once we fixed how commitments were made, software development’s reputation and performance went way up – and most were still using incremental/waterfall approaches.
When we delivered on time and with quality, most senior management rarely cared how we did it. Which was a shame, because we needed them to support and reinforce all the good principles. Otherwise, they inevitably undermined the great progress with well meaning, but contradictory, policies and initiatives.
Bruce Benson
http://PMToolsThatWork.com
http://Twitter.com/BruceWBenson
It seems popular now to say agile doesn’t really help the customers/business. I think it might be more accurate to say certain dev organizations who say they are practicing agile are not helping their customers/business.
Nobody said agile dev would magically fix all problems. We all know that what makes us able to help the customers and business succeed is the people. IME, agile values/principles/processes/practices enable good people to do good work. Additionally, Lean concepts such as learning as much about the business as possible, thinking for one’s self, and a culture of respect and learning help development teams solve problems for customers.
The execs here at my company would tell you how much the development team has been able to do for the business since we implemented agile 6 years ago. But it’s not so simple as ‘doing agile’. We’ve worked hard to learn the business and improve our process.
I agree with you entirely Lisa – good people and good teams do get there in the end. I just feel that a lot of the pain that I and others I know have been through in the process of getting past the “Agile adoption” stage could be avoided if we shared more experiences between ourselves and our customers about the way we can work most effectively together rather than the current defacto which is to Scrum bash our customers into submission.
Thank you for this post.
Why don’t we call it just ‘software development’? Agile, XP, Crystal, Scrum, TDD, and others are engineering and management practices to help us deliver higher quality software.
We do not want Agile to be the new dogma. We are doing software development by implementing the latest thinking as we become more mature. We learned a lot from the low ROI of implementing large initiatives (e.g. CMMI) in some organizations.
We learned that accreditation is not even a necessary condition for developing quality software with high customer value.
My fear that one day might come we say we are TDD or Agile certified as we use to say for ISO.
I think we need to learn all of these practices to help us making better decisions.
Hear, hear.
—Michael B.
Thanks for the post Rob, interesting points all round.
However, I’m confused. I’ve been practising agile for three years now, and while I don’t claim to be the world’s biggest expert, I’m very familiar with it and similar practises. My understanding is the agile tries very hard to engage the customer in the software development process, so that we developers might understand them, their requirements and their priorities better, and they might better understand what we’re up to, what our risks are, and how we are progressing.
To my mind, this sounds like exactly the sort of engagement with customers that you say is lacking in agile. So clearly you mean something other than this – but I have not understood what that other thing is.
If an agile developer like myself doesn’t quite see what you’re getting at yet, then I feel as though you need to describe it in more specifics.
With respect,
Jonathan
Thanks for your comments Jonathan. My thoughts may not have struck a chord with you but they clearly have with many others here so I must be on to something 🙂
My point is not that Agile is not customer focused in principle, but that none of the the debate and formation of the principles, methodologies and tools under the agile umbrella have ever involved the customer. The Agile customer is almost some god like creature we worship but are too in awe of to actually ask what they think. The process of Agile adoption and our ability to solve the bigger problem of realising business value would be a whole lot easier if we did.
Hmmmm, I know you said “Agile is deeply flawed.” But if it is executed in a customer-centric way that its principles seem to indicate then it “should” deliver the added value it is looking for.
One way or another unless you extract high quality requirements from “the customer” you can’t “add value” by rapidly designing/coding/testing application that solve real business problems.
Pingback: Best Links of the Week – Christmas 2009 Version : Agile PM Scrum
Great post Rob.
Good to see folks questioning things and not accepting “just because”. I am about to launch a new book titled “SDLC 3.0: Beyond a Tacit Understanding of Agile” which is something akin to your perspective.
I agree about the “IT centricity” of our terminology and what amounts to Agile jargon. We know that most of the terminology is invented and relates to concepts and ideas that have much earlier origins. This differientiation has had success within IT but has also caused some to turn off; and it is definitely not business terminology. I believe that the world of Lean and “more for less” is now as important to customers as “being agile”. The business/customer understands “value-streams”, collaboration, waste reduction, cycle-time and throughput. They don’t care about daily scrums, chickens and pigs, YAGNI – in fact in my experience it makes IT look ridiculous.
The ideas presented in the book call for a “Complex Adaptive System of Patterns” rather than wholesale methods. I make a case for integrating practices independent of method brands such that experience can be leveraged. There are some universal truths across modern methods which serves a common ground (iteration & negative feedback, waste reduction, customer collaboration due to tacit knowledge). But each community (Lean, Agile, Unified Process) grew up from a different set of experiences, so the practices yield potentially unique value propositions. Some of the problems now faced in Agile have been solved by other communities – but alas, the communities don’t interact too well together.
The book presents a rationale for what Agile really is – a brand and a movement. And while the values of the Agile Manifesto are “superstitiously declared” as per David Anderson, the practices behind Agile are very sound, and are based on Systems Theory, Theory Y management, Emergence and other foundational science. For furthering the Agile movement beyond the “Trough of Disillusionment” as Gartner says, I believe that various bodies of knowledge – like Control Systems Theory, (or as Alistair Cockburn believes, Gaming Theory) can take us to the next level.
I have been blogging about these issues at http://www.fourth-medium.com/wordpress . And if you are looking for others that are thinking similar thoughts, check out the SEMAT initiative – http://www.semat.org/bin/view – which includes at least 4 well known Agilists – Robert Martin, Ken Schwaber, Alistair Cockburn and Scott Ambler.
Hi Mark,
I’m really pleased I posted this article now as I would not have heard of either your book or the SEMAT initiative which both sound like they’ve come from similar frustrations. I shall look forward to investigating both further.
You might be interested in content on InfoQ with the tag “Customers and Requirements”
http://www.infoq.com/cust_requirements
Including this presentation from Agile2007 by a Product Owner (i.e. “customer”)
http://www.infoq.com/presentations/alexia-bowers-agile-leadership
It is not uncommon for a coach and customer to present together at this conference. Agile2010 is in Nashville, a high-quality source of information – attended by developers and their customers.
deb
I think David Bland got it right. Think about who you are talking to, the context and the points you are trying to get across. Focus on the language you are using and your audience. Hasn’t it always been that way?
Hi Rob
I’ve come in a bit late to this thread but there are some good points made. The ‘language’ and terminology’ issue is important and I don’t think it’s surprising that some non-IT will people will be intimidated and put off by it.
There are also some rather deeper issues which people need to recognise, in my view:
– Firstly, we don’t have agile financial planning in organisations. We have waterfall financial planning where people have to secure funding months – years even – in advance. On a so-called ‘agile’ project, it is quite likely that the key decisions on cost, time, and scope will have already been made in secret, in order to get this funding. You just don’t know about it. Not so long ago I was involved in (what I thought) was the early stages of piece of work to define options, scope and so on, and was told later that the business area already knew how long it would take, what they would be prepared to pay, and even, astonishingly, what the technical platform would be. I only found out about this by accident. Furthermore it’s not a rare occurance. I wouldn’t go so far as to say that agile is often ‘window-dressing’ (I do know someone that has said this) – but it’s not far off it sometimes. If you’re wondering why stakeholders and users don’t turn up for your stand-ups or are reluctant to get involved generally, it’s probably because the decisions have already taken and promises made. In their view, how it gets delivered – be it agile, lean, fat, waterfall, pretty-spring-in-the-highlands, doesn’t make much odds.
– Securing funding DOES mean up-front-definition of what you are going to do. This is the reality of the world. (Not a world I would have created, but the world we’re stuck with). Waterfall attempted to address this but fails by assuming it is all a one-off exercise to be handed-off when ‘completed’. This is clerarly a nonsense. On the other hand, though, if you do accept that iterative or agile development is going to result in an evolved, changed, and improved product as you go along, you still need a coherent ‘whole’ to come back to. A system is greater than the sum of it’s parts. The individual pieces, stories, journeys, whatever-else, do take on a different meaning when assembled together. All of this needs to be understood, together with the business knowledge that goes along with it. So, in a away, there is a need for some ‘waterfall-esque’ activity – even if it just providing education to people (and by this I mean for IT AND the business/users – you can’t assume that latter are mystical deities with all the answers, necessarily. I get rather perturbed by the reverence in which the user is held by some agilists).
As a previous contributor wrote: “Some of the problems now faced in Agile have been solved by other communities – but alas, the communities don’t interact too well together”. This is definately true: The IT industry has a poor record of learning from it’s own past. and frankly, doesn’t learn or remember from what has gone before. Other professions do.
All the best,
Mike