I’ve find 1-2-1s to be hugely valuable, I think it’s the most important thing I do.

Why?

How many times have you been in the situation where you were stuck on a problem and simply by starting to explain it to someone you solved the problem in your head? The point there is talking is a good thing in itself.

1-2-1s are also extremely good at catching problems early, often before you realised they were a problem. Retrospectives are great way of raising and solving issues, but they’re much more about the team than the individual. There are also a lot of things people are not comfortable saying in those kind of situations.

Here’s some simple advice for anyone running 1-2-1s:

1. Get the conversation flowing

2. Listen!

3. It’s not your job to solve their problems, but help them to solve their own

4. Follow up

The next time you meet up (& this is why it’s important you are consistent with who you have your 1-2-1 with) follow up on the things you talked about last time and see if there’s been any improvement.

I did a talk at QCon London 2012 about my experiences with introducing Agile practices at 2 organisations (7digital and BBC Worldwide). It’s now available to watch on InfoQ:

http://www.infoq.com/presentations/Agile-Adoption-is-Fool-s-Gold

I regularly refer to my notes being available including links to many of the things I mention. You can access them here:

https://docs.google.com/document/d/1IN8e3UNvR1oRlvT9b_1gKHNkBE2E6fEx_dVSD3p–6g/edit

I’m a bit of a management junkie as well as a programmer, which is fortunate as I’m now in a position of senior management (although I have to admit I still prefer coding to management). I wanted to recommend some good management reading to people in my management team – particularly regarding organisational strategy and leadership as we’re a small company growing up fast and need to think about these things more seriously now. I was also keen that nothing I recommended was too polemic, software development oriented or, ideally, containing the “A” word (although one ended up doing so).

I asked Twitter and the books below are the ones most recommended or looked most interesting for us. It’s by no means an authoritative list or supposed to be the “best” of anything. I just thought it was interesting enough to be worth sharing more widely.

The Goal

The Goal is a management-oriented novel by Dr. Eliyahu M. Goldratt, a business consultant whose Theory of Constraints has become a model for systems management. It was originally published in 1984 and has since been revised and republished every few years, once in 1992 and again in 2004. This book is usually used in college courses and in the business world for case studies in operations management, with a focus geared towards the Theory of Constraints, bottlenecks and how to alleviate them, and applications of these concepts in real life.^[1] This book is widely used in leading colleges of management to teach students about the importance of strategic capacity planning and constraint management. 

Most recommended by those polled. I’ve also read it and think it’s great.

Managing the Unexpected

The unexpected is often dramatic, as with hurricanes or terrorist attacks. But the unexpected can also come in more subtle forms, such as a small organizational lapse that leads to a major blunder, or an unexamined assumption that costs lives in a crisis. Why are some organizations better able than others to maintain function and structure in the face of unanticipated change?

Authors Karl Weick and Kathleen Sutcliffe answer this question by pointing to high reliability organizations (HROs), such as emergency rooms in hospitals, flight operations of aircraft carriers, and firefighting units, as models to follow. These organizations have developed ways of acting and styles of learning that enable them to manage the unexpected better than other organizations. Thoroughly revised and updated, the second edition of the groundbreaking book Managing the Unexpected uses HROs as a template for any institution that wants to better organize for high reliability.

David Harvey, CEO of Vyclone and industry veteran recommended this because ” (1) it’s full of good stories, (2) it’s backed by _a lot_ of research, (3) no mention of the Agile or Lean words…”

How to Manage and How to Lead

Both recommended for being short, pragmatic and accessible. Thanks Marcin 🙂

Adrenaline Junkies and Template Zombies:Understanding Patterns of Project Behaviour

In Adrenaline Junkies and Template Zombies, the six principal consultants of The Atlantic Systems Guild present the patterns of behavior they most often observe at the dozens of IT firms they transform each year, around the world. 

The result is a quick-read guide to identifying nearly ninety typical scenarios, drawing on a combined one-hundred-and-fifty years of project management experience. Project by project, you’ll improve the accuracy of your hunches and your ability to act on them. 

Any book where DeMarco is involved is essential reading in my opinion. Not heard of this one before. Thanks Captain Crom!

Managing the Design Factory: A Product Developers Tool Kit

This text on product development combines the analytical tools of queuing, information and system theories with the ideas of organization design and management. The author aims to answer such questions as: when should we use a sequential or concurrent process; should there be centralized or decentralized control; and should the organization be based along functional or team lines?

Supposed to be a bit theoretically heavy in places, but worthwhile none the less. I hear Reinertsen’s name referenced regularly and know a couple of people who have been on his course.

Management 3.0

Management 3.0 is a course, a book, and an approach to inspire team members, team leaders, development managers, IT directors, project managers, Agile coaches, and HR managers, who face the challenge of transforming their organizations to an Agile mindset. We do that by providing guidance and practices, and by applying complexity thinking to the craft, art, and science of Agile management.

I’ve read this and really liked it. It’s obviously very “Agile” oriented, but starts by covering a lot of management theory and most things covered in the book could be applied far more widely than just software development.

This article is cross-posted from Blogs From The Geeks, the 7digital development team blog:

Assuming you had consent from up above*, you’d think it’d be a breeze to get an initiative like innovation time off the ground. Surprisingly at 7digital it took us three attempts before we got something to stick and speaking to someone else recently I found they’d had a similar experience. As we’ve had our “innovation time” initiative running successfully for over a year now I think it’s worth sharing how we’ve got it to work.

Someone reminded me of the Robber’s Cave experiment last night. It was quite an amusing study with a serious motive of showing how easily opposing in-groups and group hostilities can form. It also showed how having superordinate goals can counteract this phenomenon.

Basically, if you’re finding yourself in the situation (as, lets face it, we often are) of taking sides and forming prejudice about other teams or groups a la Lord of the Flies you’re succumbing to base, instinctive behaviour which was probably quite useful when we hunted in packs, but has little value in modern society.

Next time you find yourself in this situation, think about what superordinate goals you share with the “opposing group”, so that rather than back slapping each other about how amazing your group is and how weak and feeble they are, you can all work together to get what you want.

As you might know I’m Programme Chair for the SPA Conference. In the past I’ve also done this for XPDay and presented or run sessions at both (and others). I find it a hugely rewarding activity, particularly because it feels like I’m giving back to communities which have helped and inspired me so much in the past. However there are plenty of other reasons why running sessions or presentations at conferences or community events is a good thing:

Free entry
Having a session accepted at SPA means free entry. In some cases like Agile 2012 (submissions close Feb 19th) this also includes travel and hotel expenses.

Use it as an excuse to learn something like a new language or concept
My friend Willem did this at SPA last year – he wanted to learn Puppet so he organised a session about it and went away and learnt it.

Try your ideas out in front of smart people
SPA particularly attracts a wide range of people, most of them very smart and experienced. Running a session around an idea or technology is a great way to exploit their knowledge (people are generally more than happy to do so if it’s an interesting idea).

Improve your self-confidence and communication skills
There’s nothing like standing in front of 15-50 extremely smart people – and being appreciated for doing so – to boost your confidence. Having considered myself deeply inferior to the likes of those who presented at conferences I was tricked into submitting some proposals only to find they all got accepted. Even more amazing was they actually went down really well.

Career
If I see on someone’s CV that they’ve presented sessions at community events they rise up in my estimations greatly and I’m sure that’s the same for pretty much anywhere.

SPA Submissions finish at the end of January. You only need to have the idea formulated now. The conference is in July so you’ve got 6 months to make it sweet. Get it in this by Sunday and you’ll benefit from the feedback process which starts next week.

What are you waiting for?

This article is cross posted from the 7digital Blogs From the Geeks:

I see lots of articles and discussions on how you measure the productivity of a development team. Having worked with cycle time and throughput for a few years now I’ve come to the conclusion that when combined these very simple measurements are a highly effective gauge of productivity.

Cycle Time

We measure cycle time as the count of working days between work starting (“in progress”) and completion (“done” or “live”) and is shown below per item over time. The chart below also shows a trend line (going down, nice) and error bars for the standard deviation:

Ideally you want to see the trendline on your cycle time chart going down, but more importantly you don’t want to see it going up. Shorter cycle times suggests we’re delivering value to the organisation quickly and do not have money unnecessarily tied up in inventory (unreleased code). It also means we’re more predictable and able to respond quickly to change.

To be truly useful (and much more difficult to game) cycle time has to measure the time it takes for things to really be done, not some “potentially shippable” nonsense. It’s also very simple to measure – you just need to track when the work starts and when it ends. We say a piece of work starts when it’s taken off the top of the prioritised queue and done when it’s been released to production (and verified). Sometimes we’ve done the analysis and requirements before hand, sometimes we haven’t (I don’t really care to be honest as we never allow more than 5-10 items in our prioritised queue so it’s not like we’re doing loads of unnecessary up front analysis or consider it a huge time drain).

Throughput

We measure throughput as a count of items completed per month:

Obviously the more work we do (with the same man power) the better (but probably less bugs and more features).

Individually these measurements are useful, but can be misleading if considered in isolation – cycle time going down is great, but if you’re only delivering one piece of work a month it’s not so great. Throughput can go up or down, but could be because you’ve more or less people in your team.

It’s hard to say if productivity can really be measured by just cycle time and throughput and being witness to metrics being gamed in the past (or causing undesirable behaviour) I’m wary of trying to improve them directly. However as a couple of really simple measurements we can track over time they’re a really good guide to support or refute anecdotal evidence as to whether anything we’re doing is having an impact on our ability to deliver. A great example is the two charts above. In around Sept/Oct they show one of our team’s cycle times going down and throughput going up, which was largely a result of changing tack and spending a considerable amount of time improving the build process and removing/fixing flaky tests.

What can you try to increase your team’s throughput and reduce cycle time? What can these measurements tell us about the impact of X happening?

There are of course many other ways to judge the effectiveness of a team and I wouldn’t want to get too caught up with just these ones, but it certainly pleased me greatly to see how the team above’s measurements improved and it certainly corroborated with the mood of the team since they managed to turn things around.

I’m of the mind increasing throughput and reducing cycle time is A Good Thing™. It generally means we’re getting more done, more quickly.

Synopsis

I believe estimation, and the way it’s regularly misused, is at the root of the majority of software project failures. In this article I will try to explain why we’re so bad at it and also why, once you’ve fallen into the trap of trying to do anything but near future estimation, no amount of TDD, Continuous Delivery or *insert your latest favourite practice here* will help. The simple truth is we’ll never be very good at estimation, mostly for reasons outside of our control. However many prominent people in the Agile community still talk about estimation as if it’s a problem we can solve and whilst some of us are fortunate to work in organisations who appreciate the unpredictable nature of software development, most are not.

Some Background

I recently tweeted something which clearly resonated with a lot of people:

It was in response to this article Mike Cohn posted proposing a technique for providing estimates for a full backlog of 300(!) stories when you haven’t yet gone any historical data on the team’s performance. The article does provide caveats and say it isn’t ideal, but I think it’s extremely harmful to even suggest that this is something you could do.

I’ve long had a fascination with estimation and particularly why we’re so bad at it. It started when I was involved in a software project which, whilst in many respects was a success, was also a miserable failure resulting in a nasty witch hunt and blame being put at the foot of those who provided the estimates.

I’ve previously written about the estimation fallacy, that formative experience, and ran my “Dealing with the Estimation Fallacy” session at 2/3 conferences including SPA 2009. There are also plenty more estimation related articles in the archive.

Why we’ll always be crap at estimating

“It always takes longer than you expect, even when you take into account Hofstadter’s Law.”
– Douglas Hofstadter, Godel, Escher, Bach: An Eternal Golden Braid

Since my rocky experiences with estimation and my subsequent research into the area I’ve gone to great lengths to try to improve the way estimations occurs. At my current organisation we collect data from all our teams on how long work items take (cycle time) and have this going back over two years. It’s a veritable mountain of data from which we’re able to ascertain the average and standard deviation for cycle times.

Right?

Wrong.

We were finding that unless we were talking about stuff coming up in the next few weeks – by which point we’d generally done more detailed analysis – we were either at the far end of the estimation range or missing the predicted milestones dates altogether. It was almost always taking longer than we expected. You could argue that it was poor productivity on the behalf of the developers, but the projections were based on their previous performance so it’s a difficult one to argue, especially as our data was also suggesting that the cycle times for work items had actually been going down!

There are two main reasons things were taking longer than expected:

1. Cognitive Bias

From Wikipedia: “Cognitive bias is a general term that is used to describe many observer effects in the human mind, some of which can lead to perceptual distortion, inaccurate judgment, or illogical interpretation”.

In terms of estimation, cognitive bias manifests itself in the form of Optimism Bias: “the demonstrated systematic tendency for people to be overly optimistic about the outcome of planned actions”.

There is also Planning Fallacy: “a tendency for people and organizations to underestimate how long they will need to complete a task, even when they have past experience of similar tasks over-running”.

It’s a huge problem, especially when we’re trying to do long range high level estimation. Our brains are hard-wired, not only to be optimistic about our estimates, but also to tend towards only thinking of the best cases scenarios. There is very little we can do about this apart from be aware it happens.

It’s only when you get round to doing the proper analysis of the work that you start seeing other things you need to do. I remember Dan North once saying in a presentation that estimation is like “measuring the coast of Britain”, the closer you look the more edges you’ll see and the longer it gets.

2. “Known unknowns and unknown unknowns”

Donald Rumsfeld got soundly mocked for his infamous “unknown unknowns” quote, but he was right. With software development, just like any Complex Adaptive System, there are many things you simply cannot plan for. We can at best be aware there will be pieces of work we’ve not accounted for in our estimations (known unknowns), but we’ve got no way of knowing how much work they’ll represent. The best we can do is know they’re there. The unknown unknowns? Well they’re even harder to predict 🙂

Why estimation can be so harmful

Estimates are rarely ever just estimates

There is always a huge amount of pressure to know when things will be done and with good reason. People want to be able to do their jobs well and it’s hard if they can’t plan for when things they’re dependent upon will be ready. This is why an estimate is rarely ever “just” an estimate. People use this information, otherwise they wouldn’t be so keen to know in the first place. You can tell them all you like that they can’t be relied upon, but they will be. As Dan North recently tweeted, “people asking for control or visibility really want certainty. Which doesn’t exist”.

Some of us are fortunate to breath the rarefied air of an enlightened organisation where the unpredictable nature of software development is generally accepted, but lets face it, most are not.

Re-framing the definition of success (or highlighting the failure of estimation)

The most significant impact of providing estimates is that as soon as they’re in the public domain they have a habit of changing the focus of the project, especially when dates start slipping. Invariably a project of any reasonable duration, which started with the honourable objective of solving a business problem, quickly changes focus to when it will be completed.

Well, there’s actually a bigger problem here already. For most organisations the definition of success remains “on time and on budget”, not whether the project has made money, or improved productivity, or brought in new customers. It is often said the vast majority of software projects are considered failures and the Standish Choas Report is usually cited here. The problem is even they define success as meeting cost and time expectations. All this report is highlighting is the continued failure of estimation.

At best providing long range estimates just supports the “on time, on budget” mentality. At worst it takes projects started with the best intentions and drags them back down to this pit of almost inevitable failure and you – as the provider of the estimate – along with it.

The consequences…

All the typical issues dogging software development raise their heads when estimation lets us down. It’s the primary cause of death marches. Corners get cut, internal quality compromised (at the supposed sake of speed), people start focusing more on how the team works, pressure mounts to “work faster”, moral drops and so on – all resulting in a detrimental impact to the original objective (unless it was to be on time and on budget of course, but as we know even then…).

It’s a vicious cycle repeated in organisations all over world, over and over again. It’s my strong belief that the main reason it keeps occurring is because we can’t escape our obsession with estimation.

“But we can’t just not provide estimates”

This is the typical response when I say we should start being brave and just saying no to requests to provide estimates on anything more than 1 or 2 months (at most) work. My answer to that is if there’s as much chance of you coming up with something meaningful by rolling some dice or rubbing the estimate goat then what purpose are you satisfying by doing so?

What’s the least worst situation to be in? Having an uncomfortable situation with your boss now or when – based on your estimates – the project’s half a million over budget and 6 months late?

Conclusion

Some people still seem to want to cling on to the idea that estimation is still a valuable activity and particularly that we are able to provide meaningful long range estimates, but as I’ve explained here it’s a fools errand. My advice to anyone who’s put in the situation of providing long range estimates is to be brave and simply say it’s not something we can do, and then do your best to explain why. It’s against the spirit of Agile software development, but that’s not even the point, mostly it’s just in no one’s best interests – well, unless you’re a software agency who rely on expensive change requests to keep the purse full, but that’s a whole different story.

Epilogue – what do we do instead?

It’s obviously no good to just say “no” and walk away and I’m certainly not advocating that, but it’s also not the point of this article to try and explain what the alternatives are. There are a plethora of options out there, especially in Agile & Lean related literature and from well known Agile and Lean proponents, but crikey, if there’s one thing that everyone within our passionate communities can agree it’s that the best software comes from iterative, feedback driven development, something which is completely at odds with the way estimation is currently being used by most organisations.

In my last post I talked about how self-organisation occurs and how we can try to make the most of it. This time I want to talk more specifically about team size and how I think it impacts self-organisation.

I would argue that the difference between a group of individuals and a team is the ability of the group to adapt to maintain its interests. For example, an amateur football team needs to adapt to the opposition’s tactics to try and win games and stay competitive. If they couldn’t do this effectively they’d probably lose all their games, get relegated and eventually get so fed up they’d disband and spend their weekends playing golf instead.

As we know from my last article, that ability to adapt is when self-organisation occurs and therefore we can say self-organisation is a key component of effective team dynamics.

Too big?

A significant factor in being able to self-organise is for the people who make up a group to be able to communicate & collaborate effectively and form the shared identity and shared goals which determine the group’s boundaries. This is one reason why agile proponents favour co-located teams and open plan offices, but there’s only so much that will help – 100 people working in the same office on the same project are going to struggle to maintain the types of quite intimate relationships which are required for self-organisation to emerge.

Too small?

Most attention seems to be focused on how large a group can become be before it starts to lose its ability to self-organise and work as a team, but I think it’s just as important to consider how small an effective team can be. If we look to sport we very rarely (if ever) see teams of less than 5 people and when we do it’s in sports such as rowing, where the complexity of the interactions between team members is low. When you only have 3 or 4 people in a team (as opposed to 10) the flexibility of the group to adapt to situations is limited by the number of ways the group can organise itself.

For example, in a game of 3-a-side football (soccer) someone is invariably going to have to be responsible for guarding the goal to prevent the opposition just shooting from long range or provide some defence if one of the opposition breaks away whilst your team is attacking. This leaves the other two players little option but to pass it back and forth between each other, making it rather predictable for the opposition and frankly, not much of a spectator sport.

Adding one more person doubles the number of possible connections between the group. Adding two more almost quadruples it. As this is exponential the possible connections increases greatly with each new addition to the group, but this then comes back to how many people you can realistically maintain a relationship with within the boundaries of the group and the context of its environment, so clearly there’s a balance between the two.

Software teams

From what I can see the main objective of working in teams is for the whole to be greater than the sum of its parts. I’m not saying a “team” of 3 people won’t do a good job, but it will find it hard to behave like a team as it will struggle to reach and maintain the behaviours it needs to self-organise well – the limitations of the group become more acute. For example, when someone is on holiday or off sick it’s much harder to adapt to this and if you have junior or inexperienced people in the team you are very limited about how you can share work, as neither of them can work very well on their own or together.

Scrum proponents suggests the most effective size for software teams is between 5-9 people. Jeff Bezos introduced the concept of Two Pizza Teams – team’s are at the best size when comfortably fed by two large pizzas. In fact, doing a general search for optimum or effective team sizes finds that many people have come to a similar conclusion on the effectiveness of team sizes (see here and here)  – that the team only really becomes a team with around 5 members, but starts to degrade when you reach 9.

If a small team (e.g. 2-3 people) is your only option then it’s certainly no disaster, but if your decision was how to best organise a larger number of people into contextual groups then it may be better to aim for somewhere a bit higher as the overhead of managing all the small teams combined with the increased complexity of the communication networks between the teams and the risk of knowledge silos may well outweigh any perceived benefits.