How do Generative AI tools impact software developer productivity and code quality? A recent large-scale study – one of the most empirically robust I’ve seen – tackled this question by analysing 218k+ developers across 880m+ commits (with control groups and everything).
The results? A modest but consistent 4% productivity boost without sacrificing code quality.
Other key findings:
– Moderate GenAI users emerged as the highest overall performers (i.e. less is more)
– Only 1% of developers committed GenAI-authored code without significant rework.
Itโs a nice 4% gain right? Not exactly game-changing. but worth it if licence costs stay reasonable. Enough to reduce a team of 100 to 96 engineers (or boost output by 4%)?
Not that simple – the study is somewhat flawed because it’s measuring effort not outcomes and individuals and not teams. The latest DORA Report (the gold standard when it comes to measuring high performing tech teams) found AI Tools marginally improving individual productivity, but having a NEGATIVE impact on overall team productivity.
As Giovanni Asproni said to me on Bluesky:
“I think that better product management, with a decent strategy, and proper prioritisation and planning will increase output far more than 4% in most organisations. And it may actually help saving energy and CO2 emissions.”
Author Archives: Rob
Poll of engineers using AI assistants with production code

I ran a LinkedIn poll asking software developers about their experiences using AI coding assistants with production code. The results were interesting ๐
Why production code? Developers spend most of their time writing it, and AI tools can handle non-critical activities like proof-of-concepts, solo projects, and quick experiments fairly well.
I received 50 responses – not a huge sample, but interesting trends emerged nonetheless. Also, LinkedIn polls arenโt anonymous (which I didnโt know!), so I could see who voted. It allowed me to exclude non-developers and gauge experience levels from LinkedIn profiles and add it to the data.
You can see the results broken down by answer and experience level in the chart attached.
What stood out to me?
– Only one respondent – an apprentice engineer – said they โautomate most things.โ
– 74% found AI assistants helpful, but 26% felt theyโre more trouble than theyโre worth.
– All who said AI assistants โget in the wayโ (24%) had 8+ years of experience.
– 80% of respondents had 8+ years’ experience (I know a few of them, and othersโ profiles suggest theyโre legitimately experienced engineers).
My thoughts? Unless they improve significantly:
1) Theyโre unlikely to shift productivity dramatically. From my experience, โtaskโ work takes about 10-30% of a developerโs coding time. So, while helpful, theyโre not game-changing.
2) That a significant portion – especially experienced engineers – find these tools intrusive is concerning.
3) Having heard stories of AI tools introducing subtle bugs, I was surprised not to see more votes for โconsidered harmful.โ
4) The idea that AI tools might replace software engineers – or that even “conversational programming” is anywhere close – still feels a very long way off.
5) I worry about an apprentice software engineer getting an AI tool to do most of their work. Partly because more experienced developers are a lot more cautious and must have a reason for this, but mainly because they won’t be learning much by doing so.
Creating an Effective Recruitment Process
Hiring is one of the most impactful decisions for any organisation. The wrong person can badly impact the culture and performance of the organisation, absorb time in performance management and be a significant distraction.
Informal recruitment processes are leaving things down to chance.
Inefficient processes consume time and risk losing top candidates who move quickly.
This is why fixing the recruitment process is typically one of my first priorities when working with organisations.
Having interviewed thousands and hired hundreds over the past two decades, here are my top tips:
All candidates should have a positive experience
Whether or not a candidate gets the role, they should leave with a positive impression. A poor experience not only harms your brand but can dissuade other candidates from applying.
- Being warm, friendly, and engaging throughout the process costs nothing, and it doesnโt prevent you from asking tough questions.
- Give constructive feedback to unsuccessful candidates so they know where they can improve.
Short Lead Times
Itโs all too common for the time from CV received to decision made to be a very lengthy process, often taking months.
- Aim for a two week turnaround from CV received to decision made.
- Much of the following advice is for how you can achieve this ๐
Start with a brief screening call
To make sure a candidate is a basic fit, get eligibility information and to introduce them to your company, and the recruitment process.
- If youโre working with a recruiter, ideally empower them to handle this stage (if properly briefed).
- Itโs important this conversation isnโt just a box-ticking exercise. It sets the tone for the rest of the process, so make sure itโs welcoming and informative.
Pre-booked interview slots
So much time is lost in calendar tennis for scheduling interviews. Avoid this by agreeing, and putting in place, pre-booked interview slots upfront in interviewersโ calendars. Whoever is engaging with candidates can then offer them any of the available pre-booked slots. If they canโt do any of them you can always compromise and try and find other times, but make this route one.
Schedule all stages up front
Again, this saves time for both the candidate and your organisation, and gives the candidates clarity on timelines. If youโre doing multiple stages, thereโs no reason why you canโt cancel a further stage if you donโt consider itโs worth proceeding with the candidate, just let them know up-front that you may do this.
Do as many stages as you can in one go
Consolidating interview stages reduces cycle times significantly. While you may end up spending more time with candidates who may not progress, youโll save more overall by avoiding the delays of additional scheduling and decision-making between stages.
If youโre concerned about wasted effort, you can always tell the candidates up front that you may choose to finish the interview early if itโs clear to you itโs not progressing well.
Structured interviews
Every interview should be structured, with the same scripted questions for all candidates and scoring indicators for responses.
This reduces bias, ensures consistency, and simplifies decision-making. You never use scoring indicators as a pass or fail, they just help inform your decision.
I’ve developed interview templates where each stage has its questions and scoring indicators on the same form.
Two interviewers in each stage
This has several benefits:
- One can focus on the candidate while the other takes notes.
- It avoids leaving decisions to a single viewpoint.
- It provides an opportunity to train future interview leads.
- It allows candidates to meet more team members.
The second interviewer doesnโt have to be sitting there quietly, they can take part in asking follow up questions and responding to candidate questions.
The interview forms I mentioned previously? Use them as a shared document in e.g. Google or OneDrive and interviewers can see the notes as theyโre typed, and make their own comments and observations.
Same-day candidate debrief
Debrief as quickly as possible after each stage or at the end of all interviews. The structured format, shared notes, and scoring indicators will help you avoid lengthy debate, maintain consistency and make timely decisions.
Who wants an AI toothbrush?

AI toothbrush anyone? Yes this is real, but I don’t want to talk about AI. I want to talk about technology having it’s place. Not everything is best solved by technology.
Iโm a hardcore techie, but when it comes to personal hygiene, I couldn’t be more lo-fi. After years of trial and error, Iโve settled on a wooden toothbrush, basic Colgate toothpaste, and the same cheap Dove soap for both washing and shaving (with a basic razor and shaving brush).
A ~ยฃ100 “AI-powered” toothbrush solves no problems for me – or anyone, for that matter.
It’s why I challenge early stage startups if they really need to build something yet. It’s why I’m an advocate for practices like Value Stream Mapping and Service Design, and how often the desired outcome is best solved *without* needing to chuck technology at the problem.
It’s why I’m obsessed with process optimisation and systems thinking.
It’s why I loved the recent episode of “The Digital Human” on the potential for AI in the NHS (BBC Radio 4, will share link in comments), where the the eminently sensible Jessica Rose Morley, PhD pointed out thereโs little point in getting carried away when, in reality, it’s still common for three wards to share one outdated PC on wheels, and the data is in such a mess.
Technology is expensive, it comes at a cost. Only when you’ve identified the user need and/or optimised existing processes is it worth investing in.
Don’t tolerate brilliant jerks
I believe it was Reed Hastings, CEO of Netflix who coined the phrase “brilliant jerk.”
Common in tech orgs – smart, technically gifted, and highly productive. They’re seen as the people to go to for solving big problems, and they often save the day in a crisis.
But boy do they ruffle some feathers getting there.
As a leader you can spend a lot of time cleaning up after the mess they create. But you put up with them coz they get stuff done.
“Jerk” is a big bucket of behaviours of course. They can often dominate conversations, dismiss othersโ opinions, and arenโt shy about pointing out whatโs wrong with the org, proudly โtelling it like they see it,” but they’re typically the last to offer any suggestions.
At one organisation I worked, we had a toxic “them and us” culture between Sales and Dev. Interestingly, when one particular person left, it pretty much evaporated.
And don’t forget, “Your culture is defined by the worst behaviour you tolerate”
How to Plan Effectively in the Face of Uncertainty
In a recent post, I explained why weโre inherently bad at estimating, which is a major reason software projects often run late. But that doesnโt mean we canโt plan ahead for the longer term and manage expectations. Here are some techniques Iโve found effective for longer-term planning, even in the face of uncertainty:
Provide ranged forecasts ๐ฆ, not fixed estimates
Over any reasonable period of time, providing likely delivery dates as a range rather than a fixed date is better. Doing so embraces the inherent variability and help stakeholders better appreciate uncertainty. Giving a fixed date, even when you all know itโs just an estimate, sets false expectations.
Don’t just estimate time โฑ, estimate confidence ๐ช
Along with estimating how long a task might take, assess how confident you are in that estimate. Is it something weโve done before and know well? High confidence. Is it new, complex, or something weโve never tackled before? Medium or Low confidence.Multiply time by confidence. For example, a โSmallโ task (e.g. 1-3 days) with “Low” confidence can be re-forecast as 1-7 days.
Longer term planning as risk management ๐จ
Effective planning isnโt just about setting expectations on delivery, but managing risks over the project lifecycle. All those estimates which have come in as Large or with Medium-Low confidence? Those are your biggest risks and represent the most uncertainty. Identify risks early, identify potential mitigations, highlight those risks to stakeholders.
Account for Optimism Bias ๐
Whilst you canโt completely mitigate against this, there are things you can do to be a bit less vulnerable. Involve the entire team in estimates, account for full end to end delivery (not just developer time) and factor in holidays, sickness and other factors that could impact delivery.Also, those ranged forecasts which provide a cumulative lowest range for all the work? Highly unlikely due to optimism bias! Itโs probably best not to present the most optimistic forecast for this reason.
No silver bullet, but a better approach
These practices wonโt eliminate uncertainty or guarantee perfect outcomes – there’s no silver bullet in longer term planning. However, Iโve found they help organisations plan more realistically, reducing the stress and frustration that often come with missed deadlines, and enabling more effective, adaptable strategies in the face of uncertainty.
Deal with performance issues quickly
Addressing performance issues is something many shy away from, often putting it off or hoping it will go away.
The sooner you do so, the more likely there’ll be a positive outcome for very everyone.
The longer you wait, the more difficult it becomes.
If itโs clear things aren’t working out then you need to act decisively. Don’t let it drag on.
Itโs not only about the individual; it’s letting the whole team down and sets the standard for the level of performance considered acceptable.
Typing is not the bottleneck

โ๐๐ฆ ๐ฏ๐ฆ๐ฆ๐ฅ ๐ฐ๐ถ๐ณ ๐ฅ๐ฆ๐ท๐ฆ๐ญ๐ฐ๐ฑ๐ฆ๐ณ๐ด ๐ต๐ฐ ๐ฃ๐ฆ ๐ฃ๐ถ๐ด๐บ ๐ค๐ฐ๐ฅ๐ช๐ฏ๐จโ
I still regularly come across this mindset. Not with any ill-intention. Developers arenโt cheap, so naturally you want them to be productive.
Whenever I do, Iโm reminded of this meme (image created by Sebastian Hermida).
So this is a reminder that with software development…
๐ง๐๐ฝ๐ถ๐ป๐ด ๐ถ๐ ๐ป๐ผ๐ ๐๐ต๐ฒ ๐ฏ๐ผ๐๐๐น๐ฒ๐ป๐ฒ๐ฐ๐ธ ๐ตโจ
Development is about a lot more than just being heads down hammering out code – itโs about problem-solving, understanding user needs, designing scalable solutions, and ensuring that whatโs built is truly valuable. Rushing to code without seeing the bigger picture leads to missed requirements & re-work, technical debt and buggy software.
It actually slows things down more than it speeds them up ๐ ๐ง
Developers should be involved in the full end-to-end delivery process, not just the coding bit:
๐ Understanding what we need to deliver, and why
๐งช Quality Assurance (you mean you didnโt test it before you threw it over the wall to QA?)
๐ Shipping! Getting things out to customers before picking up new work
๐ค Collaborating with each other and other disciplines
Collaboration is a key one. When developers pair with each other or folks from other disciplines, it speeds up the process by reducing feedback loops and capturing issues early, and fosters shared understanding and ultimately, better solutions.
Having developers engaged more widely might seem counterintuitive if youโre focused on keeping them โbusyโ with coding, but itโs the best way to get the most out of their time.
The estimation trap: why software projects miss deadlines
Why are so many software projects late? Estimation (and how we use it) is often at the heart of it.
Humans are naturally optimistic. We overestimate how well things will go and underestimate the time required. It’s a well documented phenomenon:
๐คย Optimism Bias: Our tendency to believe things will go better than they likely will.
๐๏ธ Planning Fallacy: Underestimating time required, even when past experience tells us otherwise.
Secondly, even with careful planning, we can only account for what we know. As well as software development being inherently complex, as time progresses, unexpected factors always emerge: shifting priorities, new challenges, team changes โ the “known unknowns” as Donald Rumsfeld infamously put it.
๐๐ฎ๐ป ๐๐ผ๐ ๐ด๐ฒ๐ ๐ฏ๐ฒ๐๐๐ฒ๐ฟ ๐ฎ๐ ๐ฒ๐๐๐ถ๐บ๐ฎ๐๐ถ๐ป๐ด?
Yes, we can avoid common pitfalls and improve our estimation practices. But they are still likely to be optimistic because of the reasons above.
“It always takes longer than you expect, even when you take into account Hofstadter’s Law.”
– Hofstadter’s Law
๐๐ผ๐ฒ๐ ๐๐ด๐ถ๐น๐ฒ ๐๐ผ๐น๐๐ฒ ๐๐ต๐ถ๐?
Agile embraces uncertainty, such as breaking down work into the smallest possible pieces and delivering value early and often. But it doesnโt eliminate the challenge of longer-term planning.
๐๐ผ๐ปโ๐ ๐ฏ๐ผ๐๐ต๐ฒ๐ฟ ๐ฝ๐น๐ฎ๐ป๐ป๐ถ๐ป๐ด ๐๐ต๐ฒ๐ป?
Not at all. Organisations need to plan. Without long-term planning, you canโt set realistic budgets, allocate resources & people, or prioritise efforts. The trick is learning how to plan effectively, even in the face of uncertainty.
In my next post, I’ll share how Iโve approached longer-term planning successfully.
Elavate your change agents
If you’re looking to drive change in your organisation, ๐ฒ๐น๐ฒ๐๐ฎ๐๐ฒ ๐๐ผ๐๐ฟ ๐ฐ๐ต๐ฎ๐ป๐ด๐ฒ ๐ฎ๐ด๐ฒ๐ป๐๐:
๐ Identify them โ look for the people who are most on board with the changes you’re trying to make, the most proactive and unafraid to challenge the status quo.
๐ช Empower them โ give them a seat at the table, involve them in initiatives, elevate their voices. Don’t be afraid to break hierarchy in doing so.
๐ก Support them – protect their time, don’t overload them.
The new leaders in your organisation may already be there