How do Generative AI tools impact software developer productivity and code quality? A recent large-scale study – one of the most empirically robust I’ve seen – tackled this question by analysing 218k+ developers across 880m+ commits (with control groups and everything).

The results? A modest but consistent 4% productivity boost without sacrificing code quality.

Other key findings:

– Moderate GenAI users emerged as the highest overall performers (i.e. less is more)
– Only 1% of developers committed GenAI-authored code without significant rework.

It’s a nice 4% gain right? Not exactly game-changing. but worth it if licence costs stay reasonable. Enough to reduce a team of 100 to 96 engineers (or boost output by 4%)?

Not that simple – the study is somewhat flawed because it’s measuring effort not outcomes and individuals and not teams. The latest DORA Report (the gold standard when it comes to measuring high performing tech teams) found AI Tools marginally improving individual productivity, but having a NEGATIVE impact on overall team productivity.

As Giovanni Asproni said to me on Bluesky:

“I think that better product management, with a decent strategy, and proper prioritisation and planning will increase output far more than 4% in most organisations. And it may actually help saving energy and CO2 emissions.”