I expect this won’t go down well, but I feel it needs to be said.

Firstly, bear with me – I want to start by talking about how I ended up in this industry.

I came out of uni heading nowhere. Meandered into a job as a pensions administrator. I was seriously considering becoming an IFA, not out of passion or ambition, just because I didn’t have any better ideas.

Then I got lucky. Someone I knew started a startup – like Facebook for villages (before Facebook existed). I picked up coding again (I’d played around as a kid).

From there I blagged a job at another startup as an editor, writing articles about shopping. Then I blagged a job at Lycos as a “Web Master”.

Right place, right time. I was lucky. I benefitted from the DotCom boom. I fell on my feet.

I still pinch myself every day.

I think about teachers and nurses – low pay, long hours, no real choice about where or how they work. I think about other well-paid knowledge professions – lawyers, architects – years of eduction, working brutal hours, often in demanding environments.

Most of the places I’ve worked had food and drinks on tap. Ping pong tables. Games machines. I’ve never had to wear a suit. Most places were progressive, and while the industry doesn’t have a great reputation overall, it’s been far more accommodating of people from different backgrounds, genders, and sexual orientations than many others.

After a long bull run – which peaked post-Covid with inflated salaries and over-promotion – things feel like they are changing. Being asked to go back into the office a couple of times a week. You can’t just fall into jobs like you used to.

And GenAI, of course – currently upending the way we work. A paradigm shift far greater than anything I’ve seen in 25 years of my career.

What we had wasn’t normal. It wasn’t standard. It was unusually good.

We weren’t owed any of this.

We just got lucky.

This is an attempt at a relatively non-technical explainer for anyone curious about how today’s AI models actually work – and why some of the same ideas that made them so powerful may now be holding them back.

In 2017, a paper by Vaswani et al., titled “Attention is All You Need”, introduced the Transformer model. It was a genuinely historic paper. There would be no GenAI without it. The “T” in GPT literally stands for Transformer.

Why was it so significant?

“Classical” neural network based AI works a bit like playing Snakes & Ladders – processing one step at a time, building up understanding gradually.

Transformers allow every data point (or token) to connect directly with every other. Suddenly, the board looks more like chess – everything is in view, and relationships are processed in parallel. It’s like putting a massive turbocharger on the network.

But that strength is also its weakness.

“Attention” forces every token to compare itself with every other token. As inputs get longer and the model gets larger, the computational cost doesn’t just increase. It grows quadratically. Double the input, and the work more than doubles.

And throwing more GPUs or more data at the problem doesn’t just give diminishing returns – it can lead to negative returns. This is why, for example, some of the latest “mega-models” like ChatGPT 4.5 perform worse than its predecessor 4.0 in certain cases. Meta is also delaying its new Llama 4 “Behemoth” model – reportedly due to underwhelming performance, despite huge compute investment.

Despite this, much of the current GenAI narrative still focuses on more: more compute, more data centres, more power – and I have to admit, I struggle to understand why.

Footnote: I’m not an AI expert – just someone trying to understand the significance of how we got here, and what the limits might be. Happy to be corrected or pointed to better-informed perspectives.

A constantly updated list of guides and best practices for working with GenAI coding assistants.

These articles provide practical insights into integrating AI tools into your development workflow, covering topics from effective usage strategies to managing risks and maintaining code quality.

Importantly, the authors of all these articles state they are continually updating their content as they learn more and the technology evolves.

• Defra’s AI SDLC Playbook offers a structured approach to safely incorporating AI tools throughout the development process. Still the best resource out there in my opinion.
• Atharva Raykar’s article, AI-assisted coding for teams that can’t get away with vibes explores how AI-assisted coding fits into modern software development, highlighting benefits and challenges
• Chris Parson’s article, Coding with AI: How To Do It Well And What This Means, shares hands-on tips for coding alongside AI, as well as how he uses AI more widely in his working life.

There are some books now available on this topic, but they tend to be out of date by the time they are published due to the fast pace of AI development.