The UK’s AI Security Institute has published the first independent evaluation of Claude Mythos’s cyber capabilities. The headline finding – first AI model to complete a full 32-step simulated network attack – is notable. But there’s a finding buried in the accompanying methodology paper that puts it in a rather different light. On current pricing and reliability, according to my maths, a human expert would do the same job cheaper, faster and more reliably.

What AISI found

On capture-the-flag tasks – common security challenges AISI have been using to test models since 2023 – Mythos sits broadly on the existing trend line. Real improvement, but incremental, and not unique to Mythos. The capability has been building across multiple labs for over a year.

The more significant result is with what AISI call “chained attacks” – where a model has to execute a long sequence of steps across a network to take it over, rather than exploit a single vulnerability in isolation. AISI measured this using their “The Last Ones” simulation: a 32-step corporate network attack spanning initial reconnaissance through to full network takeover, which they estimate a human expert would complete in 14 hours.

Mythos is the first model to complete all 32 steps end to end – though Opus 4.6, Anthropic’s previous model, wasn’t far behind in its best run.

The limitations & takeaways

The model was already inside the network, and the simulated environment had no active security monitoring and no defensive tools. Real networks aren’t like that – at least they shouldn’t be.

For most organisations the biggest threats remain phishing, weak passwords, and unpatched systems. AISI’s own advice in the article reflects this: focus on the basics – patch regularly, enforce access controls, enable logging. More importantly, the most common and successful attacks continue to target humans rather than rely on technical sophistication – as the Co-op, M&S and JLR attacks last year demonstrated.

The trajectory is real and worth taking seriously – but AISI’s findings are more measured than Anthropic’s “watershed moment” framing, and the most important things you can do about it are the same things you should have been doing anyway.

There’s a finding buried in the methodology

AISI published an accompanying academic paper detailing the evaluation methodology and results for models prior to Mythos – including detailed cost and timing data. This is where things get interesting.

According to that paper, the best Opus 4.6 run at 100M tokens cost approximately $80 and took around 10 hours – completing 22 of 32 steps, equivalent to roughly 6 of those 14 human hours. Slower, and less than halfway through in human time equivalent.

Mythos is priced at 5x Opus 4.6 per token. Its best run completed 32 steps versus Opus 4.6’s 22 – but crucially the additional steps fall in the later, harder milestones which are significantly more time and token-intensive. Accounting for both the price differential and likely higher token usage on those harder steps, a rough extrapolation puts a Mythos run at approximately $880.

The variance problem

The paper shows all models have very high variance across runs. Opus 4.6’s best run reached 22 steps, its worst only 11, with an average of 15.6. And the AISI article shows Mythos only completed all 32 steps in 3 of its 10 attempts – a 70% failure rate on full completion.

To expect one successful outcome you’d need 3-4 runs on average – and each run is likely comparable in time to the Opus 4.6 runs.

That’s approximately $2,900-3,500 per successful outcome.

A human expert completing the same range: 14 hours, once, reliably. At $125-190 per hour (UK rates) that’s $1,750-2,625.

So at least today, and according to AISI data, and assuming my maths are roughly correct, an experienced cyber human would be cheaper, more reliable and at least as quick as the most capable AI model currently available.

You may be seeing posts claiming METR’s widely-cited 2025 study has been followed up with new research showing an 18% productivity boost. That’s not what the article says.

METR: We are Changing our Developer Productivity Experiment Design

In 2025, METR found experienced open-source developers using GenAI were 19% slower – and notably, developers themselves thought they were being sped up. They started a new experiment to track how things were changing – but couldn’t complete it. They say the data was too compromised to produce reliable results.

The interesting part is why the study broke down. Developers are now so reliant on AI that they won’t work without it (to be part of the control group). And the nature of how they work when using GenAI has changed too, which undermines e.g. simple time-on-task measurements.

METR believe GenAI coding productivity is improving – but say they can no longer measure it reliably with this study design and are reworking their approach. Personally I don’t see how you can practically design an effective controlled experiment considering everyone uses it now.

Worth noting too that, either way, these kinds of studies are still a narrow window on software delivery – individual task completion by autonomous open-source contributors, not developers working in teams on production codebases with all the organisational complexity that entails.

Multiple studies now suggest AI is genuinely increasing coding velocity – including CircleCI’s recent State of Software Delivery report. But the same report points to a more troubling pattern at the system level: less code reaching production and increasing instability.

My take: teams with strong engineering practices – genuine continuous delivery, high code quality, solid test coverage – appear to be realising real benefits from GenAI, and there’s data to support that. The problem is those teams represent a small fraction of the industry. For everyone else, higher coding velocity is likely resulting in a negative impact downstream.

CircleCI’s 2026 State of Software Delivery report has two findings that are already travelling: AI is meaningfully boosting software delivery, but only 1 in 20 teams are capturing that benefit. Both claims are more uncertain than the report suggests, for different reasons.

What the report is measuring

The report’s primary metric is “throughput” – the number of times a CI pipeline runs per day. A CI pipeline is the automated process teams use to build, test and progress code toward production. It is not production deployments, it is not features shipped. The report is using pipeline execution data to infer things about software delivery. That’s not unreasonable – it’s real data – but it’s worth understanding what’s actually being measured before drawing conclusions.

The headline numbers

The report measures throughput on both feature branches and main branches and aggregates both into its headline figures. Throughput as a metric on feature branches is effectively meaningless. Throughput should be an end-to-end metric – feature branches aren’t end-to-end, they get merged to main. The only meaningful “throughput” measure is against the main branch. What the feature branch data actually shows is a lot more code being written, but not much more reaching production.

• Average teams are up 4% on the aggregated figure, but main branch throughput is down 7%
• The top 10% of teams show aggregated throughput up nearly 50%, main branch essentially flat
• For 95% of teams, AI is generating more work in progress that isn’t shipping

The success rate of main branch builds compounds this further. It has fallen to 70.8%, its lowest in over five years – 30% of attempts to merge code for production are now failing.

The 1 in 20 claim

The report identifies the top 5% of teams as the only group seeing meaningful main branch throughput growth – 26% – and uses this to argue that some teams have cracked the AI delivery problem.

But the summary data for that group is odd. Their average CI pipeline duration is 6 seconds. A pipeline doing anything meaningful – compiling, running tests, scanning – it’s hard to think of a single CI step that legitimately completes in 6 seconds. Perhaps it is an error in the report. There’s also data that may be skewing the findings more broadly – one team apparently running 130,000 CircleCI workflows a day would have an outsized effect on any aggregate figures.

What to take from it

The integration bottleneck finding is credible. If you’re generating code faster than your team can review and integrate it (safely), that’s a genuine problem this data is consistent with.

The “1 in 20 teams have cracked it” conclusion is less solid than it appears. Not to say that some aren’t getting benefit I believe there are, however the data here for the teams making that case doesn’t add up clearly enough to draw confident lessons from.

According to the FT, demand for software engineers is rising again, and in relative terms is outperforming the wider jobs market. That’s the headline most people will take away.

But the more important detail is that the growth is concentrated in more experienced roles, while entry-level hiring remains weak.

That broadly lines up with a concern I wrote about recently for Ada National College for Digital Skills: GenAI is amplifying the already existing skills gap in software engineering.

Genuinely strong, experienced engineers are already a relatively small pool. If the FT data is right, demand is increasingly being concentrated into that already constrained part of the market.

That means a supply squeeze. Salary inflation, harder hiring, slower execution, and more organisations unable to deliver on the strategy and goals they have set.

Further, as we saw in the pandemic hiring boom, shortage pressure creates title inflation and level distortion, such as mid-level people taking senior or lead roles at new companies. The result is organisations paying more for talent that is, on average, less experienced than the role implies, while carrying more execution risk and less capability depth than the org chart suggests.

If companies keep optimising away that layer, it is not just bad for the industry overall, it is economically short-sighted for the individual organisation. It increases future dependence on an already constrained and expensive part of the market, while missing the real value of junior engineers, not just what they contribute now, but the capability they become.

Parkinson’s Law is mainly interpreted as “people fill the time available”. That line was really only a hook. The original paper is really about how organisations create extra work and headcount for themselves, regardless of whether there is more genuinely useful work to do.

As organisations grow, work often grows around the work itself – layers, handoffs, approvals, reporting, internal dependency. More work about work.

That’s why adding more people, more process, more tooling (more AI?) so often fails to produce the productivity gains people expect.

Adding people can be beneficial to a point, but often far less than people might expect – and beyond a point, things usually get slower and more expensive, not faster. Doug Putnam’s analysis of hundreds of software projects found small teams were generally best, with 3-5 often the economic sweet spot, and larger teams quickly suffering diseconomies of scale.

The countermeasures to keep things as small, clear and contained as possible:

• clear direction and priorities
• clear accountability, ownership and decision boundaries
• ruthless prioritisation
• less work in flight – a culture of finishing

This is why I keep coming back to the same principle:

Fewer, better people doing less, better will usually get more done.

Not because small is always magically better. But because complexity compounds, and every extra person, process, dependency and priority adds drag.

Footnote: Probably the grossest misinterpretation of Parkinson’s Law is the idea that giving individuals less time or tighter deadlines will somehow make work happen faster. In practice, that will often just compresses time without removing the underlying drag.

This is a cross post of an article I wrote for Ada, the National College for Digital Skills

All the available evidence suggests that GenAI-assisted coding is most powerful in the hands of highly experienced software engineers, while having neutral or even negative effects for less experienced ones.

It’s easy to see how this may be interpreted inside organisations. If experienced engineers can be made significantly more productive with GenAI, then it can appear rational to rely more heavily on that group. Smaller teams of senior engineers, supported by GenAI, with fewer junior or entry-level roles, can look like an attractive opportunity.

However, genuinely good, experienced software engineers are already scarce across the industry. It is difficult to put a precise figure on this, but studies on the impact of AI in software engineering suggest that only a minority of engineers and teams currently have the skills and experience needed to realise sustained benefits from GenAI-assisted software development, likely somewhere in the region of 10–30%.

That raises an obvious question about where the next generation of experienced engineers will come from.

The risk is not only that organisations hire fewer junior engineers, but that even when they do, the conditions for learning are compromised. A recent study by Anthropic, one of the organisations at the frontier of GenAI and the creators of the Claude models and tools such as Claude Code, found that developers using AI assistance completed coding tasks slightly faster but demonstrated significantly weaker understanding afterwards. When the tool was allowed to do too much of the thinking, learning suffered.

More generally, as organisations offload more work to these tools, team dynamics begin to shift. Fewer questions are asked. Explanation gives way to acceptance. Output rises, but shared understanding does not.

This all happens quietly. Everything looks efficient, right up until it’s not.

Despite repeated waves of tooling, the core skills that define good software engineering have remained remarkably stable. Effective problem-solving, system-level thinking, feedback, shared understanding, automated testing and iterative change have been recognised as good practice for decades.

Learning is not just an individual concern. Software development is a learning activity at every level. Teams learn about users, systems, risks, and constraints through the work itself.

What has also remained true, despite this being well understood, is that only a minority of the industry consistently applies them. The evidence also increasingly suggests these practices are becoming even more relevant in the age of GenAI.

GenAI accelerates this dynamic. Without deliberate effort, it can speed up delivery while quietly weakening an organisation’s ability to create and sustain expertise. When organisations optimise purely for short-term efficiency, learning is often the first thing to erode. When learning slows, capability follows.

The organisations that will thrive in the GenAI era will not be the ones that simply adopt the tools. They will be the ones that treat learning as core to how they operate.

That includes investing in early-career development, creating environments where experience is accumulated rather than bypassed, and recognising that effective software development has always depended on people who can exercise judgement, reason about systems, and learn continuously, not just produce output.

GenAI feels like another turning point for software development. It’s really just the latest moment in a long, repeating pattern of partial revelation and broad avoidance of how creating software needs to be approached.

Early limits of software development

The “software crisis” of the 1960s was the first time this became apparent – there was a sudden leap in computing power, huge optimism, and a belief that software development would simply scale to match it. It didn’t. Projects ran late and over budget, systems failed in production, and codebases became unmaintainable.

Systems had become too complex for the informal development practices at the time. The response was software engineering as a discipline, the industry recognising that writing software was not just typing instructions but coordinating human understanding. The answer was more structure, more process, more formality. The underlying assumption, though, was that software development could be treated like other branches of engineering: plan carefully, specify up front, then execute

When big up front planning began to struggle to keep up

By the 1990s that model was starting to crack. Several things had shifted. Programming languages and tooling had improved. IDEs, compilers and version control reduced friction. Hardware and compute continued to become cheaper and more powerful. But most importantly, the internet removed distribution friction. You no longer had to ship physical media or install systems on site. Software changes could reach users almost instantly. The cost of change dropped sharply.

Agile emerged in that gap. Not because people suddenly liked stand ups and sticky notes, but because fast feedback through working software was proving to be more effective than detailed upfront plans.

The problem was that the underlying assumption that software was a construction problem never really went away. Most organisations copied the rituals and missed the point. Agile in practice largely became process rather than philosophy. 

Software development in the GenAI era

And now GenAI-assisted coding. GenAI has emerged from the continued trend of compute becoming more powerful and cheaper, combined with the sheer volume of data now available as a result of the internet era. The underlying concepts of machine learning have existed for decades, but only recently have the economics and data volumes made it practical.

As a result, the distance from intent to working artefact has shrunk again. You can explore ideas, generate alternatives, test assumptions and see consequences in hours rather than days.

And yet, software engineering is still, to this day, largely treated as a construction problem. Most teams operate as feature factories, fed tickets from a backlog, optimising for “velocity” rather than fast feedback and being able to quickly adapt and respond to new information. Work is broken into tasks, handed off between roles, and judged by output rather than learning.

What building software is really about

At its core, software exists to solve one problem: managing the flow of information. Information is not static. It is provisional, contextual, and constantly changing. Every use of a system creates new information that should influence what happens next. Many of its most important inputs only appear once it is in use.

Software is therefore less a construction problem and more an ongoing conversation between users, systems and decisions. Its quality lies in the strength of its feedback loops and how effectively it enables learning and adjustment as conditions change.

Technology advances keep reducing the cost of building software, but organisations keep failing to adapt how they approach it. Each era exposes that mismatch more visibly, without resolving it.

The organisations that will thrive in this era will be the ones that optimise for learning over output, feedback over plans, and understanding over speed.

Hopefully this time it will be different.

I’ve been trying to work out why successive advances in GenAI models don’t feel particularly different to me, even as others react with genuine excitement.

I use these tools constantly and have done since ChatGPT4 was released nearly 3 years ago. I couldn’t imagine a world without them. In that sense, they already feel as transformative as the web. I’ve been thinking perhaps how its once they become ambient, the magic fades. You get used to them and stop noticing improvements. But the more I’ve thought about it, the more I think there are deeper structural reasons why the experience has plateaued, for me at least.

The lossy interface

All meaningful work starts in a physical, social, constraint-filled environment. We reason with space, time, bodies, artefacts, relationships, incentives, and history. Much of this understanding is tacit. We sense it before we can explain it.

To involve a computer, that reality has to be translated into symbols. Text, files, data models, diagrams, prompts. Every translation step compresses context and/or throws information away. There is loss from brain to keyboard. Loss from keyboard to prompt. And loss again when the output comes back and has to be interpreted.

GenAI only ever sees what makes it across that boundary. It reasons over compressed representations of reality that humans have already filtered, simplified, and distorted.

Better models reduce friction within that interface, but they don’t change its dimensionality. In that respect it doesn’t really matter how “smart” the models get, or how well they do on the latest benchmarks. The boundary stays the same.

Because of that, GenAI works best where the world is already well-represented in digital form. As soon as outcomes depend on things outside its boundary, its usefulness drops sharply.

That is why GenAI helps with slices of work, not whole systems. It is powerful, but fundamentally bounded.

Some real world examples:

• In software development, generating code hasn’t been the main bottleneck since we moved away from punch cards. The far bigger constraints are understanding the problem, communicating with stakeholders, working effectively with other people, designing the system, managing risks and trade-offs, and operating systems in complex social environments over time.
• In healthcare, GenAI can assist with diagnosis or documentation, but outcomes are dominated by staff, facilities, funding, and coordination across complex human systems. Better reasoning does not create more nurses or hospital beds.

In both cases, GenAI accelerates parts of the work without shifting the underlying constraint.

Faster horses, not trains

In that respect, GenAI feels like faster horses rather than trains. It makes us more effective at things we were already doing, writing, code, analysis, planning, and sense-making, but operates on only parts, thin slices of systems.

Trains didn’t just make transport faster. They removed a hard upper bound on the movement of people and goods. Once that constraint moved, everything else reorganised around it. Supply chains, labour markets, cities, timekeeping, and even how people understood distance and work all changed. Railways were not just a tool inside the system, they became the system.

GenAI doesn’t yet do that. It works through a narrow, virtual interface and plugs into existing workflows. But as often as not the real systematic constraints lie elsewhere.

What actually changed the world

A recent conversation reminded me of Vaclav Smil’s How the World Really Works, which I read last year.

Smil highlights that modern civilisation rests on a small number of physical pillars: energy, food production (especially nitrogen), materials like steel and cement, and transport. Changes in these pillars are what led to the biggest transformations in human life. Information technology barely registers at that level in his analysis. He doesn’t deny its importance, but treats it as secondary, an optimiser of systems whose limits are set elsewhere.

Through that lens, GenAI doesn’t (yet) register as a civilisation-shaping force. It doesn’t produce energy, grow food, create new materials, or move mass. It operates almost entirely above those pillars, improving coordination, design, and decision-making around systems whose hard limits are set elsewhere.

That doesn’t make it trivial. But it explains why, so far, it looks closer to previous waves of information technology than to steam or electricity. It optimises within existing constraints rather than breaking them.

The big if

Smil’s framing doesn’t say GenAI cannot matter at an industrial scale. It says where it would have to show up. GenAI becomes civilisation-shaping only if it materially accelerates breakthroughs in those physical pillars – things that change what the world can physically sustain.

This is where “superintelligence” comes in. If GenAI can explore hypothesis spaces humans cannot, design and run experiments, or compress decades of scientific iteration into years, resulting in major scientific breakthroughs, it moves from optimising within constraints to changing them.

This is also where my own doubts sit. Many think just scaling what we have now will get us there. For those that don’t believe that, but are still optimistic about AI’s potential, they turn to world models, embodiment, or agents that can act in the real world. There are sketches and hopes for how this may happen, but as yet, not much more than that.

So while superintelligence is the path by which AI could plausibly become industrial-scale transformative, it’s a long and uncertain one.

What kind of change are we talking about?

If you mean web-scale change, then GenAI is already there. But if we mean the kind of change associated with the industrial revolution (as it’s often compared to) – longer lives, better health, radically different working conditions, step changes in material living standards, then what we have today does not qualify. Historically, those shifts followed from breaking physical constraints, not from better information or reasoning alone.

For me, and why I’m not really feeling successive model improvements, it isn’t that GenAI lacks value. It’s that those improvements don’t change the shape of what’s possible. They operate within the same narrow, lossy interface, so they barely register in practical terms. GenAI still adds value, and already feels web-scale transformative. But until that boundary moves, or something else breaks the underlying constraints, they don’t feel like steps toward an industrial-revolution-scale shift.

The crude clickbait narrative is that AI means job cuts, replacing roles. But when I look at how AI is actually being used in real organisations, it seems more likely it’ll be more effective at expanding capacity rather than reduce headcount. Many organisations may end up doing more with the same long before they can credibly do the same (or more) with less.

This thought started for me with an observation – AI is not substituting whole roles, we’re getting micro-specialists that can do slices of work. In software you see agents for tests, code review, planning. Other sectors look much the same. Legal teams using AI for drafting. Sales teams for outreach. Finance for reconciliation. Tools handling tasks, not outcomes, and someone still has to stitch the pieces together.

There are (at least) three forces I can think of that matter when asking whether organisations will genuinely be able do more with less:

1. How automatable the work already is.
Where the work is rules based, high volume, and low variation, AI may replace labour in the same way classic automation has. Think claims processing, simple customer support, structured back office workflows. These functions already lived close to the automation frontier. AI just expands the frontier a bit.
This will reduce headcount, but mostly in places where headcount has been under pressure for decades anyway.

2. How much the organisation can absorb increased output.
Most professional work is not constrained by how fast someone types or drafts. It is constrained by coordination, sequencing, ambiguity, stakeholder alignment, and quality. Software is a good example. So is legal, consulting, product, sales. If you cut the number of lawyers because drafting is faster, you will simply overload the remaining lawyers with negotiation, risk, and client work.

3. The cost and consequences of mistakes.
In many industries, the limiting factor is not productivity, but risk. Healthcare, aviation, finance, law. Increased throughput also increases the risk surface area. If AI increases the probability or cost of an error, you cannot shrink the team. You often need more human oversight, not less.

If you put these together, the more likely outcome is is this:

• Some operational functions will shrink, but these were already at risk of automation.
• Most knowledge work will shift toward more with the same, not less.
• Some domains will accidentally create more with more, because oversight and correction absorb the gains.