One of the common assumptions underpinning nearly every conversation I hear about AI governance is that there will always be a “Human in the Loop.”
The phrase has become almost a mantra. If AI generates code, have a developer review it. If it writes documentation, have someone proofread it. If it makes a recommendation, have a human approve the decision before it moves forward.
For now, that seems reasonable. I’d even argue it’s largely a good thing.
The problem is that it assumes the rate of human review can keep pace with the rate of machine generation. I don’t think that’s going to be true for much longer.
We’re rapidly approaching a world where AI can produce software, infrastructure changes, documentation, analyses, customer communications, security findings, design iterations, and operational decisions faster than organizations can meaningfully evaluate them. Not just a little faster, but orders of magnitude faster.
At that point, “Human Oversight” stops being a governance strategy and starts becoming organizational theatre.
A manager clicking Approve.
A developer skimming a pull request containing thousands of AI-generated lines.
An architect signing off on more designs than anyone could realistically review.
A security engineer acknowledging hundreds of findings because there simply aren’t enough hours in the day.
Entire organizations producing documents faster than anyone can reasonably absorb them. Documentation slowly shifts from being a way to build shared understanding into evidence that communication happened, whether anyone actually read it or not.
The process is still there. Whether it’s providing meaningful control is another question.
AI has dramatically reduced the cost of producing work. It has done almost nothing to reduce the cost of paying attention to that work.
It reminds me of trying to work out how fast a car is travelling by looking at wheel RPM. That works perfectly until the tires lose traction.
The wheels keep spinning. The numbers keep changing. They’re just no longer telling you anything useful about what’s actually happening.
I think we’re creating the organizational equivalent.
We’ll continue measuring reviews completed, approvals granted, tickets closed, pull requests merged, CABs attended, and compliance checkboxes checked. Our dashboards will happily report that governance is working exactly as designed.
Meanwhile, the work itself is moving at a pace that makes meaningful oversight increasingly difficult.
The metrics aren’t wrong. They’re faithfully measuring a process that’s slowly becoming detached from reality.
To me, this isn’t really an AI problem. It’s a systems problem.
Most of our management structures were designed around the assumption that valuable work happens at roughly human speed. Escalation paths, approval chains, code reviews, audit processes, risk committees, and Change Advisory Boards are all built on the idea that people can pay attention to a meaningful proportion of the work flowing through them.
The challenge isn’t that people suddenly can’t understand the work. It’s that attention doesn’t scale.
As an engineer, and now as a leader, I’ve spent much of my career juggling multiple threads simultaneously. Having severe ADHD has, somewhat ironically, made me more comfortable operating in environments with a lot of parallel work and rapid context switching than many people.
But complexity and volume aren’t the same thing.
Every additional thread creates cognitive noise. Sometimes that noise is productive; it helps you make connections that wouldn’t otherwise exist. Beyond a certain point, though, it simply becomes harder to separate the signal from the background.
Organizations are rapidly beginning to experience the same problem, just at enterprise scale.
AI hasn’t just reduced the cost of producing work. It’s reduced the cost of producing things that demand our attention. Code. Documentation. Design proposals. Security findings. Analyses. Recommendations. The bottleneck is no longer creating them. It’s deciding what deserves attention in the first place.
There’s still only so much code one engineer can review in a day, so many architectural decisions one person can evaluate, or so many security findings a team can meaningfully investigate before review becomes little more than acknowledgement.
If AI increases the pace of knowledge work by even one order of magnitude, and I suspect it may be several, adding more reviewers isn’t really a solution. That’s linear thinking applied to an exponential problem.
I think the answer is to stop asking how to put humans into every loop and start asking which loops genuinely require human judgment.
That shifts governance away from inspecting every individual action and toward governing the systems that produce those actions. It means defining constraints instead of relying on approvals, validating outcomes instead of every intermediate step, and building guardrails that operate at machine speed while reserving people for ambiguity, ethics, trade-offs, and genuinely novel situations.
If anything, that elevates the role of people rather than diminishes it.
The human contribution becomes less about rubber-stamping outputs and more about designing systems that deserve to be trusted.
Many are still asking, “How do we keep a humans in the loop?”
I suspect the more important question is, “How do we build systems that remain governable when humans can no longer keep up with the loops we’ve created?”
The question isn’t whether AI will outpace human work. I think it already has in some domains.
The real question is whether our governance models will adapt before they become little more than rituals we perform to reassure ourselves we’re still in control.
