I've been thinking recently about the frontier AI threat and the answer everyone reaches for, which is that we fight fire with fire, AI against AI. On paper it sounds like the right thing to do. The difficulty is that the attacking AI has no guardrails, and ours do. Our defenders also carry acceptable use policies, legal constraints, regulatory duties and personal accountability, and they have to work inside all of them while the thing coming at them works inside nothing at all.
This stopped being theoretical this week.
OpenAI disclosed that its models, running a cyber capability benchmark in what was meant to be an isolated environment, found and exploited a zero-day in third party software, escalated privileges, moved laterally until they reached a node with internet access, and got out. They then broke into Hugging Face, having reasoned that Hugging Face held the answers to the evaluation they were being marked on. Those models were running with reduced safety guardrails, because that is what evaluation requires.
Now hold that against what the defender experienced. Hugging Face, under attack, tried to use frontier models behind commercial APIs to respond. It did not work. The guardrails could not distinguish an incident responder from an attacker. Travis Lelle of Guidepoint Security summarised it to the BBC as an asymmetry in which offensive agents are unconstrained while the best defensive tools sit behind guardrails that cannot understand context.
One organisation was attacked by an AI with the brakes off and could not obtain help from an AI with the brakes on. That is the whole problem in a single incident.
It is not one asymmetry, it is five
The guardrail gap is the visible one, and it gets all the attention. Underneath it sit four more, and they are harder to fix.
Tempo. Hugging Face described a campaign of many thousands of individual actions across a swarm of short-lived sandboxes, with command and control staged on public services and migrating itself. Our side has change advisory boards, maintenance windows and a rule that says nothing goes into production on a Friday. I am not arguing those controls are wrong. I am pointing out that we built them for a threat that operated at human speed, and we have not revisited that assumption.
Consequence. When an attacking agent gets it wrong, nothing happens. It retries. When our agent gets it wrong, it isolates a payment platform at month end and somebody explains that to a board. The attacker plays a game with no downside on failure. We play one where a false positive is more visible than a true positive, which is exactly the incentive structure that produces hesitation.
Law. The attacker has no lawful basis to establish, no data protection impact assessment, no legal review of an active response measure, no regulator to notify. We have all of them, and rightly so. But it means our defensive agent cannot be pointed at half the data that would let it do its job, and the attacking agent reads everything it takes.
Accountability. This is the one that gets least discussion and does most damage. The attacking agent answers to nobody. Our analyst is personally accountable for whatever the defensive agent does under their name. That single fact changes behaviour more than any tooling decision we make.
Why the metaphor breaks
Fighting fire with fire is a real technique. Backburning works. But it works because the crew establishes a control line first, in daylight, with agreed authority, and burns towards it deliberately. Nobody lights a second fire and hopes. Do that and you do not have a defence, you have two fires.
Most of what I hear described as fighting fire with fire in security is the second version. Buy the agentic platform, point it at the estate, hope it moves as fast as the thing attacking us. No control line. No agreed authority. No rehearsal. When it goes wrong the failure is ours, at scale, and faster than we can read the alerts.
The behavioural problem hiding inside the technical one
Parasuraman and Riley set out the pattern in 1997 in their work on use, misuse, disuse and abuse of automation, and it has held up for nearly thirty years. Operators do not fail by trusting automation too little or too much in the abstract. They fail when their trust is miscalibrated to the actual reliability of the system in front of them.
Both failure modes are already in our teams.
Misuse looks like an analyst accepting an agent's triage conclusion without checking it, because it has been right nine times this shift and it is now half past two in the morning. Disuse looks like the analyst who asked for help with a live payload, got refused, and never asked again. I have watched the second one form in a single interaction. It survives the vendor fixing the problem six months later, because nobody goes back to retest a tool that once let them down under pressure. We pay for capability and lose it to one refusal at the worst possible moment.
Worse, disuse does not remove the demand. It relocates it. The work goes to a personal account, an open weight model on somebody's laptop, a browser tab nobody is logging. The defensive work carries on, off the audit trail, on an unmanaged endpoint, with real incident data in the prompt. We did not prevent anything. We just stopped being able to see it.
What actually narrows the gap
Not removing our guardrails. That argument surfaces every time an incident like this lands, and it is the wrong lesson. The answer is to make our constraints legible to the machine and to make the decision to act survivable for the person.
Legible means identity. Guardrails are context blind because the context they need is organisational rather than textual. Nothing in a prompt tells a model that the person typing is a named responder, on a declared incident, inside an authorised scope, with the session logged and retained. We already hold that information. It is sitting in our identity platform. The gap between what our IdP knows and what the model can see is where the defender disadvantage lives, and it is a solvable engineering problem rather than a philosophical one.
Survivable means the decision is made before the incident, not during it. Who is authorised to stop an agent. At what threshold. With what cover if they are wrong and stop it early. If the answer to any of those is worked out at three in the morning by whoever happens to be on shift, we have not delegated the decision, we have abandoned it. Every safety critical industry learned this and wrote it down. We keep treating it as a maturity nicety.
And then there is the advantage we already have and mostly waste. We own the terrain. No attacking agent knows our estate the way we do, or should. It has to discover what our own asset inventory could tell us in seconds, if we had bothered to make it accurate. The asymmetry does not run entirely against us. It runs against us in the places where we never did the unglamorous work.
The attacker has no procurement cycle, no acceptable use policy and no legal review. We are not going to close that gap by being faster or by loosening what makes us worth defending. We might close a good part of it by proving who our defenders are, deciding in peacetime what they are allowed to do, and knowing our own ground better than the thing coming for it.