117
confirmed concepts
AI-safety · red-teaming
AutoResearch for Production-Agent Red-Teaming
City University of Hong Kong
Research use — run only against systems you own or are authorized to test. See SECURITY.md.
Red-teaming a production agent usually yields a pile of one-off jailbreak strings that stop working the moment the task changes. AHA turns red-teaming into autoresearch: a researcher agent iterates on its own — hypothesize an attack, run it against a Docker-sandboxed victim agent, judge the trajectory, reflect — and accumulates a persistent Vulnerability Concept Graph (VCG) of why a family of attacks works, when it fails, and how to reinstantiate it. Because the unit of discovery is a concept rather than a prompt, a finding can be frozen and reused single-shot on held-out tasks — and it keeps working. Across two production agent stacks, three frontier victim models, and three scenario families, AHA discovered 117 confirmed concepts, clustering into eight recurring mechanism families, and reached 47.0% held-out attack success — +14.2 points over the strongest frozen discovery baseline.
The finding is a concept
Every confirmed finding is a concept, not a payload — a record of the mechanism a family of attacks exploits, the falsifier that would refute it, and how to reinstantiate it. Clustering the discovered graphs yields eight families; two are cores that recur across models, agents, and scenarios.
Grants the harmful step legitimacy through an ownership / admin / compliance frame, so the agent never reconstructs the global intent.
Injected content in a tool result plants a competing goal that the agent adopts as its own task.
Each step looks benign; the harm exists only in the composition across turns, so local checks never aggregate it.
Forges identity, authority, or provenance markers ("our team's key") so the agent trusts the source of the request.
Wraps the request in fiction / roleplay so the model treats a real operation as make-believe.
A legit-sounding declared purpose — testing, debugging, a defensive fixture — launders the harmful action.
Frames the harmful action as normal, expected operations inside the environment, so it passes as ops.
Games the judge / verdict layer so a harmful result is scored or treated as acceptable.
Three concepts, verbatim from the graph — the mechanism (why), the falsifier (when it fails), and its reach.
The claimed-authorization core lights up 16 of 18 settings and, realized, produces machine takeover, fraud, exfiltration, and destruction alike. Frozen and reused single-shot, the concepts even break scenarios and victim models they were never discovered on. Patching the mechanism closes a whole class of failures — not a single prompt.
Three environments, one concept library
AgentHazard, AgentDyn, and DTap aren't three separate findings; they're three environments where the same reusable concepts land, each turning the shared mechanisms into scenario-appropriate harm. The deliverable is the concept library that carries across all three.
Interactive casebook
Browse by mechanism family to watch one concept recur across scenarios and victims — or switch to by target cell. Click any tile to read the vulnerability concepts: the mechanism (why), what the agent did, and the evidence. 117 concepts, 8 families, 17 cells — every cell broken.
Watch it work
No human in the loop: the researcher agent commits a hypothesis, designs an attack, runs it against a sandboxed victim, and promotes only replicated, non-falsified breaks into its graph.
A concept enters the graph only after it replicates and survives its own falsifier.
The same frozen concept, reused single-shot, then breaks tasks the loop never saw — across models and agents.
Reusable & transferable
Each discovery artifact is frozen and reused single-shot on held-out tasks, with no further search — so every break is the concept's, not the search's. And a concept frozen on one setting still breaks the others.
| Scenario | Claude Code | Codex | AHA avg | best baseline |
|---|---|---|---|---|
| AgentHazard | 77.4 | 74.8 | 76.1 | 58.5 |
| AgentDyn | 25.0 | 20.2 | 22.6 | 6.6 |
| DTap | 47.6 | 36.7 | 42.2 | 41.5 |
| Overall | 47.0 | 32.8 | ||
+14.2 points over the strongest frozen discovery baseline — the concepts keep working when detached from the search that found them.
A concept frozen on one setting still breaks the others: the finding lives in the agent's trajectory, not in a payload tied to where it was found. A mechanism compounds where an exploit decays.
Bring your own
The three scenarios here are where transfer was measured — not the ceiling. A build workflow turns a plain-language red-team concern into a runnable scenario; an import workflow turns an existing benchmark into one. Same loop, new target.
@online{2607.11698,
Author = {Xutao Mao and Xiang Zheng and Cong Wang},
Title = {Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming},
Year = {2026},
Eprint = {2607.11698},
Eprinttype = {arXiv},
}