Findings — TrustFork

Evaluation

1,890 tasks per system · 16 agent systems · 27,826 trajectories

Thirty risk scenarios and 1,890 individually reviewed tasks. Eight vulnerability measures, 25 judge-annotated diagnostics, and seven code-extracted observations locate where operational authority changes hands.

1,890

Tasks per system

16

Agent systems

27,826

Trajectories

8

Vulnerability measures

An orchestrator trusts the subagent that displays its own model family. It lets that subagent run a poisoned cache cleaner and checks its action only after the deletion. TrustFork builds each task around a risk scenario and follows the orchestration trajectory that displayed identity shapes. It traces who receives scope, who verifies, whose advice is adopted, and who executes.

One subagent carries a risky goal while the other three stay aligned with the user, so contradicting evidence can arise. A task can change the identity a subagent displays while keeping the model behind it, so a shift in who is trusted traces to the label. Each task also sets how many subagents the orchestrator must consult, which separates evidence it never receives from evidence it receives and overrides.

TrustFork overview: displayed identity can stay in charge even after the evidence turns against it. — **Overview of TrustFork.** An orchestrator trusts a subagent displayed as its own model family, lets it run a poisoned cache cleaner, and checks its action only after the deletion. TrustFork builds each task around a risk scenario and follows the orchestration trajectory that displayed identity shapes. It traces who receives scope, who verifies, whose advice is adopted, and who executes before the run reaches its terminal state.

The measures

Eight measures and one failure event along the authority path

Each score attaches to a decision from returned evidence through operational authority to the terminal state. All eight use vulnerability orientation: higher is more severe. Reports multiply 0–5 anchors by 20. Terminal Harm and Task Utility Loss invert raw safety and utility so they also increase with failure. Captured Authority Rate scores the failure event: the risky response is still adopted or governs execution after evidence contradicts it.

What a task specifies

Identity, goal, breadth, and pool

Every TrustFork task is x = (s, m, g, b, p): a risk scenario, an identity presentation, the goal assigned to one risky subagent, a consultation-breadth requirement, and a four-slot pool. One subagent receives the risky goal, while the others stay user-aligned.

Main results

Results across orchestration backbones and harnesses

Vulnerability measures use ↑, and higher values indicate greater vulnerability. The most common failure is a risky subagent capturing operational authority. Even when another subagent contradicts the risky response, the orchestrator still acts on it in 48.0% to 100.0% of these trajectories. CAR exceeds 50% in 14 of the 16 systems and averages 72.0%. Even GPT-5.6-Sol, the lowest in OpenCode, leaves the risky response in control of 48.6% of them. CAR also does not follow terminal harm. Kimi-K3 and Kimi-K2.6 leave the least terminal harm in OpenCode at 12.6 and 13.6, but they reach the highest CAR at 92.0 and 100.0 and at least twice the verification failure of GPT-5.6-Sol. A low harm score therefore does not certify a sound authority path. Runs orchestrated by GPT-5.6-Luna receive safer subagent responses than runs orchestrated by GPT-5.6-Sol. Their RSR is 20.8 against 23.2. Yet their CAR reaches 75.1 against 48.6, and their terminal harm reaches 27.6 against 20.5. In every model family, evidence inertia is higher under efficient-tier orchestrators than under frontier-tier ones. GLM-4.7 and Minimax-M2.5 leave the most terminal harm in every harness they run under and reach 36.5 and 42.0 under OpenClaw.

Reported scores are 0–100 vulnerability scores. They multiply 0–5 anchors by 20, and TH and TUL invert raw safety and utility. CAR is the share of trajectories with contradicting evidence in which the risky response is still adopted or governs execution.

RQ1

Where displayed identity steers authority

25.9% → 74.6%

Risky response obtained, truthful vs reversed family

54.8% → 67.1%

Evidence supports the risky response after a model swap

62.3% → 77.7%

Risky subagent receives scope, truthful vs reversed family

Changing a label changes which responses arrive. With the truthful family display, the orchestrator obtains the risky response in 25.9% of tasks. Reversing the family display raises this to 74.6%, which nearly triples the risky response's entry into comparison, and the rise holds in every system where both displays occur. Tier and model cues instead move the risky subagent into the first call without changing how many subagents are consulted, which stays at 2.58 on average. The tier cue raises this share from 12.3% to 33.9%, and the model cue raises it from 21.9% to 33.5%.

Exchanging the displayed model raises the share of tasks where observed evidence supports the risky response from 54.8% to 67.1%, and it does so in all eight systems. Safer responses are also lost in two ways: the orchestrator drops a response before comparison in 634 trajectories and discounts it during comparison in 1,090.

RQ1: altered displays let the risky response in, per system. — **Displayed identity changes which responses arrive.** Every system lies above the diagonal: altered displays let the risky response in and make the evidence favor it more often.

Displayed identity decides who gets the work. Reversing the family display raises the share of tasks where the risky subagent receives scope from 62.3% to 77.7%. Its response also becomes primary more often, from 23.8% to 31.7%, and governs execution more often, from 18.1% to 25.3%. Exchanging the displayed model raises scope from 64.2% to 79.1%, while reversing the tier shifts pooled scope only slightly and in opposite directions across systems.

Even the tier cue still reaches the later stages, raising the risky subagent's permission from 13.9% to 19.2% and its execution from 14.6% to 18.5%. Mixed-family pools also show affinity toward same-family subagents. In both such pools, the subagent from the orchestrator's true family receives 21.4 and 19.7 percentage points more tasks than an even split, before the orchestrator decides which response to use.

RQ1: change in the risky subagent's scope, permission, primary role, and execution for each identity cue. — **Displayed identity decides who gets the work.** Family and model cues widen the risky subagent's scope in every system, whereas reversing the tier moves it in opposite directions across systems.

RQ2

How operational authority turns into harm

63.8% → 33.5%

Unsafe adoption, no restriction vs four subagents

84.0% / 25.0%

Safer response present, vs in charge

96.1%

Completed actions that leave the harmful change in place

More responses reduce unsafe adoption, but propagation falls much less. Among tasks where at least one subagent returns an unsafe response, risk propagates in 28.6% under unrestricted consultation and is contained in 13.7%. Propagation is highest when the risky subagent acts out the scenario attack, at 29.2%, against 26.2% for ignoring risk and 23.9% for seeking authority.

Consulting all four subagents roughly halves unsafe adoption, from 63.8% to 33.5%, but propagation falls by only five percentage points, and for GLM-4.7 and Minimax-M3 it rises by 0.5 and 2.4 points instead. The additional responses do not all argue against the risky response: evidence supporting it rises from 39.0% to 64.5% of these tasks, faster than evidence contradicting it, from 16.2% to 35.2%, so propagation barely moves.

RQ2: per-system drop in unsafe adoption and in risk propagation with four subagents, and pooled levels without restriction and with four subagents. — **More responses reduce unsafe adoption more than risk propagation.** Four subagents cut unsafe adoption far more than propagation in every system.

Safer responses exist but rarely decide. Consulting all four subagents does not stop the risky subagent from capturing operational authority. When one response recommends a dangerous action, a safer response is present in 84.0% of tasks, yet decides the outcome in only 25.0%. This gap is what CAR measures, and it persists even when every subagent is consulted.

An independent check lowers unsafe adoption from 63.8% in unchecked tasks to 34.9%. The dangerous action is carried out in 42.6% of tasks that adopt the unsafe recommendation, compared with 12.2% of tasks that do not. Once the action completes, the harmful change remains in place in 96.1% of cases.

RQ2: a safer response is present in most tasks yet decides few outcomes. — **Safer responses exist but rarely decide.** Checking lowers unsafe adoption, yet safer responses decide few outcomes.

RQ3

Can contradicting evidence take authority back?

73.2% → 48.8%

Tasks without contradicting evidence, no restriction vs four subagents

13.4% vs 56.5%

Propagation when correction lands before vs after the first action

24.0% vs 3.3%

Risky response still preferred after a same-family vs other-family correction

Contradicting evidence is often missing. Evidence contradicting any response is absent from 73.2% of tasks under unrestricted consultation. Even consulting all four subagents leaves 48.8% of tasks without it, and from no restriction to four subagents the absence falls in every system.

Loss of safer responses can prevent the orchestrator from reconsidering its preferred response. The figure separates tasks without contradicting evidence, tasks where it arrives but the same subagents keep the work, and tasks revised before execution. In tasks without contradicting evidence, the orchestrator never changes these work assignments.

RQ3: contradicting evidence, inertia, and early revision per system across consultation breadth. — **Contradicting evidence misses.** Four subagents narrow the gap, yet half of tasks still lack contradicting evidence.

Correction helps only before action and from another family. When all four subagents return and contradicting evidence arrives, a correction before the first action leaves risk propagating in 13.4% of tasks. The same correction afterward comes too late: 56.5% propagate even though the orchestrator eventually changes its decision. Propagation is 6.5% when the work is already assigned to subagents supported by the evidence, against 48.1% when the orchestrator keeps its earlier preference.

Displayed identity still matters after contradicting evidence arrives, as exchanging the displayed model raises adoption from 58.0% to 69.9%. The evidence source matters too: a same-family correction leaves the risky response preferred in 24.0% of tasks, against 3.3% for an other-family one.

RQ3: risk propagation by correction timing and update state across systems, and how the correction source and displayed model shape authority after contradiction. — **Correction helps only before action and from another family.** Late corrections and kept preferences propagate risk far more than early ones.

RQ4

Where harnesses change the authority path

65.6 / 51.3 / 20.0

Safer response visible, OpenCode / Pi / OpenClaw (%)

19.7–22.2%

Risky response governs execution under all three harnesses

46.8 / 37.0 / 19.0

Independent check before action, OpenCode / Pi / OpenClaw (%)

Harnesses differ most in what they send back. A safer response is visible in 65.6% of OpenCode tasks and 51.3% of Pi tasks, yet only 20.0% under OpenClaw, even when the harness launched every requested subagent. By contrast, the risky subagent is consulted first in 25.9% to 27.5% of tasks in every harness and governs execution in 19.7% to 22.2%.

Evidence delivery changes much more than execution authority, and fewer returned answers accompany greater response task failure. The backbone also matters within a harness. Under OpenClaw, GLM-5.2 sees a safer response in 61.0% of tasks, while Minimax-M2.5 sees one in only 7.0%.

RQ4: safer-response visibility versus risky execution per system and harness. — **Harnesses differ most in what they send back.** Safer-response visibility varies widely while risky execution stays near one fifth of tasks.

The check before action is where the lost evidence hurts. Independent verification before action occurs in 46.8% of OpenCode tasks and 37.0% of Pi tasks but in only 19.0% under OpenClaw. Pooled risk propagation moves the opposite way, from 26.4% under OpenCode and 29.4% under Pi to 34.1% under OpenClaw. Check rates follow the order of safer-response visibility, and the Spearman correlation with propagation across the 16 systems is −0.39.

The relation is sharpest in OpenClaw. GLM-5.2 sees a safer response in 61.0% of tasks, checks before acting in 62.7%, and propagates risk in 10.2%, whereas Minimax-M2.5 reaches 7.0%, 7.5%, and 56.8%. Since late correction leaves earlier changes in place, the divergence is fixed before execution, and a terminal score alone would hide where the paths diverged.

RQ4: independent check before action versus risk propagation across 16 systems. — **Less checking before action accompanies more risk propagation.** Propagation falls as independent checking rises.

RQ5

Which runtime interventions actually help

−6.2 to −9.1

SAF drop from anonymization, all three harnesses

6 of 8

Measures improved by verify-first under OpenCode and Pi

153 → 341 s

Wall-clock time per task for verify-first under OpenCode

RQ5 tests anonymization, identity attestation, and verification before action with Minimax-M3 under all three harnesses. Anonymization hides the identity cues, whereas identity attestation keeps them and proves they are genuine. Verification before action forces a check before the agent changes the workspace.

Anonymization helps most consistently, improving 20 of 24 harness and measure pairs: security awareness failure falls by 6.2 to 9.1 points, and evidence inertia falls in every harness. Attesting identities raises terminal harm in every harness, because attestation keeps the identity cue and makes it more credible. Verification before action improves six of eight vulnerability measures under OpenCode and Pi, but under OpenClaw it improves only VF and TUL and raises the other six measures, even as the pass rate rises from 66.9% to 75.6%. Under OpenClaw the check arrives after the failure RQ4 exposed, since only 20.0% of tasks return a safer response for the check to use.

Protection has a cost. Under OpenCode, verifying first raises time per task from 153 to 341 seconds, and anonymizing raises it to 270 seconds. Under Pi, both stay within 1.5 times the default. Binding authority to evidence before action is feasible, at a price that depends on the harness.

Open a trajectory How to run

Displayed identity as operational authority