Agent harness choice materially changes autonomous red-teaming outcomes

At a glance

What this is: This research shows that the agent harness, not just the model, can materially change autonomous red-teaming outcomes and even flip attack success on the same mission.

Why it matters: It matters because IAM, PAM, and AI security teams need to govern the full agent execution stack, including tool orchestration, prompt assembly, and runtime state, not only the model itself.

By the numbers:

• Out of a 100 such pairings, 43 are outright flips: one harness breaks into the target at least once, the other never does.
• The two harnesses correlate at just 0.35 across individual model-and-mission pairings.

👉 Read Lasso Security's analysis of Claude Agent SDK vs. deepagents in autonomous red-teaming

Context

Autonomous red-teaming is useful only if the evaluation reflects the real execution environment. In this case, the primary governance gap is assuming the model is the whole control surface when the harness also shapes prompt packaging, tool dispatch, state retention, and termination behaviour.

For AI security and identity teams, that matters because an agent harness is part of the runtime identity and privilege boundary for the system. If the scaffold changes how tools are exposed or how long an agent persists, then access governance, auditability, and containment all shift with it.

The article's core claim is that harnesses are not interchangeable substrates. That is the right starting point for any team trying to govern AI agents, red-team them, or compare their real-world risk.

Key questions

Q: How should security teams evaluate autonomous agents when the harness changes outcomes?

A: Security teams should evaluate the model and harness together, using the exact runtime stack that will run in production. A standalone model score can hide major differences in tool exposure, prompt packaging, and turn management. The right test is the deployed pair against the real task, with outcomes measured on the full execution path.

Q: Why does the harness matter so much for agentic AI governance?

A: Because the harness controls how the model sees instructions, reaches tools, retains state, and decides when the task ends. Those functions shape effective authority, persistence, and refusal handling. In governance terms, the harness helps define the non-human actor's operational identity, so it belongs in the control model, not outside it.

Q: What do security teams get wrong about AI agent benchmarks?

A: They often measure end-to-end complexity or general model quality instead of the exact failure moment. The result is a score that looks useful but does not isolate whether the backbone, tool access, or orchestration caused the problem. Benchmarks need a specific state, attack vector, and scoring function to be operationally meaningful.

Q: How can organisations reduce risk when deploying general-purpose agent frameworks?

A: They should standardise approved harnesses, restrict tool exposure to task-scoped permissions, and verify that each framework preserves the same authorisation boundaries and logging behaviour. General-purpose orchestration increases portability, but it also creates room for hidden behavioural differences that can widen risk. Governance has to follow the runtime, not just the model vendor.

Technical breakdown

Why agent harness architecture changes red-team outcomes

An agent harness is the runtime wrapper that assembles the system prompt, injects tool schemas, manages turn-by-turn calls, stores working state, and decides when the task ends. In practice, that means the harness influences the model's observable behaviour at every step, not only at initialization. Different harnesses can present the same model with different instruction ordering, different middleware blocks, and different tool exposure patterns. In adversarial workflows, those differences matter because attack persistence, refusal recovery, and tool selection are all mediated by the harness. The key point is that the model is only one part of the execution chain; the harness can change what the model can see and do.

Practical implication: Benchmark the model and harness together, because a model score alone does not describe the operational risk of the deployed agent runtime.

How tool schemas and prompt scaffolding alter agent behaviour

Tool schemas are not just metadata. They are part of the request the model reads, and their length, ordering, and wording can compete with or reinforce the agent prompt. A harness that places capabilities in the system prompt creates a different behavioural environment from one that hides capability detail in tool descriptions. That affects not only planning but also when the model reaches for a tool, how it recovers after refusal, and whether it stays aligned across a long chain of actions. In a red-team setting, those small differences compound turn after turn, producing large differences in outcome even when the underlying model is unchanged.

Practical implication: Treat tool descriptions and prompt scaffolds as security-relevant configuration, not as implementation noise.

Why shared gateways do not eliminate harness risk

Even when two agent stacks use the same model endpoint and the same gateway, the request shape can diverge enough to trigger different upstream guardrail behaviour. That means the observed security result is not just a property of the model provider or the filter layer. It is also a property of how the harness packages the call. In this study, the same filter rejected one harness more often than the other, which shows that orchestration can change both attack success and defensive interception. For governance, the lesson is that control validation has to include the full request path, from agent runtime to model endpoint.

Practical implication: Test the complete agent path, including middleware and gateway effects, before assuming a control works consistently across harnesses.

Threat narrative

Attacker objective: The attacker objective is to complete a multi-step AI application compromise, including leakage, harmful output, or sensitive disclosure, by exploiting the agent runtime path rather than the model alone.

1. Entry begins when the attacker agent reaches the target through a harness-mediated request path that packages the prompt and tools in a way the target will accept.
2. Escalation occurs as the harness sustains multi-turn probing, retries, and tool use, allowing the agent to adapt after refusals rather than stopping at the first denial.
3. Impact follows when the harness allows enough persistence and tool execution for the attack to succeed, or when guardrails are bypassed by the request shape itself.

NHI Mgmt Group analysis

Harness governance is now part of AI security governance. The article shows that the runtime scaffold can materially alter autonomous behaviour even when model, prompt, tools, and target stay fixed. That means security teams cannot treat the harness as a neutral implementation detail. The governance question is whether the runtime preserves the same access boundaries, tool exposure, and termination rules across environments. Practitioners should govern the harness as an enforceable control layer, not as an engineering convenience.

Agent orchestration creates a new trust boundary around tool use. When a harness decides how tools are exposed, ordered, and retried, it becomes part of the effective privilege model for the agent. That is directly relevant to NHI governance because the agent runtime behaves like a non-human actor whose authority is defined operationally, not just by policy. Teams managing AI agents need to account for the runtime's ability to expand, narrow, or repackage access during execution. Practitioners should align runtime design with least-privilege and task-scoped access assumptions.

Named concept: harness-induced capability variance. This is the gap where the same agent model behaves differently because the scaffold changes prompt assembly, tool packaging, or gateway interaction. The study shows that capability reporting at the model level can hide large operational differences at the model-plus-harness level. That undermines any governance approach that certifies only the model while leaving the runtime unreviewed. Practitioners should evaluate agent capability as a paired system, not a standalone model.

Autonomous red-teaming needs more than model benchmarks. The article makes clear that the same model can appear safe or unsafe depending on the harness that drives it, which means benchmark design itself is a governance issue. If the evaluation environment does not mirror the deployed runtime, the resulting assurance is incomplete. This is especially relevant for AI security programmes that are building controls around agent oversight, tool permissions, and audit logging. Practitioners should validate the full execution stack before using benchmark results for risk decisions.

For IAM and PAM teams, agent identity cannot stop at authentication. If the harness shapes how long an agent persists, what it can call, and how it recovers from failure, then the control model must extend into runtime authorisation and session behaviour. That is the same design problem identity programmes face with privileged service accounts and other NHIs: initial access is not enough to describe risk. Practitioners should map agent runtime behaviour to access governance, audit, and containment controls.

What this signals

Harness variance is becoming a practical governance issue because agent evaluation, incident response, and access control all depend on the runtime path behaving consistently. If the scaffold can change refusal behaviour or guardrail activation, then the security team needs environment-level evidence, not abstract model assurance. For practitioners, the control objective is to make agent runtime behaviour observable, repeatable, and reviewable.

Execution-path drift: when the same agent produces different security outcomes because middleware, tool schemas, or gateway translation changed. That drift matters for identity programmes because AI agents increasingly behave like privileged non-human actors with runtime authority. Teams should compare production harnesses against their approved access model and align them with the NIST AI Risk Management Framework.

As agent fleets grow, the operational risk shifts from one model decision to the consistency of the whole orchestration layer. The clearest warning signal is not just whether an agent can act, but whether two approved runtimes can make the same agent behave differently. That is where access governance, logging, and containment controls need to converge.

For practitioners

• Define the agent harness as a governed control component Inventory every runtime wrapper, middleware block, and gateway in the agent path, then assign ownership for each one. Document which parts alter prompt assembly, tool exposure, working state, and termination so reviewers can assess the full control surface, including the shared LiteLLM gateway path.
• Benchmark model and harness as a single deployment unit Stop reporting model-only evaluation results for agentic systems. Score the exact model-plus-harness pairing against the real downstream task, because the same model can behave very differently when the scaffold changes the request shape or tool orchestration.
• Audit tool schemas for privilege inflation Review tool descriptions, planning blocks, and sub-agent capabilities for hidden authority expansion. Remove unnecessary tool options, shorten capability text where it changes behaviour, and make sure the runtime does not expose more action surface than the task requires.
• Test guardrails against request-shape variance Run the same agent workflow through each production harness and compare filter outcomes, refusal rates, and execution completion. If one scaffold triggers guardrails while another passes, treat that as a configuration problem with security impact, not as a model quirk.

Key takeaways

• The study shows that autonomous agent risk is shaped by the runtime harness, not only by the model that powers it.
• Large outcome swings across identical tasks mean model-only benchmarking is insufficient for agent governance and red-teaming.
• Security teams should treat harness design, tool exposure, and gateway behaviour as part of the agent control plane.

Key terms

• Agent Harness: The agent harness is the runtime layer that wraps a model and turns it into an acting system. It usually includes the loop, tools, context handling, permissions, hooks, and logs. In security terms, it is often the real place where privilege sits and where identity evidence must be governed.
• Model-Plus-Harness Configuration: A model-plus-harness configuration is the specific pairing of a base model with the runtime framework that drives it. This pairing matters because the same model can produce different security outcomes when the scaffold, tool routing, or prompt assembly changes.
• Execution Path: An execution path is the chain of systems, roles, functions, and permissions that an identity uses to reach a target service. For AI agents, this matters more than the label on the agent itself because effective authority often comes from the path, not the object.
• Request Shape: Request shape is the structure of the final payload a harness sends to a model, including prompt ordering, tool schema size, and surrounding framework text. Different request shapes can alter both model behaviour and upstream guardrail reactions even when the model endpoint is the same.

What's in the full report

Lasso Security's full article covers the operational detail this post intentionally leaves for the source:

• The exact side-by-side request assembly differences between Claude Agent SDK and deepagents across the system prompt and tool schema layers.
• The full objective success table across all ten model-and-harness combinations and twenty attack missions.
• The scoring method used to separate self-reported wins from independently judged outcomes.
• The authors' recommended next test for multi-tool agent harnesses and broader red-team evaluation.

👉 The full Lasso Security post covers the harness assembly differences, scoring method, and attack-outcome table.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It gives security and identity practitioners a common language for governing non-human access across modern environments.