Notifications
Clear all
Topic starter
11/08/2026 3:16 pm
TL;DR: Large swings in autonomous red-teaming can come from the harness alone, including 43 outright flips across 100 model-and-mission pairings and success rates ranging from 1% to 34%, according to Lasso Security’s comparison of Claude Agent SDK and deepagents. The result shows that agent evaluation and governance must treat the runtime scaffold as part of the control surface, not just the model.
NHIMG editorial — based on content published by Lasso Security: Claude Agent SDK vs. deepagents: How Much Does the Agent Harness Move Autonomous Red-Teaming?
By the numbers:
- The two harnesses correlate at just 0.35 across individual model-and-mission pairings.
Questions worth separating out
Q: How should security teams evaluate autonomous agents when the harness changes outcomes?
A: Security teams should evaluate the model and harness together, using the exact runtime stack that will run in production.
Q: Why does the harness matter so much for agentic AI governance?
A: Because the harness controls how the model sees instructions, reaches tools, retains state, and decides when the task ends.
Q: What do security teams get wrong about AI agent benchmarks?
A: They often measure end-to-end complexity or general model quality instead of the exact failure moment.
Practitioner guidance
- Define the agent harness as a governed control component Inventory every runtime wrapper, middleware block, and gateway in the agent path, then assign ownership for each one.
- Benchmark model and harness as a single deployment unit Stop reporting model-only evaluation results for agentic systems.
- Audit tool schemas for privilege inflation Review tool descriptions, planning blocks, and sub-agent capabilities for hidden authority expansion.
What's in the full report
Lasso Security's full article covers the operational detail this post intentionally leaves for the source:
- The exact side-by-side request assembly differences between Claude Agent SDK and deepagents across the system prompt and tool schema layers.
- The full objective success table across all ten model-and-harness combinations and twenty attack missions.
- The scoring method used to separate self-reported wins from independently judged outcomes.
- The authors' recommended next test for multi-tool agent harnesses and broader red-team evaluation.
👉 Read Lasso Security's analysis of Claude Agent SDK vs. deepagents in autonomous red-teaming →
Agent harness choice and autonomous red-teaming: what changes most?
Explore further
11/08/2026 3:28 pm
Harness governance is now part of AI security governance. The article shows that the runtime scaffold can materially alter autonomous behaviour even when model, prompt, tools, and target stay fixed. That means security teams cannot treat the harness as a neutral implementation detail. The governance question is whether the runtime preserves the same access boundaries, tool exposure, and termination rules across environments. Practitioners should govern the harness as an enforceable control layer, not as an engineering convenience.
A question worth separating out:
Q: How can organisations reduce risk when deploying general-purpose agent frameworks?
A: They should standardise approved harnesses, restrict tool exposure to task-scoped permissions, and verify that each framework preserves the same authorisation boundaries and logging behaviour. General-purpose orchestration increases portability, but it also creates room for hidden behavioural differences that can widen risk. Governance has to follow the runtime, not just the model vendor.
👉 Read our full editorial: Agent harness choice materially changes autonomous red-teaming outcomes
