Topic starter
11/08/2026 3:16 pm
TL;DR: Large swings in autonomous red-teaming can come from the harness alone, including 43 outright flips across 100 model-and-mission pairings and success rates ranging from 1% to 34%, according to Lasso Security’s comparison of Claude Agent SDK and deepagents. The result shows that agent evaluation and governance must treat the runtime scaffold as part of the control surface, not just the model.
Editorial analysis by NHI Mgmt Group, based on content published by Lasso Security: “Claude Agent SDK vs. deepagents: How Much Does the Agent Harness Move Autonomous Red-Teaming?”.
By the numbers:
- Objective success rates ranged from 1% to 34% across the harness and model combinations, according to Lasso Security.
Key questions
Q: How should security teams evaluate autonomous agents when the harness changes outcomes?
A: Security teams should evaluate the model and harness together, using the exact runtime stack that will run in production.
Q: Why does self-scoring create false confidence in autonomous red-teaming?
A: Self-scoring fails because the agent’s own judgment is not an independent measure of success.
Q: What are the signs that an agent harness is affecting security outcomes?
A: Look for large swings in success rate across the same model, toolset, and mission, especially when one scaffold repeatedly reaches the target and another does not.
Practitioner guidance
- Benchmark the full agent stack Test the exact model, harness, toolset, and target combination you plan to run in production, rather than scoring the model alone.
- Separate generation from judging Use an external judge or deterministic checks for autonomous red-teaming so the attacker cannot grade its own success.
- Inventory harness-specific control differences Document how each harness assembles prompts, exposes tools, manages state, and terminates loops so authorisation assumptions are explicit.
Bottom line: The study shows that harness choice can materially change autonomous red-teaming results, so the runtime scaffold belongs in the security control surface.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 4 hours ago by NHI Mgmt Group
11/08/2026 3:28 pm
Harness governance is now part of agent governance. The article shows that an agent’s runtime scaffold can materially alter the outcome of the same model, prompt, and tools. That means security teams cannot treat harness selection as engineering convenience; it is a control decision that changes how access, tool invocation, and termination are enforced. For AI security and NHI governance, the practical conclusion is straightforward: the execution layer must be evaluated with the same discipline as the model and the permissions it can exercise.
A few things that frame the scale:
- 53% of security leaders expect AI to run major portions of their infrastructure autonomously within the next three years, according to the 2026 Infrastructure Identity Survey.
- 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
A question worth separating out:
Q: Should organisations treat agent harness design as an authorisation decision?
A: Yes. The harness determines what the agent can reach, how long it can continue, and how tool calls are packaged across turns, so it directly affects effective privilege. That makes harness design part of authorisation governance, especially when the agent can act as a non-human identity with persistent runtime state.
👉 Read our full editorial: Agent harness choice materially changes autonomous red-teaming outcomes
This post was modified 4 hours ago by NHI Mgmt Group