What do security teams get wrong about agent harnesses?

Why This Matters for Security Teams

Agent harnesses define how an AI agent receives instructions, what it can call, how it recovers from failure, and which outputs are accepted as trustworthy. That makes the harness a governance boundary, not just a developer convenience. Security teams often focus on the model while missing the operational controls that determine whether the agent can be coerced, over-scoped, or quietly steered into unsafe actions. Guidance such as the NIST AI Risk Management Framework points to the need for measurable oversight across the full lifecycle, while the OWASP Agentic AI Top 10 highlights prompt injection, excessive agency, and tool misuse as concrete risks.

The practical failure is that a harness often inherits production trust without production-grade review. Teams may assume a clean system prompt and a limited tool list are enough, even though retries, schema repair, hidden defaults, and fallback paths can reopen risk. In practice, many security teams encounter harness weaknesses only after an agent has already taken an unsafe action through a permitted tool path, rather than through intentional control testing.

How It Works in Practice

A secure harness should be treated as the layer that constrains agent behaviour before any tool call is executed. That means documenting the agent’s allowed objectives, enforcing a narrow tool allowlist, validating inputs and outputs, and making retries deterministic rather than open-ended. The harness should also separate untrusted content from instructions so that retrieved text, user input, or downstream tool output cannot silently override the system’s policy.

In operational terms, teams should design the harness around failure containment. That usually includes:

• Explicit permission boundaries for each tool, including read versus write actions.
• Schema validation on prompts, tool arguments, and returned data before downstream use.
• Bounded retries and timeouts so the agent cannot loop into repeated risky calls.
• Human approval steps for high-impact actions such as deletion, payment, or privilege changes.
• Logging that preserves the instruction path, tool selection, and final decision for review.

This is also where agentic AI security overlaps with broader threat modelling. MITRE ATLAS adversarial AI threat matrix is useful for mapping attack techniques such as prompt manipulation and output exploitation, while the CSA MAESTRO agentic AI threat modeling framework helps teams reason about control placement across planning, execution, and oversight. When the harness governs an agent with access to internal systems, its permissions should be reviewed with the same discipline used for privileged automation, because the agent is effectively acting as an identity-bearing actor. These controls tend to break down when the harness depends on brittle prompt text, because prompt-only controls do not survive tool chaining, retries, or malformed output.

Common Variations and Edge Cases

Tighter harness controls often increase friction for developers and operators, requiring organisations to balance safety against speed of iteration. That tradeoff becomes sharper when the agent is expected to perform useful work across multiple tools, data sources, or business units. Best practice is evolving, but current guidance suggests that high-flexibility harnesses need stronger guardrails, not lighter review.

One common edge case is the “safe in test, unsafe in production” harness. A sandbox may expose dummy tools and clean inputs, while production adds real credentials, broader data access, or richer fallback behaviour. Another is retrieval-augmented workflows, where the model is technically unchanged but the harness now accepts untrusted retrieved content that can carry indirect prompt injection. Security teams should also watch for hidden escalation through retries, where a failed call is reissued with broader context or alternative parameters.

The most important exception is when the agent is allowed to act on behalf of a person or service account. At that point, the harness is not just an AI wrapper; it is part of identity and privilege governance. That is where NHI and PAM controls start to matter naturally, because the agent’s execution path can inherit standing access unless the harness enforces zero standing privilege and explicit session boundaries. For emerging agentic systems, there is no universal standard for this yet, which makes documented control ownership more important than vendor assurances.

Related resources from NHI Mgmt Group

• What do security teams get wrong about local-only agent controls?
• What do security teams get wrong about AI agent identity governance?
• What do security teams get wrong about AI agent scope control?
• What do security teams get wrong about agent authorization?