It fails at verification. A policy engine embedded in the agent can block some actions, but it cannot independently prove that every action was seen or stopped. If the agent changes code paths, spawns child processes, or is influenced through prompt injection, the control may report success without trustworthy evidence. That is why runtime enforcement needs an external observation plane.
Why This Matters for Security Teams
When governance lives only inside the agent process, the control path and the thing being controlled become the same trust boundary. That makes it hard to prove whether an action was actually evaluated, whether the evaluation was bypassed, or whether the agent simply reported a compliant outcome. Current guidance from the NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 points toward separation of duties, traceability, and external verification for this reason.
The practical risk is not only that an unsafe tool call occurs. It is that the organisation loses trustworthy evidence about decision-making, so incident response, audit, and containment all start from incomplete telemetry. Embedded governance can also be rewritten, shadowed, or skipped if the agent can alter its own execution path. That is especially important in agentic systems with tool use, memory, delegation, or chained prompts, where a single process may conceal several decision points.
In practice, many security teams discover the control gap only after an unexpected action has already been executed, rather than through intentional verification of agent behaviour.
How It Works in Practice
Effective agent governance needs two distinct layers: policy enforcement inside the agent workflow and independent observation outside it. The first layer can decide whether a request should proceed. The second layer records what actually happened, with enough context to support review, detection, and containment. That external layer is what gives security teams a verifiable trail when the agent uses tools, invokes sub-agents, or attempts a risky sequence of actions.
A useful design pattern is to treat the agent as an untrusted decision-maker and the surrounding platform as the source of evidence. Telemetry should capture prompts, tool calls, policy decisions, denied actions, and identity context for the workload or NHI involved. Where agents operate over sensitive data or privileged APIs, the governance plane should also preserve immutable logs and strong correlation identifiers so that SIEM and SOAR workflows can reconstruct the chain of events. This is consistent with the direction of the NIST Cybersecurity Framework 2.0 and the NIST SP 800-53 Rev 5 Security and Privacy Controls.
- Place policy decision points outside the agent runtime where possible.
- Log every tool invocation, denial, and policy override in an append-only control plane.
- Bind actions to workload identity, not just session state or prompt text.
- Use independent detectors to flag prompt injection, abnormal tool chaining, and policy drift.
- Test whether governance still works if the agent retries, forks, or spawns child processes.
This design aligns with threat modelling guidance from the MITRE ATLAS adversarial AI threat matrix and the CSA MAESTRO agentic AI threat modeling framework, both of which emphasise adversarial manipulation of AI behaviour and the need for control-plane visibility. These controls tend to break down when the agent can execute local code with broad filesystem or network access because policy checks and evidence generation are then exposed to the same runtime compromise.
Common Variations and Edge Cases
Tighter governance often increases latency, integration effort, and operational overhead, so organisations must balance stronger assurance against runtime simplicity. There is no universal standard for exactly how much enforcement belongs inside the agent versus outside it, but current guidance suggests the highest-risk decisions should be independently observable even if some low-risk checks remain embedded.
The edge cases are usually architectural rather than theoretical. Some teams rely on agent-side guardrails for convenience, then assume those logs are sufficient for audit. Others add a reverse proxy or orchestration layer but forget that the agent can still make side effects through cached credentials, delegated tokens, or unmonitored subprocesses. The risk is higher when the agent has long-lived memory, can call third-party tools, or can change its own prompt or code path. That is where the distinction between policy enforcement and evidence collection becomes critical.
Emerging practice is also evolving around how much to trust model outputs versus runtime logs. The safest approach is to require independent confirmation for high-impact actions, especially where a workflow can affect production systems, identity stores, or customer data. The NIST AI Risk Management Framework and the OWASP Top 10 for Agentic Applications 2026 both support this separation because trustworthy governance depends on independent verification, not self-attestation.
In environments with high automation, the hardest failures occur when everything appears healthy inside the agent while the surrounding platform has already lost the ability to prove what really happened.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATLAS address the attack and risk surface, while NIST AI RMF, NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | AI RMF stresses trustworthy, accountable AI governance and evidence of controls. | |
| OWASP Agentic AI Top 10 | Agentic AI risks include prompt injection, tool abuse, and self-bypassed controls. | |
| MITRE ATLAS | ATLAS maps adversarial techniques that can manipulate agent decisions and logs. | |
| NIST CSF 2.0 | DE.CM-1 | Continuous monitoring is needed when governance must be evidenced externally. |
| NIST SP 800-53 Rev 5 | AU-2 | Audit event capture is essential when self-reported agent controls are not trustworthy. |
Instrument external telemetry so agent actions are continuously monitored and logged.
