They were built mainly for policy, documentation, testing, and threat modeling. That helps teams understand risk, but it does not stop prompt injection, excessive tool use, or unauthorized actions during execution. In practice, the gap appears when governance exists on paper but request-level controls, logging, and filtering are missing where the model and agent actually run.
Why This Matters for Security Teams
AI security frameworks are useful, but many are still strongest at governance and weakest at runtime enforcement. That matters because live systems fail in the execution layer: a model can be approved, documented, and tested, yet still accept malicious prompts, over-call tools, or expose data through a weak integration path. Current guidance suggests pairing policy controls with request-level guardrails, logging, and decision traces, rather than assuming design-time review is enough. The NIST Cybersecurity Framework 2.0 is helpful here because it pushes teams to connect governance with continuous protection and detection outcomes.
The real issue is that AI operations are dynamic. Prompts change, retrieval sources change, tools change, and agent permissions often expand as teams add features. A framework can describe what good looks like, but it cannot by itself prevent a model from following an attacker’s instructions if the surrounding application trusts the wrong input. That is why practitioners increasingly treat ai runtime security as a control plane problem, not just a model risk problem. In practice, many security teams encounter this only after an agent has already taken an unsafe action or disclosed sensitive context, rather than through intentional runtime testing.
How It Works in Practice
Live AI operations usually sit across multiple layers: the model, the orchestration layer, the retrieval system, tool connectors, and the identity and access controls that govern what the agent can do. Frameworks such as the CSA MAESTRO agentic AI threat modeling framework help teams reason about those layers, but implementation still depends on engineering choices. The practical challenge is that many controls are applied before launch, while the highest-risk failures happen during live inference and tool execution.
- Validate prompts and retrieved content before they reach the model, especially where external or user-supplied text can trigger prompt injection.
- Constrain tool use with explicit authorization, narrow scopes, and step-up checks for sensitive actions.
- Log prompts, tool calls, policy decisions, and outputs so investigators can reconstruct what happened after an incident.
- Filter or redact sensitive data before it enters the context window, because the model cannot protect what it is allowed to see.
- Test the full execution path, not just the model in isolation, because failures often emerge in orchestration and plugin layers.
That operational view aligns with agentic AI guidance emerging from sources such as Anthropic Project Glasswing, which reflects the growing recognition that agent security requires runtime containment as much as model evaluation. For governance teams, the key question is whether an AI system can still be made to act outside policy once it is live, not whether the documentation looks complete. These controls tend to break down when agents are granted broad tool access in fast-moving product environments because permission design lags behind feature delivery.
Common Variations and Edge Cases
Tighter runtime control often increases latency, engineering overhead, and false positives, requiring organisations to balance safety against user experience and operational speed. Best practice is evolving, and there is no universal standard for how much control belongs in the model layer versus the application layer versus the identity layer. In some environments, especially customer-facing copilots, aggressive filtering can degrade usefulness. In others, especially internal agents with access to secrets or production systems, permissive design is the more serious risk.
The edge cases matter. RAG systems can fail when trusted retrieval sources are poisoned or stale. Multi-agent workflows can fail when one agent passes unsafe instructions to another. Human approval steps can also become cosmetic if reviewers are given too little context to judge a request. Where the question touches identity, the gap often widens further: if an agent inherits a user session, service account, or privileged token without true least privilege, the framework may describe the risk, but the runtime still allows it. For that reason, live AI operations need continuous control validation, not one-time certification, and organisations should treat the AI execution layer as part of the security boundary, not a separate innovation zone.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATLAS, OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | GOVERN | Governance is where most AI frameworks start, but runtime gaps appear when it stops there. |
| MITRE ATLAS | T1595 | Threat modeling must include active probing and adversarial testing of AI systems. |
| OWASP Agentic AI Top 10 | Agentic AI risks center on tool misuse, prompt injection, and unsafe autonomous action. | |
| NIST CSF 2.0 | PR.PT | Protective technology must operate at runtime, not only in documentation and design. |
| CSA MAESTRO | MAESTRO addresses agentic AI threat modeling across orchestration and control layers. |
Implement technical controls that constrain, monitor, and validate AI execution paths continuously.
