Model security focuses on the weights, training data, and generation behavior. Application stack security covers the system that wraps the model, including authentication, authorization, logging, prompt handling, retrieval connectors, and output moderation. In practice, most real incidents involve the application layer, where injected instructions and overbroad access are easier to exploit.
Why This Matters for Security Teams
Model security and application stack security are different control problems, and confusing them leaves the real attack surface unprotected. The model can be hardened against unsafe outputs or training-data issues, but the surrounding application still decides who can reach it, what data it can retrieve, and what tools it can invoke. That is where prompt injection, secret exposure, overbroad permissions, and unsafe connectors usually land.
For security teams, the distinction matters because most production failures happen around orchestration, not inside the weights. NIST’s NIST AI Risk Management Framework treats governance, mapping, and measurement as separate from model development, which mirrors how practitioners should split responsibilities. NHIMG research on OWASP Agentic Applications Top 10 also shows that application-layer issues dominate once models are connected to tools, retrieval, and identity systems.
In practice, many security teams encounter model concern only after an application-layer abuse path has already exposed data or invoked an unintended action.
How It Works in Practice
Securing the model means reducing risks intrinsic to the model itself: unsafe completions, prompt sensitivity, training-data leakage, and model tampering. Securing the application stack means protecting the entire workflow that wraps the model, including authentication, authorization, prompt assembly, retrieval-augmented generation, logging, rate limiting, and output filtering. The model may be the reasoning engine, but the stack determines whether that reasoning can be misused.
That separation is why current guidance suggests treating the application as the primary enforcement point. A model can only be trusted to the extent that the surrounding system constrains what it sees and what it can do. Frameworks like the OWASP Top 10 for Agentic Applications 2026 and CSA MAESTRO agentic AI threat modeling framework both emphasize controls around orchestration, tool access, and data pathways rather than model internals alone.
Practical controls usually include:
- Strong identity and access control for users, agents, and service accounts.
- Prompt handling that sanitizes untrusted inputs and separates system instructions from user content.
- Connector governance for search, ticketing, code, and document systems.
- Secret isolation so the model never receives long-lived credentials directly.
- Logging and monitoring for prompts, tool calls, retrieval results, and abnormal output patterns.
NHIMG’s analysis of the McKinsey AI platform breach illustrates the point: the system exposure came from the surrounding platform, not a failure of the model’s parameters. These controls tend to break down when multiple plugins, retrieval sources, and automation hooks share the same trust boundary because one weak integration becomes a path into the whole stack.
Common Variations and Edge Cases
Tighter model controls often increase latency and operational overhead, requiring organisations to balance safety against developer velocity and user experience. That tradeoff becomes sharper in regulated environments, where teams may need stronger logging, retention, and review processes even when the model itself is externally hosted.
There is no universal standard for this yet, but best practice is evolving toward layered responsibility. Some teams secure only the model vendor boundary, while others treat every connector, retrieval index, and agent tool as a separate attack surface. The second approach is usually more defensible because application security failures are easier to exploit than model-layer weaknesses.
Edge cases matter. If the system is a simple chat interface with no tools, the model layer carries more of the risk. If the model can search internal documents, call APIs, or act autonomously, the application stack becomes the real security boundary. That is also why AI Agents: The New Attack Surface report is so relevant: it shows how quickly access drift and unauthorized data use emerge once agentic behavior is introduced.
For most production deployments, the safe assumption is simple: the model needs guardrails, but the stack needs policy, identity, and monitoring. If the application can hand the model secrets, retrieve sensitive data, or execute tools without independent checks, the model security work alone will not stop abuse.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, CSA MAESTRO and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A2 | Application-layer abuse and tool misuse are core agentic risks. |
| CSA MAESTRO | TRM | MAESTRO separates model risk from orchestration and trust boundaries. |
| NIST AI RMF | GV | AI RMF governance clarifies responsibility across model and application layers. |
| OWASP Non-Human Identity Top 10 | NHI-01 | Stack security depends on protecting non-human identities and secrets. |
| NIST CSF 2.0 | PR.AC-4 | Least privilege is essential for the surrounding application stack. |
Map prompts, tools, and connectors to agentic abuse scenarios and enforce controls at each runtime decision point.
Related resources from NHI Mgmt Group
- What is the difference between securing AI agents and securing the surrounding data security stack?
- What is the difference between securing an AI model and securing an MCP-enabled agent?
- What is the difference between securing an AI model and securing an AI agent?
- What is the difference between securing the model and securing the skill layer?
