An application design in which models, agents or LLM-enabled features are part of the runtime architecture, not just the development process. This expands the attack surface because AI components may fetch data, call tools or interact with other systems as active participants in the application.
What AI-in-the-Stack Means in Practice
AI-in-the-Stack describes an application architecture where models, agents, or LLM-enabled features are runtime participants, not just offline development aids. That distinction matters because the AI component can actively influence data flow, system behaviour, and downstream decisions.
The term is useful when the AI feature is inside the request path, orchestration layer, or decision path of the application. If the model only helps developers build the system, it is not AI-in-the-Stack in this sense.
How It Changes the Application Boundary
Once AI becomes part of the runtime, the boundary of the application expands beyond traditional code, APIs, and databases. The system may now depend on prompts, model outputs, retrieval sources, tool permissions, and agent instructions as live inputs to production behaviour.
That changes how architects should think about trust. A model that can fetch data or call tools is not a passive component, it can become a mediator of business logic and a new place where authorization, data handling, and execution control need to be defined carefully.
Why AI-in-the-Stack Expands Security Exposure
AI-in-the-Stack introduces risks because the AI layer can be influenced, misled, or over-empowered while still appearing to operate normally. A malicious prompt, poisoned context, unsafe tool call, or weakly governed integration can turn an ordinary feature into a path for data exposure or unwanted actions.
The risk is not limited to model quality. It also comes from how the application lets AI touch data, invoke functions, and interact with other systems, especially when those interactions are broader than the original user request.
When this pattern is present, practitioners should also account for access paths and control boundaries using OWASP API Security Top 10, because AI features often rely on APIs to reach data and execute actions. Runtime trust decisions also benefit from NIST Cybersecurity Framework 2.0, which helps organise governance, protection, detection, response, and recovery around the expanded attack surface.
Design and Governance Implications
AI-in-the-Stack is not just an engineering pattern, it is also a governance decision about what the AI is allowed to do. The organisation needs clarity on which actions are advisory, which are automated, which require human review, and which systems or data sources the AI can reach.
That separation matters because runtime AI can create policy drift if teams treat it as a normal feature toggle. Good design keeps the AI component observable, bounded, and aligned to the same trust and change-control expectations as any other production dependency.
For AI-specific governance, NIST AI Risk Management Framework is a strong fit for structuring risk thinking around valid use, governance, and measurement. Where the application uses autonomous or multi-step agent behaviour, OWASP Agentic AI Top 10 directly addresses tool misuse, identity and privilege abuse, and other runtime failure modes.
Risk and Threat Considerations
AI-in-the-Stack increases exposure because the AI layer may be able to shape decisions, retrieve sensitive data, or trigger actions across systems. The main threat is not that the model is “intelligent”, it is that it can be manipulated into using legitimate access in unsafe ways.
Failure mechanism: Prompt injection, context poisoning, excessive tool permissions, or weak output validation can cause the AI layer to act on attacker-influenced instructions or to cross intended trust boundaries.
Impact: The result can include unauthorized data access, harmful automation, privilege misuse, business logic abuse, or broader compromise of connected systems.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP API Security Top 10 and OWASP Agentic AI Top 10 address the attack and risk surface, while NIST CSF 2.0 and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP API Security Top 10 | API5 — Broken Function Level Authorization | AI runtime tool calls can expose function-level access boundaries. |
| Recommendation — Enforce function-level authorization on every AI-triggered action path. | ||
| NIST CSF 2.0 | PR.AA-05 — Least Privilege and Permissions Management | AI-in-the-Stack expands runtime access and privilege decisions. |
| Recommendation — Restrict AI components to the minimum permissions needed for runtime duties. | ||
| NIST AI RMF | Govern | AI-in-the-Stack is a governance decision about runtime AI use. |
| Recommendation — Define ownership, risk acceptance, and oversight for AI at runtime. | ||
| OWASP Agentic AI Top 10 | ASI02 — Tool Misuse | Runtime agents may call tools in ways that change system behaviour. |
| ASI03 — Identity & Privilege Abuse | Runtime AI can misuse delegated authority and access. | |
| Recommendation — Constrain and verify every tool action an agent can invoke. Bind agent actions to tightly scoped identities and privileges. | ||
Practitioner Guidance
What to watch for: Treat any AI component that can read production context, call tools, or initiate downstream actions as part of the application’s control plane, not as decorative intelligence. The key question is whether the feature can change system state or expose data in ways the rest of the application must now govern.
Practitioner takeaway: If the AI can act at runtime, it needs explicit boundaries, explicit authorization, and explicit monitoring, just like any other privileged production component.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on October 12, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org