LLM applications change the review model because the model can launder untrusted input into output that appears safe but still drives a privileged action. In traditional apps, scanners can often rely on sanitisation patterns. With LLMs, reviewers must understand prompt sources, tool access, and downstream impact, because the model itself becomes part of the injection path.
Why LLM Applications Change the Security Review Model
LLM applications do not behave like ordinary deterministic web apps, because the model can transform untrusted input into natural language or structured output that appears trustworthy even when it is not. That changes the review problem from simple input validation to source tracing, tool authorization, and downstream effect analysis. Reviewers have to ask not only what is displayed, but what the model can cause other systems to do.
This matters because the security boundary is no longer just the browser, API gateway, or application server. The model may sit between user content and privileged actions such as search, ticket creation, email sending, code generation, or workflow execution. OWASP Agentic AI Top 10 is useful here because it frames agentic and tool-using systems around misuse of authority, not just classic web injection. In practice, many security teams discover the problem only after a benign-looking prompt has already influenced a privileged workflow.
Traditional review heuristics still matter, but they are no longer sufficient. Sanitisation, allowlists, and WAF coverage can reduce obvious injection paths, yet they do not fully explain model-mediated behaviour. The reviewer must understand where prompts originate, whether retrieval content is trusted, how the model composes instructions, and whether outputs are merely informational or action-bearing. The key difference is that the application can be secure at the HTTP layer and still unsafe at the decision layer.
How LLM Workflows Actually Introduce New Review Steps
LLM applications add review questions that ordinary web applications rarely create. First, the prompt itself becomes an input surface, and that prompt may be assembled from user text, retrieved documents, prior conversation, system instructions, and hidden templates. Second, the model may call tools or APIs on behalf of the user, which means the review must cover the authority of each tool and the conditions under which it can be triggered. Third, the output can be consumed by another system, so the real security question is often whether the output is safe to trust as a decision input rather than whether it merely looks clean.
That is why a useful review usually maps the flow from source to model to tool to impact. A team should identify:
- which inputs are untrusted and can influence the model;
- which instructions are higher priority than user content;
- which tools are reachable from the model;
- which outputs are advisory versus executable;
- what validation happens after the model responds.
Where an ordinary app can often be reviewed with familiar application security controls, an LLM app also needs review of instruction hierarchy, retrieval trust, and action gating. NIST’s generative AI guidance helps here because it emphasises AI-specific risk management rather than treating the system as a standard form handler. NIST AI 600-1 Generative AI Profile is especially relevant when the system blends generation with downstream enterprise decisions.
The practical consequence is that security reviewers cannot stop at “is the input escaped?” They must ask whether the model can be induced to ignore intended policy, whether retrieval can inject misleading context, and whether a seemingly harmless reply can trigger real-world actions. This guidance breaks down when the LLM is only used for narrow, offline text assistance with no tool access and no privileged downstream effect.
Where the Standard Web App Review Model Breaks Down
Tighter control over LLM behaviour often increases design and review overhead, requiring organisations to balance simplicity against the risk of hidden action paths. One genuine tradeoff is that the more autonomy you give the model, the less reliable ordinary per-request testing becomes.
There are also edge cases where the answer is less about “LLMs are dangerous” and more about “the deployment pattern changes the review scope.” A read-only summarisation tool may still need prompt-injection testing, but the risk profile is very different from an agent that can open tickets or execute code. The consensus is strong that tool-using systems need deeper review than chat-only systems, but there is not complete consensus on where to draw the boundary between prompt injection testing and broader application security review.
Another common misconception is that a strong model provider removes the need for review. It does not. Provider safeguards can reduce some classes of abuse, but the application owner still controls retrieval sources, tool permissions, session handling, and the business logic that decides whether model output becomes an action. That is why cross-functional review matters: the security team, product owner, and platform engineer each hold part of the evidence needed to judge whether the system is merely informative or functionally autonomous.
In practice, many review failures occur when teams assess the model in isolation rather than the whole action chain from prompt to privilege.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATLAS address the attack and risk surface, while NIST AI 600-1, NIST AI RMF and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A2 — Prompt Injection | Directly addresses model-mediated injection and tool abuse. |
| Recommendation — Test prompts and tool flows for injection paths that can alter actions. | ||
| NIST AI 600-1 | MAP — Measure, Assess, and Manage Risks | Fits AI-specific risk review across prompts, retrieval, and outputs. |
| Recommendation — Assess generative AI workflows for misuse, failure, and downstream impact. | ||
| NIST AI RMF | GOVERN — Govern | Applies to organisational AI governance and accountability for LLM use. |
| Recommendation — Establish accountability for how LLM outputs are trusted and used. | ||
| MITRE ATLAS | AML.TA0003 — Evasion | Relevant where adversaries shape model behaviour to bypass controls. |
| Recommendation — Map adversarial prompt patterns to ATLAS techniques and detect evasion. | ||
| CIS Controls v8 | 6 — Access Control Management | Relevant to limiting which tools and actions the model can reach. |
| Recommendation — Restrict model-accessible actions to the minimum necessary authority. | ||
Practitioner Guidance
What to prioritise: Review the highest-impact action path first, not the most visible prompt path. If the model can trigger external actions, the review should focus on tool authority, escalation points, and whether the output can be trusted as an instruction.
What to verify: Confirm which content is untrusted, which instructions are privileged, and where output is consumed by another system. If reviewers cannot trace that chain clearly, they do not yet have a reliable security decision.
Decision rule: Treat a chat-only assistant and an agentic workflow as different security classes. If the model cannot act, the main risk is misleading content; if it can act, the main risk becomes unauthorised execution through manipulated context or output.
Practitioner takeaway: The most important shift is to review LLM applications as instruction-and-action systems, not as text-only interfaces, because the real security question is where model output gains authority.
Related resources from NHI Mgmt Group
- Why do AI agents create a different red teaming problem from ordinary AI applications?
- Why do agentic AI systems create a different security problem from static applications?
- Why do AI agents create a larger security risk than ordinary web applications?
- Why do 3D CAD models create a different data security problem than ordinary project documents?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 7, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
