What fails when LLMs are protected only with traditional application security controls?

Why Traditional AppSec Controls Break on LLM Behavior

Traditional application security assumes inputs are either clean or malicious in a way that can be checked before execution. LLMs do not behave that way. They interpret instructions, retrieved content, user text, and tool output inside one shared context, so the security boundary is no longer just the request payload or the API edge.

That means controls built for static validation, pattern matching, or fixed protocol rules can miss the real failure mode. The dangerous instruction may arrive through a document, a search result, a chat message, or even a tool response, and the model may treat it as actionable context rather than hostile data.

This is why LLM security needs to be treated as a runtime governance problem, not only a web or API hygiene problem. A useful framing is the shift from blocking bad strings to governing what the model is allowed to see, remember, retrieve, and act on, which is exactly the kind of control posture discussed in the OWASP Agentic AI Top 10 and the NIST AI Risk Management Framework.

Which Defences Stop Working First

The first controls to fail are usually the ones that assume the model can cleanly separate trusted application logic from untrusted content. Signatures and WAF rules are weak against prompt injection because the attack is often semantic, not syntactic. The model can be persuaded to follow malicious instructions even when no classic exploit pattern is present.

Ordinary DLP also struggles when the model generates the sensitive text itself or exfiltrates information by summarising, transforming, or selectively revealing it. Retrieval poisoning creates the same problem upstream: the source may look legitimate to the pipeline, but the content can still manipulate the model’s behaviour once retrieved. The practical lesson is that retrieval sources, prompt templates, memory, and tool outputs all become security-relevant surfaces.

That is why appsec checklists that focus only on input validation, authentication, and transport security are incomplete for LLM systems. For testing and control design, the relevant question is whether the model can be misled after it has already accepted apparently valid context. The OWASP ASVS still helps for the surrounding application, but it does not by itself address the model’s blended trust boundary.

What a Correct Control Model Looks Like

The right control model treats prompts, outputs, retrieval, and tools as governed channels with different trust levels. The model should not get unrestricted access to every document, every connector, or every action just because the surrounding app is authenticated. Access needs to be scoped, observable, and bounded by the actual decision being delegated to the model.

In practice, that means separating retrieval permissions from generation permissions, constraining tool invocation, and recording enough context to reconstruct why the model acted as it did. If a system lets the model fetch documents, call APIs, or trigger workflows, then those actions need explicit authorization and reviewable policy, not just a hidden prompt instruction. NHIMG’s Permission-Aware RAG Guide is a useful reference for making retrieval respect user access, while the Enterprise AI Copilot Security Guide shows how over-sharing and connector sprawl become practical exposure points.

Where the system has agent-like behaviour, the control problem expands further. Tool chaining, memory, and delegated action can turn a simple prompt issue into privilege abuse or unintended execution, which is why agent-specific guidance such as the Agentic AI Security Guide matters when the model can actually act on behalf of a user or service.

Risk and Threat Considerations

LLM systems create a larger blast radius than conventional app inputs because the model can convert untrusted text into decisions, actions, or disclosures after the original request has passed. That makes prompt injection, retrieval poisoning, and tool abuse especially dangerous in environments where the model can reach sensitive data or operational systems.

Failure mechanism: The attacker or malicious content reaches a trusted context channel, then changes model behaviour indirectly by shaping what the model believes is instruction, evidence, or required action. Static filters miss this because the harmful content is not always obviously malicious at the point of entry.

Impact: The result can be secret leakage, unsafe actions, policy bypass, or data exposure through generated output, retrieval, or tool calls. In systems with delegated execution, the model can become the path from weak text filtering to real operational compromise.

Practitioner Guidance

What to prioritise: Put policy around retrieval scope, tool permissions, and output handling before tuning prompt filters. If the model can see or do too much, the control problem is already lost.

What to verify: Confirm that every connector, vector store, and tool endpoint has an explicit trust boundary and an audit trail. If you cannot explain why the model was allowed to see a source or invoke an action, the control is incomplete.

Common mistake: Treating LLM security as a content-moderation problem. The real issue is contextual authority, not just bad text.

Practitioner takeaway: Traditional appsec remains necessary, but it is only the outer shell; the decisive control layer for LLMs is governance over context, retrieval, and delegated action.

Related resources from NHI Mgmt Group

• Why do LLMs complicate traditional application security assumptions in production systems?
• Why do AI applications expand the application security problem beyond traditional controls?
• What happens when cloud workloads are protected only with traditional security controls?
• What is the difference between embedding security into application runtime and relying on traditional build-time or container security controls?