Traditional application security focuses on code flaws, configuration errors, and known infrastructure risks. LLM security adds risks tied to model behaviour, training data, prompt handling, and supply chain integrity. Security teams still need conventional controls, but they also need defenses for prompt injection, jailbreaks, overreliance, model theft, and the governance of AI-specific data and tooling.
Why the Security Model Changes for LLMs
Traditional application security assumes the application is mostly deterministic: you can test inputs, validate outputs, patch code, and lock down infrastructure. LLMs add a probabilistic layer that can change behavior without any code change, so the security question expands from “is the software vulnerable?” to “can the model be steered, confused, or induced to act outside its intended bounds?”
That shift matters because the attack surface is no longer limited to the app’s code and runtime. It also includes prompts, retrieval sources, fine-tuning data, tool calls, model endpoints, and the training or dependency chain that feeds the model. For a practical reference point on model-specific risk patterns, see OWASP Top 10 for Agentic Applications 2026 and NIST AI 600-1 Generative AI Profile.
The difference is not that conventional controls stop mattering. Input validation, secrets handling, access control, logging, dependency review, and secure deployment remain foundational. The difference is that LLM systems can fail in ways that look correct to the user while still being unsafe, which means assurance has to cover behavior, not just code paths.
What LLM Security Adds Beyond Conventional AppSec
Traditional appsec focuses on injection, broken authorization, misconfiguration, and vulnerable dependencies. LLM security adds prompt injection, jailbreaks, hallucination-driven misuse, overreliance on model output, model theft, and data leakage through prompts or retrieved context. It also adds new trust decisions around which data the model may see, which tools it may call, and what actions it can trigger.
That is why LLM security often becomes a layered discipline: protect the application, then protect the model interaction layer, then protect the surrounding ecosystem. Supply chain integrity matters more because the model may inherit risk from datasets, embeddings, plugins, connectors, hosted model services, and third-party copilots. A useful example of this broader attack surface is LiteLLM PyPI package breach, which illustrates how dependency compromise can become a model-adjacent security issue.
Practitioners should also think about output trust differently. In conventional software, a bad response usually means a defect. In LLM systems, a fluent response may still be unsafe, incomplete, or manipulated. The control problem is therefore partly about limiting authority, constraining tool access, and validating model-driven decisions before they become real-world actions.
Risk and Threat Considerations
LLM systems widen the failure modes that defenders must consider. A prompt can alter behavior, retrieved content can smuggle hostile instructions, and a compromised dependency or connector can turn a model into an access path for data exposure or unauthorized action. The risk is highest when teams treat the model as a trusted decision-maker instead of a component whose outputs and actions must be bounded.
Failure mechanism: Attackers exploit instruction-following behavior, weak separation between data and commands, overbroad tool permissions, or unsafe third-party integrations to steer the model into leaking information, taking unwanted actions, or reinforcing false outputs.
Impact: The result can be data disclosure, fraudulent or destructive actions, model or prompt theft, business logic abuse, and a larger blast radius than a standard application defect because the model may execute across multiple systems at once.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST AI RMF, NIST AI 600-1 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | AI Risk Management Framework | Directly governs AI risk, trust, and lifecycle issues in LLM systems. |
| Recommendation — Apply AI RMF functions to identify, measure, and govern LLM-specific risks across the model lifecycle. | ||
| NIST AI 600-1 | Generative AI Profile | Addresses generative AI governance, provenance, testing, and incident handling. |
| Recommendation — Use the GenAI profile to structure testing, provenance checks, and incident response for LLM deployments. | ||
| OWASP Agentic AI Top 10 | OWASP Agentic AI Top 10 | Covers prompt injection, tool misuse, and AI-specific attack surfaces central to LLM security. |
| Recommendation — Map LLM threats to agentic risks and harden prompt, tool, and autonomy boundaries. | ||
| CIS Controls v8 | CIS 16 — Application Software Security | Supports secure development and dependency review for LLM-enabled applications. |
| Recommendation — Apply secure development and dependency controls to LLM applications and their integration points. | ||
| MITRE ATT&CK | T1190 — Exploit Public-Facing Application | LLM-facing services can be abused through exposed endpoints and application-layer exploitation. |
| Recommendation — Hunt exposed LLM services as public-facing applications and validate their attack surface. | ||
Practitioner Guidance
What to prioritize: Treat prompt and tool boundaries as control points, not just user experience features. If an LLM can access production data, send messages, approve actions, or call external tools, those capabilities need explicit authorization, logging, and rollback paths.
What to verify: Check whether the model can be influenced by untrusted text, whether retrieved content is isolated from instructions, and whether tool invocations are constrained by policy rather than model judgment alone. Also verify that training and fine-tuning sources are vetted for secret exposure and poisoning risk.
Practitioner takeaway: Traditional appsec answers “is the system technically secure?”, while LLM security also asks “can the system be socially or contextually manipulated into doing the wrong thing?” That distinction should drive testing, access design, and governance.
Related resources from NHI Mgmt Group
- What is the difference between developer-centric application security and traditional application security programs?
- What is the difference between shift left application security and traditional late-stage testing?
- What is the difference between a developer-first AppSec platform and a traditional enterprise application security suite?
- What is the difference between agentic identity management and traditional IAM in cloud and application security?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 20, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
