LLM Security Risks are the ways large language models can be misused, manipulated, or exposed to cause harm. They include prompt injection, data leakage, unsafe tool use, model theft, and harmful outputs. From an identity security view, they often arise when agents, NHI credentials, or permissions are not tightly governed.
What LLM Security Risks Actually Cover
LLM security risks are not limited to one failure mode. They include direct prompt attacks, leakage of sensitive context, unsafe actions taken by connected tools, and abuse of model outputs that persuade, reveal, or automate harm.
The term is broad because the model, its surrounding application, and the data it can reach all create different exposure paths. A secure design has to consider the model’s inputs, outputs, memory, tool access, and the permissions inherited from adjacent systems.
Why These Risks Matter in Real Deployments
The danger is not only that an LLM may say something wrong. A compromised or poorly governed LLM can expose private data, reveal system prompts, trigger unwanted actions, or amplify mistakes across many users and workflows.
That is why the most serious incidents often involve the surrounding stack, not just the model itself. LLM provider API key security and LLMjacking shows how stolen credentials can turn a model service into an abuse channel, while permission-aware RAG shows how retrieval over-sharing becomes a data exposure problem.
LLM risk therefore scales with privilege, integration depth, and trust. The more the model can read, write, or invoke, the more a small weakness can become an organisation-wide security issue.
Common Failure Modes and Attack Paths
Prompt injection and indirect prompt injection can steer the model away from policy, cause it to reveal hidden context, or make it misuse connected tools. Data leakage can happen through prompts, retrieved documents, logs, memory, or generated responses that echo sensitive material.
Other failure paths are more operational. Unsafe tool use can cause the model to send emails, alter records, or query systems without the right guardrails, while model theft and training data exposure create intellectual property, privacy, and governance concerns.
These patterns are especially dangerous when the model is allowed to act as a proxy for a person or workflow. Agentic AI Security Guide is useful here because it frames the risks around inputs, memory, tools, orchestration, and identity rather than treating the model as a standalone component.
What Security Teams Should Check First
Start by identifying where the model is deployed, what data it can see, and which tools or APIs it can reach. Then verify whether prompts, connectors, retrieval sources, and logs can expose sensitive material or expand the model’s effective permissions.
Review whether the application has clear boundaries for system prompts, memory, file access, and outbound actions. For deeper technical and governance context, NIST AI 600-1 GenAI Profile and the NIST AI Risk Management Framework help structure risk review, testing, and incident handling around generative AI systems.
When the deployment uses agents or tool chains, a dedicated threat model matters. MITRE ATLAS adversarial AI threat matrix and CSA MAESTRO agentic AI threat modeling framework are both useful for mapping how adversarial behaviour, tool misuse, and context poisoning show up in practice.
Risk and Threat Considerations
LLM security risks become material when the model is connected to sensitive data, external tools, or privileged workflows. In that state, a single injection, leaked key, or unsafe output can turn a language interface into a control plane for data exposure or unauthorized action.
Failure mechanism: Attackers or users exploit the model’s trust in prompts, context, retrieved data, or tool calls to override intended boundaries, exfiltrate secrets, or trigger actions the operator did not mean to permit.
Impact: The result can be confidential data loss, account or key abuse, corrupted decisions, harmful content at scale, and in agentic environments, wider compromise through inherited permissions or chained actions.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATLAS and OWASP API Security Top 10 address the attack and risk surface, while NIST AI 600-1, NIST AI RMF and CSA Cloud Controls Matrix set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI 600-1 | N/A — Generative AI Profile | Addresses generative AI governance, testing, provenance, and incident handling for LLM risk. |
| Recommendation — Use the GenAI profile to structure testing, disclosure, and risk treatment for deployed LLM systems. | ||
| NIST AI RMF | N/A — AI Risk Management Framework | Defines AI risk governance and lifecycle risk management for systems that include LLMs. |
| Recommendation — Apply the AI RMF to govern, measure, and manage LLM risk across the lifecycle. | ||
| MITRE ATLAS | N/A — Adversarial Threat Landscape for Artificial-Intelligence Systems | Catalogs prompt injection, poisoning, tool misuse, and other adversarial AI techniques relevant to LLM threats. |
| Recommendation — Map observed abuse patterns to ATLAS techniques and test detections against them. | ||
| CSA Cloud Controls Matrix | IAM — Identity and Access Management | LLM risk often depends on controlling who and what can access data, tools, and model services. |
| Recommendation — Restrict model, connector, and tool access to the minimum permissions required. | ||
| OWASP API Security Top 10 | API5 — Broken Function Level Authorization | LLM applications often expose tool and action endpoints that fail when model-driven calls lack proper authorization. |
| Recommendation — Authorize every model-invoked action independently of the prompt or caller context. | ||
Practitioner Guidance
Why practitioners should care: The main governance mistake is treating LLM safety as a content problem only. Security teams should instead review the full execution path, including data access, tool authority, and post-processing, because those are often the real sources of harm.
What to watch for: Repeated prompt anomalies, unexpected tool calls, unusually broad retrieval results, and any place where the model can reveal or reuse secrets are strong signals that the deployment needs tighter controls or redesign.
Practitioner takeaway: If the model can read it, retrieve it, or act on it, assume it can also expose it unless permissions, isolation, and output handling are explicitly constrained.
Related resources from NHI Mgmt Group
- Guide: LLM Security Guide
- What security risks remain after fine-tuning an LLM?
- How should security teams scan LLM application code for prompt injection risks in pull requests?
- Why do AI and LLM applications create security risks that traditional tools often miss?
- How should security teams review LLM and agent code for prompt injection risks in production workflows?
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on October 24, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org