Vulnerabilities in LLMs

How LLM vulnerabilities arise

LLMs are vulnerable when the model’s inputs, training data, retrieval sources, tools, or outputs can be influenced in ways the system does not properly control. The weakness is not the model alone, but the surrounding application design that lets untrusted text, data, or actions cross trust boundaries.

This is why prompt injection, poisoning, extraction, and unsafe tool execution are usually discussed together. They are different failure modes, but they share a common property: the system treats model-adjacent content as if it were safe enough to steer behavior.

In practice, the risk grows when an LLM is embedded into search, chat, coding, customer support, or workflow automation without strong permission checks. If the model can see data it should not, call tools it should not, or emit actions that downstream systems accept automatically, the vulnerability becomes operational rather than theoretical.

Common vulnerability classes in LLM systems

Prompt injection is the most visible class. It occurs when attacker-controlled text changes the model’s instructions, causes policy bypass, or tricks the system into revealing hidden context. Indirect prompt injection is especially dangerous because the malicious instructions can arrive through documents, web pages, emails, or retrieval pipelines that the application assumes are benign.

Data poisoning is the corruption of training, fine-tuning, retrieval, or memory data so the model learns or repeats malicious behavior. Model extraction is the theft of model behavior, weights, or proprietary outputs through repeated querying or abuse of exposed interfaces. Unsafe output handling turns ordinary model text into a security problem when the application executes, trusts, or republishes that text without validation.

Tool and connector abuse deserves separate attention because it turns language into action. When an LLM can search, send messages, query systems, or trigger code, the attack surface expands from content manipulation to delegated execution. NHIMG’s Agentic AI Security Guide is useful here because it frames inputs, memory, tools, orchestration, and identity as one attack surface.

Why these weaknesses are dangerous

LLM vulnerabilities matter because they can expose both data and authority. A compromised model workflow may leak prompts, secrets, or retrieved records, but it can also persuade a downstream system to take an action the user never intended. That is why security issues in LLMs often become authorization, privacy, and trust issues at the same time.

The danger increases when the model sits near credentials, sessions, APIs, or administrative workflows. An attacker does not always need to break the model itself; it is often enough to corrupt the context around it, abuse a tool chain, or exploit over-trusting automation. NHIMG’s Permission-Aware RAG Guide shows why retrieval-time permissioning matters when LLMs can surface data the caller should never have seen.

LLM vulnerabilities also create governance risk because they blur responsibility. If a model outputs harmful advice, leaks a secret, or triggers an external action, teams must determine whether the failure came from data handling, model configuration, tool authorization, or downstream process design. The security problem is therefore systemic, not just model-specific.

How organisations reduce exposure and detect abuse

Exposure assessment starts with inventory. Teams need to know which LLMs are deployed, what data they can access, which tools they can call, and where their prompts, memory, logs, and outputs are stored. Without that map, prompt injection and data leakage are hard to contain because the attack path is spread across application layers.

Detection should focus on abnormal prompt content, unusual tool calls, unexpected retrieval patterns, and repeated attempts to elicit hidden instructions or sensitive context. Post-incident review should include logs, memory stores, retrieval indexes, connected APIs, and any automation that consumed model output. NHIMG’s AI Agent Memory Security Guide is a useful companion for understanding how memory design can create cross-session leakage and poisoning paths.

For broader control alignment, the relevant external references are the NIST AI 600-1 GenAI Profile, the NIST AI Risk Management Framework, and the OWASP Agentic AI Top 10, which all help translate LLM failure modes into governance and testing requirements.

Practitioner Guidance

Why practitioners should care: Treat LLMs as privileged application components, not passive text engines. The fastest way to reduce risk is to limit what the model can read, what it can call, and what the application will trust from its output.

What to watch for: Be especially cautious when retrieval, tools, or memory can cross trust boundaries without explicit permission checks. If an LLM can influence external systems, review the full chain from prompt ingestion to downstream action, not just the model response.

Practitioner takeaway: The safest LLM designs assume the model can be manipulated, so security must come from isolation, validation, and tightly governed tool access rather than from model behavior alone.

Related resources from NHI Mgmt Group

• Why do raw LLMs miss so many vulnerabilities in code review?
• Why do general-purpose LLMs miss vulnerabilities that human testers still find?
• How should security teams use LLMs to find vulnerabilities in large codebases?
• What breaks when organisations rely on LLMs alone to find OWASP Top 10 vulnerabilities?