Why do LLMs create new risk even when existing security tools are in place?

Why LLMs Outgrow Traditional Security Assumptions

LLMs create a different security problem because the thing being protected is no longer a fixed execution path. Their outputs vary with prompt, context, memory, data retrieval, and tool access, so the same request can lead to different behaviour at different times. Existing controls still matter, but they were not designed to govern a system whose responses are probabilistic and adaptive.

That matters because conventional controls assume predictable inputs and outputs. A firewall, static rule set, or content filter can still reduce exposure, but it cannot guarantee the model will behave consistently under every prompt, plugin call, or retrieval path. The security boundary shifts from the application code alone to the runtime interaction around the model.

In practice, this is why teams talk about model behaviour, not just model code. For broader threat context around runtime abuse, the NIST AI 600-1 GenAI Profile and NIST AI Risk Management Framework both frame generative AI risk as something that must be managed through governance, testing, and ongoing monitoring, not only pre-deployment checks.

Where Existing Tools Still Help, and Where They Stop

Traditional tooling is not obsolete. Logging, identity controls, network segmentation, DLP, secrets management, and API gateways still reduce blast radius and improve visibility. They become insufficient when they are used as if they were complete controls for a system that can improvise, follow indirect instructions, or generate harmful output without any code change.

The weak point is the assumption that static policy can fully describe dynamic behaviour. An LLM may comply on one request, refuse the next, and then reveal sensitive data or misuse a tool through a different prompt structure, retrieved context, or connected integration. That means the control problem is not just allowing or blocking traffic, it is constraining runtime actions and watching for drift.

For practitioners mapping this to established security guidance, the model governance layer and the surrounding control plane matter as much as the model itself. The NIST AI 600-1 GenAI Profile is useful here because it specifically addresses testing, provenance, and incident handling for generative systems, while the NIST Cybersecurity Framework 2.0 helps structure the supporting functions around governance, detection, response, and recovery.

Why Runtime Observation Becomes Part of the Security Model

The practical shift is that security must observe the model while it is operating. Prompt injection, unsafe tool invocation, data leakage through retrieval, and policy bypass are runtime failures, not just design-time defects. The relevant question is no longer only whether the model is “safe,” but whether each interaction is bounded, attributable, and monitored well enough to detect unexpected behaviour quickly.

This is why organisations are increasingly pairing model gateways, policy enforcement, and trace logging with human review for higher-risk actions. The goal is not to trust the model to self-limit, but to make risky behaviour visible and interruptible before it becomes an incident. Existing controls are still valuable, but they must be wrapped around the model lifecycle and interaction surface.

For teams securing agentic or tool-using systems, the OWASP Agentic AI Top 10 is a strong reference for the runtime issues that static controls miss, especially identity and privilege abuse, tool misuse, and cascading failures. MITRE ATLAS adversarial AI threat matrix adds useful technique-level detail for prompt injection, context manipulation, and agent hijacking.

Risk and Threat Considerations

LLMs expand risk because they can turn ordinary inputs into unexpected actions or disclosures at runtime. The same deployment may be safe in one context and unsafe in another, especially when the model can retrieve internal data, call tools, or influence downstream systems.

Failure mechanism: Attackers, careless users, or flawed integrations exploit the model’s variability, then steer it through prompt injection, context poisoning, data exfiltration, or unintended tool use. Static controls may see a normal request path while the model produces a harmful decision or leaks sensitive information.

Impact: The result can be sensitive-data exposure, unsafe automation, privilege abuse, operational disruption, or loss of control over model-mediated workflows. At scale, a weakness in one LLM integration can become a repeated failure mode across many sessions, users, or connected systems.

Practitioner Guidance

What to prioritise: Treat runtime controls as the primary security layer for LLM deployments that can read, retrieve, or act. Focus first on the paths where model output can trigger tool calls, data access, or external side effects.

What to verify: Confirm that logging, policy enforcement, and approval steps cover the model’s actual runtime decisions, not just the surrounding application. If you cannot reconstruct what the model saw and did, you do not yet have adequate control.

Common mistake: Assuming that an existing web, endpoint, or content-security stack is sufficient because the LLM is “just another app.” The hard part is not blocking all bad text, it is constraining unpredictable behaviour when the model is embedded in a workflow.

Practitioner takeaway: The right security model for LLMs is continuous supervision of behaviour, context, and authority, because the risk emerges from runtime variability, not from a stable code path.

Related resources from NHI Mgmt Group

• Why do AI deployments create new data security risk even when traditional cloud controls are in place?
• Why does weak user awareness still create risk even when organisations have security tools and monitoring in place?
• Why do shadow apps create identity risk even when inventory tools are in place?
• Why does data sprawl increase risk even when security tools are already in place?