LLM applications can expose sensitive data when users paste secrets, when agents retrieve privileged context, or when responses echo internal material back to users. That creates an identity problem because the model may handle information it was never meant to disclose. Teams should govern what the model can see and what it can return.
Why This Matters for Security Teams
LLM applications are not just new interfaces. They are new decision points where identity, data handling, and privilege intersect. When a user pastes a token into a prompt, when an assistant can retrieve internal documents, or when an agent can call tools on behalf of a user, the risk is no longer limited to the model output. The model becomes a conduit for secrets, regulated data, and privileged context. That is why this is an identity issue, not only a content moderation issue.
The practical concern is scope. Traditional IAM assumes access is granted to known identities with fairly stable patterns. LLM apps, especially agentic ones, can behave more dynamically: they summarize, transform, retrieve, and chain actions in ways that are hard to predict at design time. Current guidance from OWASP Agentic AI Top 10 and NIST AI Risk Management Framework both point to the need for tighter governance around context, tools, and disclosure, not just model prompts.
NHIMG research shows why this matters operationally: 79% of organisations have experienced secrets leaks, with 77% of those incidents resulting in tangible damage, according to the Ultimate Guide to NHIs. In practice, many security teams encounter leakage only after a user reports an unexpected answer or a downstream system logs an over-broad tool call, rather than through intentional testing.
How It Works in Practice
The leakage path usually starts with overexposure. An LLM application may ingest chat history, connected files, ticketing systems, or API-backed enterprise search. If identity controls are weak, the model can surface information that the human requester never had standing permission to see. If an agent is involved, the problem expands: the agent may have a workload identity, tool credentials, and delegated rights that exceed the user’s need for that specific task.
The safest pattern is to treat the agent as a distinct workload identity with explicit, time-bound authority. That means separating the identity of the end user from the identity of the agent, then using intent-based authorization at runtime. Instead of granting broad static access, policy should evaluate what the agent is trying to do, what data it wants, and whether the request fits the current context. This is where just-in-time provisioning and short-lived secrets matter. Long-lived API keys and shared service accounts create persistent blast radius; short-lived credentials reduce the window for misuse.
Practical controls usually include:
- scoping retrieval so the model only sees the minimum data needed for the task;
- using workload identity primitives such as OIDC, SPIFFE, or SPIRE for agent authentication;
- issuing ephemeral secrets per task and revoking them automatically when the task ends;
- enforcing policy-as-code at request time rather than relying only on static RBAC;
- logging tool use, retrieved objects, and returned snippets for audit and investigation.
The real lesson from 52 NHI Breaches Analysis is that compromise often follows privilege sprawl, not sophisticated prompt tricks. For implementation alignment, CSA MAESTRO agentic AI threat modeling framework and the OWASP Top 10 for Agentic Applications 2026 both reinforce the need to model tool chaining, lateral movement, and over-privileged delegation. These controls tend to break down when legacy applications expose broad internal data stores through a single prompt layer because the model inherits access faster than the governance model can constrain it.
Common Variations and Edge Cases
Tighter access control often increases friction, requiring organisations to balance user experience against leakage reduction. That tradeoff is real in customer support copilots, research assistants, and developer tools where users expect fast answers and broad retrieval. Best practice is evolving here, and there is no universal standard for how much context an LLM may safely retain across sessions.
One common edge case is session persistence. If chat history, memory stores, or vector databases retain sensitive snippets too long, the model can reintroduce data that should have been forgotten. Another is delegation across multiple tools: even if each tool is individually low risk, chained actions can expose data that no single permission review would catch. This is why static, role-based IAM often fails against autonomous or goal-driven workloads. The agent does not follow a fixed human job description; it adapts to the prompt, the context window, and the tools available.
Where governance is still maturing, current guidance suggests using real-time policy evaluation, narrow retrieval scopes, and explicit separation of duties between model, agent, and user. For broader context on identity lifecycle and secret handling, the Ultimate Guide to NHIs remains the best foundation, while the OWASP NHI Top 10 is useful for mapping agent access and disclosure failures to concrete control gaps. This guidance breaks down most often in multi-tenant environments with shared embeddings, inherited permissions, and legacy secrets stored outside a vault because the system cannot reliably distinguish intended context from accidental exposure.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | LLM07 | Covers data leakage through model context, retrieval, and output handling. |
| CSA MAESTRO | T2 | Maps agent tool use and delegated access to runtime threat modeling. |
| NIST AI RMF | GOV | Requires accountable governance for AI systems that handle sensitive information. |
Limit retrieval, scrub prompts, and validate outputs before the model can expose sensitive data.
