Notifications
Clear all
Topic starter
19/08/2026 3:28 pm
TL;DR: Sensitive data can leak from LLMs through prompt injection, memorized training data, and poorly isolated test data, according to Fiddler. The key security shift is treating leakage as a full-lifecycle governance issue, not a single-model defect, and layered controls plus continuous monitoring are needed to catch exposure in time.
NHIMG editorial — based on content published by Fiddler: Information Leakage Security Optimization Model for LLMs
By the numbers:
- A 2023 study of 36 real-world LLM applications found that approximately 86% were susceptible to prompt injection attacks.
- Fiddler says enterprises can incur approximately $260K annually at 500K traces per day from external evaluation costs, with costs rising to $520K at 1M traces per day.
Questions worth separating out
Q: How do teams reduce the risk of sensitive data leaking from LLM outputs?
A: Teams reduce leakage by adding a response inspection layer that checks generated text for secrets, regulated data, and disallowed disclosures before delivery.
Q: Why do multi-agent AI workflows increase information leakage risk?
A: Because each handoff can widen or drop the original user’s permissions.
Q: What signals show that an LLM may be leaking sensitive data?
A: Watch for abnormal response length, unusual output entropy, prompt patterns that resemble injection attempts, and rising PII detection rates.
Practitioner guidance
- Separate prompts from secrets Keep API keys, database credentials, and service tokens out of system prompts and template text.
- Enforce permission checks at every agent boundary Re-authorise retrieval, tool execution, and response generation separately, and ensure user identity travels through multi-agent handoffs.
- Sanitise training and evaluation datasets Remove or tokenise PII, PHI, and customer records before fine-tuning or evaluation.
What's in the full article
Fiddler's full blog covers the operational detail this post intentionally leaves for the source:
- Concrete prompt validation patterns for injection detection and response blocking
- Examples of output filtering logic for PII detection, system prompt fragments, and entropy anomalies
- Implementation detail for secrets vault separation and runtime credential injection
- Traceability and SIEM integration patterns for distributed audit trails across agent handoffs
👉 Read Fiddler's analysis of information leakage security for LLMs →
LLM information leakage: are your controls keeping up?
Explore further
19/08/2026 4:12 pm
LLM leakage is an identity and data-governance problem, not a model-only problem. The article correctly shows that prompts, training corpora, and agent handoffs all create separate disclosure paths. In identity terms, the control failure is allowing runtime systems to inherit broad data access without re-checking the user’s entitlement at each step. Practitioners should treat access context as a first-class control plane concern.
A question worth separating out:
Q: Who is accountable when an AI agent accesses sensitive data it was not meant to use?
A: Accountability sits with the team that approved the agent, its connectors, and its policy boundaries, not with the runtime behaviour alone. Organisations need ownership for intent, permissions, monitoring, and validation so they can prove whether the agent stayed inside its approved purpose. Without that, audit and regulatory response become retrospective guesswork.
👉 Read our full editorial: Information leakage in LLMs is a governance problem, not a point fix
