Notifications
Clear all
Topic starter
09/07/2026 12:37 pm
TL;DR: Enterprise copilots can fabricate sensitive personal information when they cannot find source data, creating risks that look and spread like real leaks, according to Knostic research. In regulated environments, that makes AI governance a truth-control problem as much as a data-loss problem.
NHIMG editorial — based on content published by Knostic: New Knostic research on fabricated sensitive information in enterprise LLMs
By the numbers:
- 70% of employers have caught employees using AI assistants for salary research, with 38% saying these tools are pushing salary demands higher than ever.
- 72% of organisations have experienced or suspect they have experienced a breach of non-human identities, with 46% confirmed and 26% suspected.
Questions worth separating out
Q: What breaks when enterprise AI fabricates sensitive information?
A: When enterprise AI fabricates sensitive information, the main failure is trust collapse.
Q: Why do hallucinations create compliance risk in AI assistants?
A: Hallucinations create compliance risk because regulated decisions depend on accurate, source-backed statements.
Q: How can security teams measure whether AI responses are trustworthy?
A: Security teams should measure how often the assistant refuses ungrounded requests, how often answers can be traced to source documents, and how often sensitive prompts return unsupported claims.
Practitioner guidance
- Implement grounded-response controls Require enterprise copilots to cite source documents or refuse when they cannot ground a sensitive claim in retrievable records.
- Classify fabricated statements as incidents Treat hallucinated sensitive information as a security and compliance event when it crosses into employee data, confidential projects, or regulated content.
- Extend least privilege into AI retrieval paths Limit what the AI can search, summarise, or surface based on the user’s authorised access and the sensitivity of the underlying dataset.
What's in the full article
Knostic's full research covers the operational detail this post intentionally leaves for the source:
- How the enterprise copilot behaved when it could not find HR data and why it fabricated a response
- The specific demo setup used to test salary and personal-information prompts inside an artificial company environment
- Examples of how false salary or personnel answers can affect employee relations, compliance handling, and managerial decisions
- The vendor’s framing of its own AI governance approach and product positioning
👉 Read Knostic's research on fabricated AI data and enterprise governance risk →
LLM hallucinations in enterprise AI: what security teams need to know?
Explore further
09/07/2026 12:43 pm
Fabricated sensitive information is a governance failure, not just an AI accuracy problem. The core issue is that enterprise users treat an AI response as authoritative even when it is generated from incomplete context. That breaks the boundary between data security, human trust, and compliance handling. For identity teams, the lesson is that AI output control now belongs in the same governance conversation as access control and data classification.
A few things that frame the scale:
- 72% of organisations have experienced or suspect they have experienced a breach of non-human identities, with 46% confirmed and 26% suspected, according to The 2024 ESG Report: Managing Non-Human Identities.
- Enterprises that have experienced a compromised NHI averaged 2.7 separate incidents in the past 12 months, according to The 2024 ESG Report: Managing Non-Human Identities.
A question worth separating out:
Q: Who is accountable when an AI assistant gives a false answer about employee data?
A: Accountability sits with the organisation that deployed the assistant, not with the model alone. Teams that expose employee or policy data through AI must define ownership for retrieval access, output approval, logging, and incident response. Under regulated conditions, the business remains responsible for what the assistant says and how users act on it.
👉 Read our full editorial: LLM hallucinations can mimic data leaks in enterprise AI
