Glean AI search increases oversharing risk beyond static permissions

At a glance

What this is: This is an independent analysis of Glean security that finds AI search can still overshare sensitive information even when source permissions are respected.

Why it matters: It matters because IAM, IGA, and security teams need controls that govern what AI may reveal, not just what a user can technically access.

By the numbers:

• Employees spend around 1.8 hours per day, about 9 to 10 hours per week, searching for internal information.
• Glean supports over 100 enterprise integrations, providing connections to systems such as Jira, Confluence, Zendesk, Box, as well as internal tools via APIs.
• Glean retains logs based on customer-defined policies, typically between 30 and 365 days.

👉 Read Knostic's analysis of Glean AI security and oversharing risk

Context

Enterprise AI search changes the access problem from simple retrieval to contextual disclosure. A user may be authorised to reach a source document, yet still receive an AI-generated answer that exposes strategy, timelines, or related content that was never meant to be surfaced together.

That is why AI search governance cannot stop at source-system permissions, encryption, or tenant isolation. The central issue is need-to-know, because large language models can combine fragments across systems and produce answers that satisfy access rules while still violating organisational intent.

Key questions

Q: How should security teams govern AI search so it does not overshare sensitive content?

A: Treat AI search as a disclosure system, not just a retrieval system. Enforce policy at answer time, not only at source access time, and test whether combined fragments reveal more than the user needs to know. The goal is to stop contextually inappropriate answers, even when every source snippet is individually permitted.

Q: Why do static access controls fail in enterprise AI search?

A: Static access controls answer a different question from the one AI search creates. They say whether a user can reach content, but they do not govern whether an LLM should combine that content into a sensitive answer. That gap is why need-to-know enforcement must sit alongside permissions.

Q: What do security teams get wrong about AI search oversharing?

A: They often assume encryption, tenant isolation, and source permissions solve the disclosure problem. Those controls protect storage and access, but oversharing happens at the response layer when an AI model fuses fragments from different systems. The real failure is treating answer generation like ordinary search.

Q: How do you know if AI search controls are actually working?

A: Look for evidence that prompts, retrieved sources, and responses are being tied to policy decisions in an auditable trail. If you can only show that a user could open a document, you have not proven the system prevented oversharing. Effective control means you can explain why an answer was allowed or blocked.

Technical breakdown

Why static ACLs fail in AI search oversharing

Traditional access controls decide whether a user may open a source object. AI search adds a second decision point at answer generation, where retrieved fragments are merged into a new response. That makes static ACLs incomplete, because permissible snippets can become sensitive when recombined by an LLM. This is especially visible in retrieval-augmented generation, where context is assembled dynamically from multiple repositories. The model may never directly expose a restricted file, yet still disclose its substance through synthesis, paraphrase, or ranking. Practical implication: treat query-time output control as a distinct governance layer, not a proxy for source permission enforcement.

Practical implication: govern AI answers separately from source permissions and use query-time policy enforcement.

How zero-copy indexing changes the risk profile

Zero-copy indexing reduces duplication by keeping content in the source system and caching only the vectors and metadata needed for retrieval. That lowers sprawl, but it does not eliminate oversharing risk, because exposure now depends on how retrieval and summarisation behave at runtime. In this model, the search layer becomes a disclosure boundary rather than a storage boundary. Tenant isolation, secure connectors, and encryption still matter, but they do not answer the need-to-know question that AI search introduces. Practical implication: evaluate AI search as a live disclosure system, not just a secure indexing architecture.

Practical implication: assess the search layer as a disclosure boundary and test it with realistic prompts.

Why AI search auditability must include prompt and response traces

Audit logs that record only source access are not enough when the risk is an AI-generated answer. Security teams need visibility into the full path from prompt to retrieval to response, including which documents contributed to the output and under which policy. That is how investigators distinguish benign search activity from oversharing events and prove whether a response crossed a usage boundary. This is also where contextual classification matters, because a document may be acceptable in one workflow and sensitive in another. Practical implication: retain searchable evidence for prompt, retrieval, and answer lineage so you can investigate disclosure, not just access.

Practical implication: preserve prompt, retrieval, and answer lineage so investigations can reconstruct disclosure paths.

Threat narrative

Attacker objective: The attacker seeks to extract sensitive internal information through a seemingly legitimate AI query without needing direct file access.

1. Entry occurs when a user submits a normal business query to an AI search interface that is connected to many internal systems.
2. Escalation occurs when the model combines permissible fragments from multiple sources into an answer that exceeds the user's operational need-to-know.
3. Impact occurs when sensitive strategy, credentials, or internal context are exposed through a truthful but overbroad AI-generated response.

Breaches seen in the wild

• McKinsey AI platform breach — McKinsey AI platform hack exposed 46M chats and sensitive data.
• ASP.NET machine keys RCE attack — 3,000+ exposed ASP.NET machine keys enabled remote code execution.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Oversharing is an identity governance failure, not a search-quality issue. The core problem is that AI can respect source permissions while still violating need-to-know boundaries. That means the governance question is not whether the user could access the underlying document, but whether the system should have assembled that answer at all. Practitioners should treat AI disclosure policy as part of IAM and IGA, not as a search add-on.

Need-to-know is the missing control plane for enterprise AI search. Traditional ACLs are built around access entitlement, while AI search operates around answer construction. Those are not the same control objectives, and mixing them creates a false sense of protection. Organisations need policy logic that evaluates the context of the question, the source lineage, and the sensitivity of the combined output before disclosure is allowed.

Contextual access control is now a named governance gap. Static labels and file-level sensitivity tags cannot fully model how LLMs blend fragments from different systems into a single response. The result is exposure that is technically permitted at source level but operationally inappropriate at output level. Security teams should recognise this as a distinct control class, not a tuning problem in the search layer.

AI search governance must extend audit and accountability into the answer path. If logs do not show prompt, retrieval, and response lineage, teams cannot prove whether an oversharing event occurred or which policy failed. That weakens both incident response and compliance evidence. Practitioners should therefore align AI search controls with the same evidentiary standards they expect from privileged access and sensitive data handling.

From our research:

• Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation, according to AI Agents: The New Attack Surface report.
• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.
• For a broader control perspective, see OWASP Agentic AI Top 10 for the exposure patterns that show up when AI systems act beyond intended scope.

What this signals

Need-to-know enforcement will become the practical test for AI search governance. Teams that only validate ACL inheritance will miss the disclosure layer where LLMs blend context into answers. The programme signal to watch is whether your controls can explain not just who accessed a source, but why a specific answer was allowed to be generated.

Contextual disclosure deserves its own control category. In practice, that means classifying prompts, retrieved fragments, and answers against sensitivity and business context, not just file labels. For teams already tracking AI agent exposure, the 52% auditability figure from AI Agents: The New Attack Surface report is a warning that visibility remains incomplete even before oversharing is investigated.

The operational signal is whether your AI search estate can produce evidence for answer lineage within the same governance workflow used for privileged access and sensitive data handling. If not, oversharing will remain an investigative afterthought rather than a controlled outcome.

For practitioners

• Separate source access from answer authorisation Apply a distinct approval or policy check to AI-generated responses when the system synthesises content from multiple repositories. Source permissions alone are not sufficient to govern disclosure risk.
• Test for oversharing with realistic business prompts Use prompts that mirror how employees ask for strategy, roadmap, personnel, and operational context. Measure whether the AI returns more than the requester’s need-to-know even when each source is individually accessible.
• Retain full prompt-to-answer audit trails Log the prompt, retrieved documents, response text, user identity, and governing policy so investigators can reconstruct disclosure paths after a suspected oversharing event.
• Map sensitive data lineage across connected systems Track how information moves from Jira, Confluence, ticketing tools, file stores, and chat systems into AI answers. That lineage is what exposes where context becomes disclosure.

Key takeaways

• AI search can overshare sensitive information even when source permissions are technically enforced, because answer generation creates a new disclosure layer.
• The practical control gap is need-to-know, not storage security, and that gap becomes wider when LLMs combine fragments across multiple systems.
• Teams need policy checks, lineage-aware audit trails, and realistic prompt testing if they want AI search to remain governable at enterprise scale.

Key terms

• Oversharing: Oversharing is the unintentional exposure of information through an AI system that combines otherwise permissible content into a sensitive answer. It is not a storage failure. It is a disclosure failure that appears when access rights are broader than operational need-to-know.
• Need-to-know: Need-to-know is the governance principle that a user or system should receive only the information required to perform an approved task. In AI search, it is stricter than mere access entitlement because the model may reveal context that the requester could technically access but should not see in aggregate.
• Zero-copy indexing: Zero-copy indexing keeps source content in place and stores only what is needed to retrieve it, usually vectors and metadata. It reduces duplication and sprawl, but it does not remove disclosure risk, because the AI can still synthesise sensitive output from permitted fragments at query time.
• Answer lineage: Answer lineage is the trace from a user prompt to the retrieved sources and final AI response. It gives investigators evidence for how disclosure occurred, which content contributed to the answer, and whether a policy decision should have blocked the output.

What's in the full article

Knostic's full article covers the operational detail this post intentionally leaves for the source:

• Connector and ingestion design details for Glean environments, including how permission graphs and metadata are synchronised.
• Knostic's prompt simulation and response-analysis workflow for identifying overshared outputs before users see them.
• Audit-dashboard examples showing how answer lineage, access attempts, and policy decisions are surfaced for compliance teams.
• Implementation notes for integrating monitoring into existing security workflows without changing the underlying search architecture.

👉 Knostic's full article covers the Glean security model, oversharing scenarios, and monitoring details for practitioners.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an identity security programme, it is worth exploring.