Notifications
Clear all
Topic starter
09/07/2026 12:34 pm
TL;DR: AI data security now extends beyond files and networks to prompts, retrieved context, outputs, and usage, because inference exposure, oversharing, prompt injection, and unauthorized tool use can reveal sensitive information even when classic access controls are satisfied, according to Knostic. The governance challenge is no longer whether AI can answer, but whether it can answer safely, with traceable policy and context checks in place.
NHIMG editorial — based on content published by Knostic: Key findings on AI data security in enterprise AI systems
By the numbers:
- A 2024 study shows that defense techniques can reduce prompt extraction by 83.8% for Llama2-7B and 71.0% for GPT-3.5.
Questions worth separating out
Q: How should security teams prevent AI systems from oversharing sensitive data?
A: They should enforce policy at answer time, not just at source access time.
Q: Why do enterprise AI systems create new identity and access risks?
A: Because they can transform a permitted request into an unpermitted disclosure.
Q: What breaks when AI tools are connected to broad knowledge sources without guardrails?
A: The model can overshare, retrieve beyond need-to-know, and amplify sensitive context through its output.
Practitioner guidance
- Enforce answer-time policy checks Apply policy at the moment of retrieval and generation, not only when a user opens a source file.
- Inventory AI connectors and service identities Map every model, plugin, retrieval source, and API connection to an owner, privilege scope, and logging path.
- Test for prompt injection and oversharing Run adversarial prompt suites against production-like workflows to measure leakage, broad retrieval, and unsafe tool invocation.
What's in the full article
Knostic's full article covers the operational detail this post intentionally leaves for the source:
- Prompt-by-prompt examples of how oversharing appears in enterprise AI workflows and chat tools
- Detailed mitigation patterns for RBAC, ABAC, and real-time output filtering across AI systems
- Operational logging and observability requirements for prompt, retrieval, and response provenance
- Specific references to enterprise tools such as Copilot, Glean, and related AI search environments
👉 Read Knostic's analysis of AI data security risks and controls →
AI data security in enterprise LLMs: are your controls keeping up?
Explore further
09/07/2026 12:39 pm
AI data security is becoming an access-governance problem, not just a model-safety problem. The article correctly shows that prompts, retrievals, and outputs now behave like a separate knowledge layer with its own exposure risk. That means classic data security tools are necessary but insufficient if they stop at file permissions or DLP filters. Practitioners should treat the response stream as a governed access surface.
A few things that frame the scale:
- A systematic investigation of multi-turn interactions showed prompt leakage rates rising from 17.7% to 86.2% under specific attack patterns, according to The State of Non-Human Identity Security.
- Only 1.5 out of 10 organisations are highly confident in their ability to secure NHIs, compared to nearly 1 in 4 for securing human identities, according to The State of Non-Human Identity Security.
A question worth separating out:
Q: Who is accountable when AI systems expose regulated or proprietary data?
A: Accountability should sit with the teams that own the model, the connectors, the data classification policy, and the access governance path. For regulated data, audit trails must show who initiated the request, what data was retrieved, and why the response was allowed. Without that evidence, compliance and incident review both fail.
👉 Read our full editorial: AI data security exposes the governance gap in enterprise LLM use
