Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

AI governance and oversharing: are your controls keeping up?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 10158
Topic starter  

TL;DR: AI governance needs controls that act at inference time, because prompt injection, oversharing, weak logging, and unfaithful citations can all expose sensitive data even when storage controls are sound, according to Knostic. The governance gap is no longer policy design but enforcing identity-aware need-to-know at the moment an answer is generated.

NHIMG editorial — based on content published by Knostic: Why AI governance is important

Questions worth separating out

Q: How should security teams implement access controls for enterprise AI assistants?

A: Security teams should enforce access at inference time, not just at the data source.

Q: Why do AI assistants create new governance risks for IAM teams?

A: AI assistants can reveal information from across multiple systems in a single answer, so access decisions are no longer limited to a database query or application screen.

Q: What breaks when prompt injection is not controlled in AI systems?

A: When prompt injection is not controlled, the model can follow hidden instructions embedded in documents, URLs, or user text and override intended safeguards.

Practitioner guidance

  • Implement inference-time access policies Apply persona-based access controls to prompts and generated output so the system can suppress sensitive fields, truncate answers, or block responses when need-to-know is not met.
  • Gate tools and connectors by intent Classify external inputs as trusted or untrusted, then require intent checks before the model can call tools, fetch documents, or invoke connectors.
  • Log prompts, policy hits, and outputs Record what the model saw, which policy blocked or allowed the response, and what was returned to the user.

What's in the full article

Knostic's full article covers the operational detail this post intentionally leaves for the source:

  • How persona-based access controls are applied at inference time across prompts and outputs.
  • Examples of layered defenses for prompt injection, including tool gating and output filters.
  • Guidance on logging, retention, and audit trails for regulated AI deployments.
  • Implementation notes for aligning governance controls with board reporting and compliance evidence.

👉 Read Knostic's analysis of why AI governance matters for enterprise adoption →

AI governance and oversharing: are your controls keeping up?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 2 months ago
Posts: 9713
 

AI governance has become an identity problem as much as a model problem. Once an assistant can answer from enterprise context, the critical question is no longer only whether the model is safe. It is whether the right person, in the right role, gets the right answer at the right moment, with a defensible policy trail. That makes identity-aware authorization part of AI governance, not an adjacent concern. Practitioners should treat AI responses as privileged disclosures that need access governance.

A few things that frame the scale:

  • 91.6% of secrets remain valid five days after the targeted organisation is notified, showing a critical gap in remediation procedures, according to Ultimate Guide to NHIs.
  • Only 5.7% of organisations have full visibility into their service accounts, according to Ultimate Guide to NHIs.

A question worth separating out:

Q: Who is accountable when an AI system leaks sensitive data?

A: Accountability should sit with the teams that own the AI policy, logging, and access decisions, not only with the model vendor. Organisations need clear ownership for prompts, retrieval rules, output filters, and audit records so they can explain why a response was allowed and who approved the control design.

👉 Read our full editorial: AI governance fails at the answer layer without real-time controls



   
ReplyQuote
Share: