Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

GenAI security risks and the governance gap teams are missing


(@lalit)
Member Admin
Joined: 1 year ago
Posts: 257
Topic starter  

TL;DR: GenAI is driving faster adoption of AI tools across business functions, but governance is lagging: 88% of organisations now use AI in at least one function while only 24% have robust AI risk governance frameworks, according to IBM research and McKinsey. The control gap turns convenience, automation, and prompt-driven workflows into a broad security and compliance exposure.

NHIMG editorial — based on content published by Proofpoint: GenAI security risks, real-world scenarios, and mitigation steps

By the numbers:

Questions worth separating out

Q: What fails when organisations let GenAI read and act on untrusted content?

A: Untrusted content can steer the model into leaking data, bypassing policy, or producing instructions that look legitimate but are operationally unsafe.

Q: Why do AI tools create governance risk even when humans stay in charge?

A: AI tools create risk when they reshape the real decision path without changing formal ownership.

Q: How do security teams know whether AI access is actually working safely?

A: Look for three signals: complete discovery of the AI estate, clear mapping of source data to each system, and logs that prove what was accessed and why.

Practitioner guidance

  • Define prohibited prompt content Publish a data-classification policy that explicitly bans source code with secrets, customer records, internal strategy, and regulated personal data from public or unmanaged GenAI tools.
  • Inventory AI tools and connectors Build a live register of sanctioned and shadow AI services, then map each one to its data sources, plugins, and retained prompt or output records.
  • Apply least privilege to AI integrations Limit every AI connector to the minimum systems and datasets required, and remove broad access to mailboxes, document stores, and cloud APIs where not essential.

What's in the full article

Proofpoint's full blog covers the operational detail this post intentionally leaves for the source:

  • Detailed best-practice guidance for data controls around prompts, including acceptable-use policy language and DLP enforcement patterns.
  • Concrete examples of monitoring signals for shadow AI, prompt submissions, output records, and plugin activity across collaboration tools.
  • Stepwise recommendations for least-privilege access to AI integrations, including where to audit connector permissions.
  • The 30-day rollout plan with week-by-week actions for AI inventory, monitoring, and governance testing.

👉 Read Proofpoint's analysis of GenAI security risks and practical mitigation steps →

GenAI security risks and the governance gap teams are missing?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 2 months ago
Posts: 11787
 

GenAI governance debt is now a control problem, not a policy problem. The article’s core point is that adoption has raced ahead of oversight, and that mismatch creates a durable governance deficit. Security programmes that treat AI as an application feature miss the fact that prompts, outputs, and connectors create new trust boundaries. Practitioners should anchor AI governance in access control, validation, and data handling rather than standalone guidance documents.

A question worth separating out:

Q: Who is accountable when an AI agent triggers a banking error or compliance breach?

A: Accountability sits with the institution that granted the agent access, defined its scope, and failed to govern its actions. Banking regulators will focus on whether the bank can prove effective oversight, traceability, and control over both human prompts and autonomous actions.

👉 Read our full editorial: GenAI security risks are outpacing enterprise governance controls



   
ReplyQuote
Share: