Join our Newsletter — 33% off our NHI Course

Generative AI security: are your controls keeping up with AI chatflows?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 19129
Topic starter  

TL;DR: Enterprises can see who uses ChatGPT or Copilot, but often cannot see what is sent, what returns, or how shadow AI and agent actions expose proprietary data, according to WitnessAI. Legacy DLP and CASB controls struggle because AI risk lives in conversational payloads and intent, not file transfers or keywords.

Editorial analysis by NHI Mgmt Group, based on content published by WitnessAI: “Generative AI Security: A Practical Guide for Enterprise Teams”.

Key questions

Q: What breaks when security teams inspect only prompts instead of full AI workflows?

A: Prompt level inspection misses the sequence of actions that creates real risk.

Q: Why do generative AI workflows create risk even when employees use approved tools?

A: Approved tools still process content that may contain sensitive business data, code, or regulated information.

Q: How do organisations decide which AI interactions should be blocked versus routed?

A: Decide by risk, purpose, and data sensitivity.

Practitioner guidance

  • Deploy bidirectional AI traffic visibility Capture both prompts and responses across approved and shadow AI tools so security teams can see what enters and leaves the model boundary.
  • Classify AI interaction intent before enforcement Use intent-based policy logic to distinguish legitimate work from sensitive data exposure or risky model use, rather than relying on keywords alone.
  • Apply runtime controls at the point of interaction Block prompt injection, jailbreak attempts, and sensitive content before the model processes the request, then filter outputs before they reach users or tools.

Bottom line: Generative AI changes the control problem because the exposure point is the prompt, response, and action path rather than the file transfer event.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 18837
 

Bidirectional visibility is now a baseline governance requirement for generative AI. Seeing only that a user accessed ChatGPT or Copilot is not enough when the real risk is inside the prompt and response stream. Enterprises need governance over both directions of the exchange because exposure can occur before a file ever exists and long before a traditional DLP rule would fire. The practitioner conclusion is simple: if you cannot inspect both inbound and outbound AI content, you do not control the interaction.

A few things that frame the scale:

  • Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
  • Only 23% of IT leaders were very confident in their organisation's ability to manage security and governance for GenAI deployments, according to a 2025 Gartner survey of 360 IT leaders.

A question worth separating out:

Q: How should organisations govern autonomous AI agent actions separately from human sessions?

A: Treat the agent as a governed execution identity with its own permissions, logs, and guardrails. Human authentication does not prove the agent should be able to call tools or modify systems. Governance has to distinguish who started the workflow from what the agent is authorised to do at runtime.

👉 Read our full editorial: Generative AI security needs bidirectional visibility and runtime controls



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.