Notifications
Clear all
Topic starter
22/06/2026 9:58 am
TL;DR: Traditional security controls do not map cleanly to LLMs because their inputs, outputs, data sources, and runtime behavior are probabilistic rather than deterministic, according to Orca Security and IBM Institute for Business Value. The governance gap is no longer theoretical: access scope, data exposure, and runtime monitoring all need AI-specific treatment, not legacy application assumptions.
NHIMG editorial — based on content published by Orca Security: Why Generative AI Breaks Traditional Security Models
By the numbers:
- 82% of executives say secure and trustworthy AI is essential to their business, but only 24% of current generative AI projects have a security component built in.
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.
- 72% of organisations have experienced or suspect they have experienced a breach of non-human identities, with 46% confirmed and 26% suspected.
Questions worth separating out
Q: How should security teams govern generative AI workloads without breaking existing IAM models?
A: Start by treating AI workloads as a distinct identity surface with their own service accounts, API keys, data flows, and approved tools.
Q: Why do generative AI systems create more security risk than traditional applications?
A: Because their behaviour is probabilistic, context-dependent, and often connected to external data sources and tools.
Q: What breaks when employees use public AI tools with sensitive company data?
A: The organisation loses control over where the data is stored, how long it persists, and whether it may be reused in ways the business cannot govern.
Practitioner guidance
- Inventory every AI workload and shadow AI path Create a live inventory of model endpoints, RAG services, vector stores, API keys, and unsanctioned AI tools.
- Scope service accounts to the narrowest AI function Review the service accounts and machine identities used by training, inference, and retrieval services.
- Classify training inputs before they reach the pipeline Scan data lakes, document stores, and vector databases for sensitive content before fine-tuning or retrieval indexing begins.
What's in the full article
Orca Security's full blog post covers the operational detail this post intentionally leaves for the source:
- Step-by-step explanation of how the vendor groups AI security into AI-SPM, DSPM, guardrails, and runtime monitoring
- Detailed examples of prompt injection, shadow AI, poisoning, and model inversion in enterprise workflows
- The article’s own mapping of AI security controls to NIST AI RMF, OWASP LLM guidance, and MITRE ATLAS
- Tool-specific discussion of Orca's agentless visibility approach across cloud environments
👉 Read Orca Security's analysis of generative AI security and LLM governance →
Generative AI security: what IAM teams are missing?
Explore further
22/06/2026 10:39 am
Traditional IAM was designed for predictable workloads, not probabilistic model behaviour. The article’s core point is that LLMs change inputs, outputs, and data dependencies at runtime, which makes static entitlement thinking incomplete. Least privilege still matters, but it no longer describes the full security problem when the system can retrieve, infer, and act in ways that were not known at provisioning time. Practitioners need to treat AI as a distinct identity and data governance surface, not as a normal application with a new interface.
A few things that frame the scale:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials, according to AI Agents: The New Attack Surface report.
- 48% of companies say they cannot track and audit the data their AI agents access, creating a complete blind spot for compliance and breach investigation.
A question worth separating out:
Q: Who is accountable when an AI model leaks data or triggers an unauthorised action?
A: Accountability should sit with the business owner of the AI use case, the team managing the underlying identities, and the security function that approved the control boundary. If the model can access data or tools, ownership cannot be vague. The programme must assign responsibility for discovery, access review, monitoring, and response before deployment.
👉 Read our full editorial: Why traditional IAM fails for generative AI security
