GenAI awareness training now has to cover agent risk

At a glance

What this is: This is an analysis of five GenAI risks that security awareness programmes must now cover, including shadow AI, hallucinations, bias, automation complacency, and AI agent misuse.

Why it matters: It matters because IAM, PAM, and identity governance teams now have to account for AI agents and human users in the same risk model, especially where permissions, data access, and delegated actions overlap.

By the numbers:

• 92% agree governing AI agents is critical to enterprise security, yet only 44% have implemented any policies to do so.
• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%).

👉 Read Living Security Human Risk Management Platform's analysis of 5 Gen AI risks awareness training must cover

Context

Generative AI changes the risk model because it is not just another application layer, it is a new way for people and software to interact with sensitive data, approval workflows, and identity controls. For IAM and security teams, the core issue is governance: who can use AI tools, what data they can touch, and how the resulting actions are monitored across human and non-human identities.

The article frames Human Risk Management as the answer to a blended threat surface, and that is directionally right even if the broader problem is larger than awareness training alone. Once employees, copilots, and AI agents are all operating in the same business processes, the control plane has to extend beyond user training into policy, visibility, and lifecycle governance.

The starting position described in the article is increasingly typical for enterprises that adopted GenAI before their control model caught up. That makes the guidance relevant well beyond awareness training, because the same governance gaps show up in access review, data handling, and delegated authority.

Key questions

Q: How should security teams govern AI features embedded in SaaS applications?

A: Treat embedded AI as a machine identity problem with data access implications. Inventory the feature, map the connected permissions, define what data it may use, and monitor retention and sharing paths. If the AI feature can read corporate content, it needs explicit approval, logging, and periodic review like any other privileged integration.

Q: Why do AI agents create more governance risk than ordinary integrations?

A: AI agents can connect quickly, run continuously, and accumulate broad permissions across multiple services. That combination makes ownership blur and scope drift more likely, so the real risk is not the tool itself but the uncontrolled access path it creates across enterprise systems.

Q: What do security teams get wrong about Shadow AI?

A: They often treat Shadow AI as an approval problem for software, when it is usually also an identity problem. The hidden risk can be an undocumented token, an over-permissioned service account, or an autonomous agent with unreviewed reach. Inventory the identity layer before you decide the tool is the issue.

Q: How do you know if GenAI training is actually reducing risk?

A: Look for changes in behaviour, not just course completion. Useful indicators include fewer policy violations, better verification in high-risk workflows, reduced use of unsanctioned AI tools, and improved alignment between identity logs, threat signals, and the actions employees or agents actually take.

Technical breakdown

Shadow AI creates an identity and data governance gap

Shadow AI appears when employees use unsanctioned GenAI tools to complete work tasks, often by pasting in sensitive content. The technical issue is not only data leakage, but loss of control over identity context, retention, and downstream reuse. Once data enters a public model or unmanaged service, the enterprise can lose visibility into who accessed it, where it went, and whether it is still exposed. That makes shadow AI partly a data-security problem and partly an identity-governance problem, because the user action and the data exposure are inseparable.

Practical implication: classify approved AI tools, bind them to access policy, and block unsanctioned data sharing paths before they become unmanaged identity channels.

Hallucinations and automation complacency weaken human control

Hallucinations are plausible but false outputs from GenAI systems, and automation complacency is the tendency to accept those outputs without verification. In security and business workflows, the danger is not the model's error alone, but the human decision to trust it as authoritative. That creates a control failure in validation, especially where AI-generated content is used for coding, financial decisions, or incident triage. From a governance perspective, the issue is whether humans remain accountable for decisions that were materially shaped by machine output.

Practical implication: require human verification checkpoints for high-impact AI outputs and train users to treat generated content as input, not evidence.

AI agents expand the attack surface through delegated permissions

AI agents differ from ordinary automation because they can select actions, tools, and timing at runtime. When they are integrated with SaaS platforms, they inherit or request permissions that may be broader than the underlying task needs. That creates a non-human identity problem: the agent can move across systems with credentials or tokens that were never intended for open-ended use. If lifecycle controls, access boundaries, and audit trails are weak, a single compromised agent can amplify its reach quickly across business workflows.

Practical implication: map AI agents to specific identities, constrain their permissions to task scope, and review their access like any other privileged workload.

Threat narrative

Attacker objective: The objective is to exploit human trust and machine delegation to gain data, manipulate decisions, or perform unauthorised actions at enterprise scale.

1. Entry occurs when employees adopt unapproved GenAI tools or when AI agents are connected to SaaS systems with broad delegated permissions.
2. Escalation happens when those tools or agents are trusted to process sensitive data, interact with workflows, or act on instructions without sufficient verification.
3. Impact follows when sensitive data leaks, false outputs shape business decisions, or over-permissioned agents execute actions that exceed intended scope.

NHI Mgmt Group analysis

Shadow AI is now an identity governance problem, not just a policy problem. The article correctly treats unapproved AI usage as a data-leak risk, but the deeper issue is that unsanctioned tools create unmanaged identity paths between employees, SaaS apps, and external models. Once access is informal, the enterprise loses lifecycle control over what the tool can see, retain, or reuse. The governance lesson is that AI usage must be treated as part of identity governance, not a separate awareness campaign.

AI agents introduce a new category of non-human identity sprawl. These systems are not passive scripts because they can choose actions across tools and services. That means permissions, token scope, and auditability matter in the same way they do for service accounts and workload identities, with even more urgency because runtime behaviour can change. The practical conclusion is that agent identities need explicit ownership, narrow entitlements, and continuous review.

GenAI training fails when it stops at recognition and never reaches decision control. The article is right that employees need to spot deepfakes, hallucinations, and social engineering, but awareness alone does not change exposure if users can still approve high-risk actions unchecked. The more useful model is policy plus verification plus monitoring. Organisations should align training with the controls that actually interrupt risky behaviour, not with content volume.

Human Risk Management only works when identity, behavior, and threat signals are correlated. That is the strongest analytical point in the article, because GenAI risk rarely shows up in one control domain alone. Behavioural risk, access patterns, and threat intelligence each look incomplete in isolation, but together they reveal where a user or agent is drifting outside intended scope. That makes the case for joined-up governance across IAM, PAM, and security awareness.

Role-specific risk is the named concept security teams should operationalise now. The article shows that finance, development, legal, and security teams face different GenAI failure modes, which means generic training produces generic protection. A role-specific model lets security teams target the exact trust gaps that matter for that job function. The practical conclusion is to align AI governance, access policy, and training to role-based exposure rather than enterprise-wide averages.

What this signals

Role-specific risk is the operational pattern teams should prepare for. Finance, development, legal, and security functions face different GenAI failure modes, so a single training curriculum will keep missing the highest-risk behaviours. The practical move is to align policy, access boundaries, and simulations to role-based exposure rather than enterprise-wide averages.

Human Risk Management will increasingly depend on identity telemetry. The more AI tools and agents are embedded into SaaS workflows, the more important it becomes to correlate behaviour with identity events and threat signals. That makes the identity layer a core source of risk evidence, not just an access-control function.

If your programme already tracks NHI lifecycle and access scope, extend that discipline to AI agents now. The same governance patterns that constrain service accounts, ephemeral tokens, and privileged access can help contain AI-driven misuse before it turns into unbounded workflow authority.

For practitioners

• Inventory sanctioned and unsanctioned AI tools Build an approved AI register that ties each tool to business purpose, data classes allowed, and the identity controls that govern access. Include personal use exceptions and shadow AI detection in the same review cycle so usage cannot drift outside policy.
• Treat AI agents as managed non-human identities Assign each agent a named owner, explicit task scope, and least-privilege access boundaries. Review agent permissions, token lifetime, and data access paths with the same discipline used for service accounts and privileged workloads.
• Use role-specific simulations for GenAI risk Target training scenarios to the actual failure modes each team faces, such as finance deepfake fraud, developer code leakage, or analyst over-trust in generated output. Reinforce verification steps and escalation paths inside the workflow, not only in annual training.
• Correlate behaviour, identity, and threat signals Feed usage telemetry, identity events, and threat intelligence into one risk view so abnormal AI-related actions can be prioritised before they become incidents. This is especially important where a user and an AI agent both operate in the same SaaS workflow.

Key takeaways

• GenAI risk is no longer confined to human judgment errors because AI agents now operate as governed or unguided actors inside enterprise workflows.
• The evidence points to a control gap, not just an awareness gap, because many organisations still cannot track what their AI agents access or do.
• Security teams need role-specific training, identity-bound agent governance, and correlated telemetry if they want to reduce exposure rather than simply document it.

Key terms

• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• AI Agent Identity: The digital identity used by an autonomous AI agent to authenticate to external systems, APIs, and services. Managing AI agent identities is an emerging and rapidly evolving area of NHI security.
• Human Risk Management: The practice of managing how people interact with security controls, especially under pressure, distraction, or deception. It combines training, policy, and friction management so identity systems are still usable enough that users do not bypass them in day-to-day work.

What's in the full article

Living Security Human Risk Management Platform's full blog post covers the operational detail this post intentionally leaves for the source:

• Role-specific GenAI awareness examples for finance, development, and security teams
• The article's full treatment of shadow AI, hallucinations, bias, and deepfake-enabled social engineering
• Practical guidance on human-in-the-loop oversight and how Living Security frames predictive risk correlation
• The vendor's explanation of how HRM connects behaviour, identity, and threat data in one programme

👉 The full Living Security Human Risk Management Platform post expands the training model, risk examples, and HRM workflow implications.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity controls to the broader risk patterns created by AI agents and other non-human actors.