EU AI Act compliance exposes the identity gap in AI governance

At a glance

What this is: KOBIL argues that EU AI Act compliance is exposing a governance gap: companies must now inventory AI systems, classify their risk, and prove who accessed, changed, and approved them.

Why it matters: This matters to IAM and security teams because AI governance now depends on identity-bound accountability, access control, and auditable oversight across developers, operators, and business users.

By the numbers:

• Violations of the EU AI Act can result in fines of up to €35 million or 7% of global annual turnover, whichever is higher.

Context

The EU AI Act turns AI governance into a control problem as much as a legal one. Organisations now have to know where AI is used, who can touch it, what data it uses, and whether its outputs are visible and explainable enough to satisfy regulatory scrutiny. That is a familiar pattern for identity teams, but the twist is that AI systems increasingly sit inside third-party software, cloud services, and delegated business workflows.

For practitioners, the hardest part is not the regulation itself but the operating model it implies. Risk classification, documentation, logging, human oversight, and AI literacy all depend on accountable access and traceable change control. Where AI systems are shared across teams or embedded in vendor platforms, the identity gap becomes the compliance gap.

Key questions

Q: How should organisations govern embedded AI in third-party software under the EU AI Act?

A: Treat embedded AI as part of your own governance perimeter, even when the model or platform is supplied by a third party. You still need inventory, ownership, risk classification, logging, and an accountable approval path. If the vendor cannot provide traceable evidence, the compliance burden shifts back to your programme.

Q: How should SMEs choose an identity and access management approach that fits a limited budget without weakening security?

A: SMEs should start with core controls that address their highest risks, then expand as needs grow. Prioritise cloud-based IAM with subscription pricing, single sign-on, multi-factor authentication, user lifecycle management, and access control. These capabilities reduce upfront infrastructure cost, improve user experience, and support compliance without forcing a large custom implementation or unnecessary feature spend.

Q: What breaks when AI agent governance is treated as access control?

A: The control boundary breaks first. Governance tools can document what an agent is supposed to do, but they cannot stop the agent from authenticating, requesting privileges, or calling systems unless a separate runtime IAM layer enforces those decisions. That leaves a gap between policy intent and actual containment.

Q: How should security teams separate oversight of AI systems from normal user access?

A: Use distinct roles for development, testing, approval, and production operation, then require review workflows that cannot be bypassed by the same person or team. Oversight needs procedural separation and technical enforcement, not just policy statements.

Technical breakdown

Why EU AI Act compliance depends on identity-bound accountability

The EU AI Act creates obligations that are impossible to satisfy with policy alone. High-risk systems need documented risk management, traceable testing, human oversight, and version-controlled evidence of changes. In practice, that means compliance depends on being able to bind access, approval, and modification events to named identities, whether the actor is a developer, operator, auditor, or business approver. Without identity-bound logs and role separation, organisations can say what happened but cannot prove who did it or under what authority.

Practical implication: tie AI system changes, approvals, and review activity to verified identities and preserve those records in tamper-resistant logs.

Why third-party AI embeds widen the governance gap

The EU AI Act does not stop at internally built models. It also applies to AI functions embedded in purchased software, cloud platforms, and outsourced services, which means the organisation may own the regulatory duty without owning the system boundary. That makes inventory, supplier visibility, and delegated accountability central to AI governance. The real issue is not only whether the model is compliant, but whether the business can identify the AI component, its owner, its access paths, and its impact on users and regulated decisions.

Practical implication: extend AI inventory and ownership checks into SaaS, cloud, and third-party platforms before you rely on their embedded AI functions.

How human oversight becomes an access-control problem

Human oversight is often described as a governance principle, but the operational reality is access control, segregation of duties, and review workflow design. If the same people can train, deploy, approve, and monitor an AI system without separation, oversight becomes ceremonial. The EU AI Act pushes organisations toward stricter control over who can see training data, change model configuration, approve releases, and override outputs. That is where IAM, PAM, and auditability intersect with AI governance.

Practical implication: separate development, testing, approval, and production access for AI systems, and require review workflows that cannot be bypassed.

NHI Mgmt Group analysis

AI Act compliance is becoming an identity governance test, not just a legal checklist. The regulation only works in practice when organisations can prove who accessed, changed, approved, and monitored AI systems. That shifts AI governance into the same control territory as privileged access, logging, and accountability. For identity teams, the compliance question is no longer whether AI exists, but whether each AI action can be tied to a verified identity and an authorised role.

The identity gap is widest where AI is embedded in third-party platforms. Many organisations will not own the model, the hosting layer, or the vendor’s internal controls, yet they still carry the compliance burden for use of the system. That creates a delegated-governance problem: you need inventory, ownership, and evidence across services you do not directly operate. Practitioners should treat embedded AI as an extension of supplier governance and access governance, not as a standalone legal category.

Identity-bound logging is the named control concept that matters here. The article’s core operational insight is that revision-proof documentation is not enough unless actions are attributable to real people and constrained by role. That is the difference between having records and having defensible accountability. In AI governance programmes, identity-bound logging should become the baseline for proving oversight and separating responsibility across development, approval, and operation.

AI literacy will fail if it is treated as training only. The EU AI Act expects organisations to embed awareness of risk, bias, misuse, and escalation into operating practice. That means the human side of governance must be coupled to access design, approval authority, and exception handling. Where staff can act on AI systems without understanding their regulatory impact, the organisation has a control gap as well as a knowledge gap.

Digital sovereignty concerns reinforce, rather than replace, the identity question. The article frames infrastructure choice as strategic because Europe wants traceable, legally compliant AI operations within its own regulatory model. But sovereignty without governance is just localisation. Practitioners should use this moment to align AI platform decisions with identity architecture, auditability, and accountable control ownership.

What this signals

Identity-bound logging is quickly becoming a baseline control for AI governance. As organisations prepare for the EU AI Act, the practical challenge is proving that AI-related actions were performed by authorised people in the right role and at the right time. That shifts the programme from documentation-first thinking to control-first design, where auditability and access scope matter as much as model performance.

The more AI is embedded in vendor platforms, the more governance depends on supplier visibility and lifecycle control. Security and IAM teams should assume that the compliance risk will sit in the gaps between ownership, approval, and runtime oversight unless those responsibilities are explicitly assigned and continuously reviewed.

For practitioners

• Inventory every AI system and embedded AI function Build a live register that includes internal models, SaaS features, cloud-hosted capabilities, and third-party embedded AI so ownership and risk classification are never inferred.
• Separate AI development, approval, and production access Limit who can train, change, test, approve, and deploy AI systems, and make those roles visible in IAM and PAM workflows.
• Bind AI actions to verified identities in logs Require identity-bound logging for model changes, data access, approvals, and overrides so audit trails support both governance and regulatory evidence.
• Extend supplier governance to embedded AI Ask vendors how AI features are identified, configured, monitored, and audited, then map those answers to your own accountability model.

Key takeaways

• The EU AI Act turns AI oversight into a governance problem that depends on identity, access, and auditable responsibility, not just policy language.
• Embedded AI in third-party software expands compliance scope because organisations still need to inventory and classify systems they do not fully control.
• Role separation, traceable approvals, and identity-bound logs are the controls that make AI governance defensible in practice.

Key terms

• AI Governance: AI governance is the set of controls used to discover, classify, approve, restrict, monitor, and revoke AI-enabled access. It connects identity, data, and policy so organisations can manage what AI can reach, what it can share, and when it should be stopped.
• Identity-bound logging: Identity-bound logging is telemetry that ties a session, query, or action to a verified user or service identity. It is more valuable than raw activity logs because it supports accountability, incident investigation, and compliance evidence by preserving who did what, and under what authorisation path.
• High-Risk AI System: A high-risk AI system is one whose outputs can materially affect a person’s rights, opportunities, or safety. These systems need stronger oversight because errors, bias, or unauthorized actions can create legal exposure as well as security and trust problems.
• Human Oversight: Human oversight is the requirement that a person remains responsible for reviewing, approving, or correcting AI-driven output before it causes a material action. In governance terms, it is the control that prevents automation from becoming unowned authority.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and identity lifecycle controls that help security teams build defensible access models. It is useful for practitioners aligning AI governance, IAM, and accountability across modern systems.