EU AI Act readiness depends on evidence, not policy alone

At a glance

What this is: DeepKeep frames EU AI Act readiness as an evidence problem for high-risk AI systems, where policies must be backed by testing, logs, oversight, monitoring, and incident records.

Why it matters: That matters because IAM, AI security, and governance teams now need auditable proof of control effectiveness, not just written commitments, especially when AI systems change after launch.

Context

The gap is not whether an organisation has an AI policy, but whether it can prove the policy was enforced when the system was tested, used, and monitored. For high-risk AI, the practical problem is evidence: logs, test traces, control decisions, and post-incident records that stand up to audit and regulatory review.

The EU AI Act ties readiness to lifecycle governance, which means the control surface includes data, documentation, transparency, oversight, robustness, and monitoring. That makes AI governance a cross-functional evidence problem, not a one-off legal exercise, and the same logic applies wherever EU AI Act compliance has to be translated into operational proof.

Key questions

Q: What should teams do first when they need to prepare for EU AI Act compliance?

A: The first practical step is to identify every AI system in use, including chatbots, models, and agents, then prioritise the highest-risk deployments. After that, map the relevant obligations, assign ownership, and close gaps in data quality, testing, transparency, and oversight. Starting with critical systems prevents wasted effort and gives leaders a realistic compliance roadmap.

Q: Why do AI compliance programs fail when they rely on launch-time approval?

A: Launch-time approval fails because AI systems keep changing after deployment. Models, prompts, retrieval sources, tools, and usage patterns all shift the risk profile. A system that was acceptable at release can become non-compliant later. Mature programs treat change as a trigger for revalidation, not as a reason to assume old evidence still applies.

Q: How can organisations tell whether AI governance is actually working?

A: Organisations can tell AI governance is working when they can inventory every agent, explain its purpose, show who owns it, and prove that permissions are tightly scoped. If those four things are missing, the programme has policy language but not operational control. Auditors will notice the gap quickly.

Q: Which AI governance controls matter most when regulators can inspect documentation later?

A: The most important controls are version ownership, change tracking, release gating, and post-market monitoring. Together they keep the evidence package current, show accountability for each update, and give auditors a consistent trail from design to deployed behaviour.

Technical breakdown

Why policy statements are not enough for high-risk AI systems

A policy describes intent, while evidence shows control operation. Under the EU AI Act, high-risk systems need technical documentation, logs, and records that prove what was tested, what failed, what was blocked, and what was fixed. Without that chain, a policy cannot demonstrate that the system stayed within approved behaviour. The real challenge is reconstructability: can the organisation explain a control decision after the fact with artefacts, not memory? This is why AI governance has to be instrumented like a security process, not managed like a document library. See the AI Risk Assessment Guide.

Practical implication: Treat policy as the control statement and require supporting artefacts for every material AI decision.

How lifecycle change breaks launch-time compliance models

High-risk AI systems are not static. Models change, prompts change, retrieval sources change, and agents receive new tools or permissions. That means a system that was compliant at launch can drift out of bounds quickly, even if the original assessment was sound. Continuous iterative risk management matters because the threat surface shifts with usage, integrations, and model updates. In practice, AI readiness depends on revalidation after material change, not on a single pre-release review. See the AI Security Maturity Model.

Practical implication: Reassess controls after model, data, prompt, or tool changes rather than relying on launch approval.

What useful AI logs need to capture

Useful logs are not just timestamps. They need to show the system involved, the policy or guardrail triggered, the input and output context, the action taken, and whether the event was escalated, blocked, redacted, or allowed. For governance and incident response, logs become evidence of control effectiveness and traceability. If the record cannot answer who, what, why, and what happened next, it is operationally weak even if it is technically present.

Practical implication: Define log content around decision traceability, not around raw event volume.

Threat narrative

Attacker objective: The practical objective is to push the AI system outside its governed operating boundary and leave the organisation unable to prove what occurred.

1. Entry occurs when employees or developers use high-risk AI systems in live workflows without sufficient visibility into prompts, retrieved context, or agent actions.
2. Escalation happens when the system follows unsafe instructions, exposes sensitive data, or takes tool actions that exceed the expected control boundary.
3. Impact appears when the organisation cannot reconstruct what happened, prove which control fired, or show that the issue was remediated and retested.

Breaches seen in the wild

• 12,000 secrets in LLM training data: Truffle Security found 11,908 live API keys and passwords hard-coded in web pages captured by Common Crawl, a dataset used to train LLMs.
• Microsoft SAS token exposure 2023: An over-permissive Azure SAS token in a Microsoft AI GitHub repo exposed 38TB, including workstation backups and Teams messages, for 3 years.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

EU AI Act readiness is now an evidence architecture problem, not a paperwork exercise. The article is right to frame compliance around proof, because high-risk AI obligations only become defensible when documentation, logs, tests, and control actions line up. Security teams are the natural custodians of that evidence layer because they already operate the monitoring and response machinery. The practitioner conclusion is that AI governance must be built as a traceable control system, not a narrative statement.

Continuous iterative risk management is the named governance gap this article surfaces. One-off launch reviews do not survive model updates, prompt changes, tool additions, or new retrieval sources. That makes the real failure mode stale assurance, where yesterday's assessment is treated as today's evidence. The practitioner conclusion is to govern AI changes as lifecycle events, not as isolated releases.

Human oversight only works when the system is observable. The article correctly links oversight to visibility, because a person cannot intervene meaningfully without knowing what the system did, why it did it, and whether the output was constrained. This is especially important where AI agents act with access to tools, files, or APIs, because oversight depends on more than a user sitting in the loop. The practitioner conclusion is to connect oversight design to logging, policy enforcement, and escalation paths.

AI security and IAM now meet at control evidence, not just access control. Where AI systems can access email, files, databases, or business applications, identity governance has to extend to the agent's effective permissions and recorded actions. That is where the intersection with agentic AI security policy becomes operational, because the question is no longer only who may use the system, but what the system can do and how that is evidenced. The practitioner conclusion is to treat AI access as governed runtime identity, not static application access.

From our research library:

• Only 23% of IT leaders were very confident in their organisation's ability to manage security and governance for GenAI deployments, according to a 2025 Gartner survey of 360 IT leaders.
• Read next: AI Risk Assessment Guide

What this signals

Evidence-first governance is becoming the practical shape of AI compliance. The organisations that will cope best with EU AI Act obligations are those that can convert monitoring, testing, and response into durable artefacts, not those that rely on policy wording alone.

The deeper shift is that AI governance now behaves like security operations: change management, logging, oversight, and incident handling all have to be evidentially linked. Where AI agents or high-risk systems touch business data, the control question becomes whether the programme can prove runtime behaviour, not whether the paperwork was complete.

For practitioners

• Build an evidence pack for every high-risk AI system Collect test traces, policy decisions, logged events, remediation records, and retest results so audits can reconstruct what happened end to end.
• Reassess controls after material AI change Trigger review when the model, prompts, retrieval sources, integrations, or agent tools change, because launch-time approval does not prove current compliance.
• Define logs around control decisions Capture which system acted, which policy fired, what response occurred, and whether the event was blocked, redacted, flagged, or allowed.
• Map AI oversight to clear intervention paths Give reviewers enough context to recognise failure, understand output risk, and escalate or intervene when the system behaves outside policy.
• Separate legal findings from technical evidence Use technical telemetry, testing artefacts, and monitoring records to support legal assessments rather than treating policy language as proof.

Key takeaways

• The core risk is not the absence of AI policy but the absence of proof that controls worked when the system was tested, monitored, and used.
• High-risk AI readiness depends on lifecycle evidence, including logs, test results, oversight records, and post-change revalidation.
• Security and governance teams need operational artefacts that can survive audit and incident review, not just compliance language.

Related NHI Mgmt Group resources: AI Risk Assessment Guide · AI Security Maturity Model · Agentic AI Security Policy Template · EU AI Act Compliance Guide

Key terms

• High-Risk AI System: A high-risk AI system is one whose outputs can materially affect a person’s rights, opportunities, or safety. These systems need stronger oversight because errors, bias, or unauthorized actions can create legal exposure as well as security and trust problems.
• Technical Documentation: The evidence package that describes how an AI system was built, tested, and controlled. It typically includes architecture, training data provenance, performance results, known limitations, and monitoring plans, and it must be complete before assessment begins.
• Post-market monitoring: Post-market monitoring is the ongoing collection and review of system behaviour after deployment so emerging risks, drift, and incidents can be detected and corrected. In regulated AI programmes, it is part of the evidence chain and must connect operational telemetry back to governance decisions.
• Human Oversight: Human oversight is the requirement that a person remains responsible for reviewing, approving, or correcting AI-driven output before it causes a material action. In governance terms, it is the control that prevents automation from becoming unowned authority.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps practitioners connect identity control to the broader security and governance work their programmes already own.