AI model pre-release testing exposes the governance gap in frontier AI

At a glance

What this is: This is an analysis of a U.S. executive order that creates voluntary pre-release testing for frontier AI models judged capable enough to be “covered.”

Why it matters: It matters because AI governance teams now have to plan for external review, model-risk scrutiny, and inconsistent policy expectations across jurisdictions.

👉 Read Swarmnetics' analysis of the Trump executive order on AI model pre-release testing

Context

Frontier AI model governance is moving from internal safety review toward external scrutiny, but the control model is still unsettled. The article centres on a U.S. executive order that would let government agencies evaluate models before release if they are deemed threat-capable enough to be covered, with the National Security Agency helping define the benchmarking process.

For identity and access teams, the relevant issue is not only model safety but who can approve, test, and release systems that may later act as agents, tools, or decision makers. When AI models become operational systems rather than static software, governance has to account for delegation, evaluation authority, and downstream access to tools and data.

Key questions

Q: How should organisations govern frontier AI models before release?

A: Organisations should require a formal approval gate before any frontier model is released or connected to tools, data, or users. That gate should combine benchmarking, red-team assessment, ownership sign-off, and a clear decision record. If the model cannot be tested consistently, it should not move into production until the control gap is closed.

Q: Why do AI evaluations need identity and access context?

A: Because many AI failures happen through who can retrieve, prompt, or act on data, not just through model quality. If evaluations ignore permissions, connectors, and retrieval paths, they miss the mechanisms that let sensitive information surface in production. Identity context makes the evidence operational.

Q: What do security teams get wrong about AI safety testing?

A: The common mistake is treating AI safety testing as if it were just another security scan. It is not. Safety testing is about proving how a model or agent fails under pressure, while traditional security tooling is about who can access the system. Those are different governance questions and need different evidence.

Q: Who should be accountable for AI risk when multiple teams deploy models?

A: Accountability should sit with named lifecycle owners, backed by a governance forum that includes legal, privacy, security, data and business leads. Shared responsibility does not mean shared ambiguity. Each model needs one accountable owner who can answer for the data, use case, controls and retirement state.

Technical breakdown

How pre-release model evaluation changes the AI control plane

Pre-release evaluation creates a control point before public deployment, where a model is inspected for threat capability, misuse potential, and national-security risk. In practice this looks less like traditional code review and more like a gated assurance process that combines red teaming, benchmark testing, and policy classification. The challenge is that the process only works if the benchmark is consistent, the scope is clear, and the release decision has enforceable consequences. Without that, testing becomes advisory rather than control-bearing.

Practical implication: Security and AI governance teams should treat pre-release review as a formal approval gate, not a documentation exercise.

Why voluntary frontier AI testing can still reshape governance

A voluntary regime can still change behaviour when access to markets, buyers, or public trust depends on passing evaluation. That is especially true for frontier models, where developers may accept external scrutiny to reduce policy risk or demonstrate credibility. The weakness is obvious: voluntary participation creates uneven coverage, and the most capable or most controversial systems may be the least likely to submit. The governance question is therefore not whether testing exists, but whether it is broad enough to matter.

Practical implication: Program owners should map which models would remain outside any voluntary review path and how that affects residual risk.

Model benchmarking is becoming an identity and access problem

Once an AI model can be evaluated, approved, and later connected to tools, it starts to resemble a governed runtime entity rather than a static artifact. That creates an identity problem around who is authorised to test it, who can alter its behaviour, and which systems it can subsequently access. In agentic environments, evaluation and delegation are linked, because a model that passes assessment may still be unsafe once it can call tools, query data, or trigger actions. Governance must therefore span model risk and access control together.

Practical implication: Identity teams should align model approval workflows with tool-access and delegation controls before models are released.

Threat narrative

Attacker objective: The objective is to exploit an under-governed AI model path so that capability, access, or delegation risk is discovered only after operational exposure.

1. Entry occurs when a frontier model is released or evaluated without a consistent benchmark, leaving its capabilities insufficiently characterised before deployment.
2. Escalation follows when the model is connected to tools, data, or workflows that expand its effective authority beyond the original test conditions.
3. Impact arises if an inadequately assessed model is used in sensitive environments and enables misuse, unsafe automation, or downstream security failure.

Breaches seen in the wild

• DeepSeek breach — DeepSeek breach exposed 1M+ log lines and sensitive secret keys.
• Poland Military Breach — Polish military email credentials compromised exposing sensitive government communications.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Voluntary pre-release testing is a governance signal, not a governance solution. The order recognises that frontier models can create security externalities before release, but voluntary participation means assurance remains uneven. In practice, the most sensitive systems may still be the least consistently reviewed. Practitioners should read this as a cue to build their own model acceptance gates rather than waiting for policy to harden.

AI model review is converging with identity governance. Once a model can be tested, approved, and connected to tools, it becomes a governed actor in the environment. That means approval alone is not enough, because post-approval access to data, APIs, and execution paths determines real risk. Identity and AI governance teams should treat model release as the start of lifecycle control, not the end of validation.

Fragmented state and federal AI rules will push enterprises toward their own control baselines. The article points to a patchwork risk, where organisations cannot rely on a single regulatory timetable to define acceptable model assurance. That increases the value of internal policy, common benchmarking, and repeatable approval criteria. Security leaders should prepare for model governance to become a standing control domain, not a one-time legal review.

Model benchmarking needs a named failure mode: release-before-assurance drift. The core risk is that models reach users, tools, or sensitive data paths before evaluation standards catch up. That failure mode is especially dangerous for frontier systems that can be repurposed after release. Practitioners should define clear thresholds for when a model cannot move into production until testing and access controls are both complete.

From our research:

• Only 1.5 out of 10 organisations are highly confident in their ability to secure NHIs, compared to nearly 1 in 4 for securing human identities, according to The State of Non-Human Identity Security.
• Lack of credential rotation is cited as the top cause of NHI-related attacks by 45% of organisations, followed by inadequate monitoring and logging (37%) and over-privileged accounts (37%).
• For a wider lifecycle view, see Ultimate Guide to NHIs , Lifecycle Processes for Managing NHIs for provisioning, rotation, and offboarding control patterns.

What this signals

Release governance for frontier models is increasingly an identity problem as much as an AI problem. Once models can act through tools and service accounts, approvals need to cover delegated access, not just model quality. Teams that already struggle with non-human identity visibility will find the same control gaps reappearing in AI deployment workflows, especially where model-connected access is implicit rather than inventoried.

Model assurance will become a standing control domain for security programmes that already use the NIST Cybersecurity Framework 2.0. The practical shift is toward repeatable evidence collection, approval records, and runtime monitoring for model-enabled systems. That means linking AI governance to identity lifecycle, logging, and response processes rather than treating it as a separate policy exercise.

Release-before-assurance drift: the longer organisations let models move from evaluation into production without a hard access boundary, the harder it becomes to prove what the system can do, what it can touch, and who approved it. That is the same governance pattern that makes unmanaged NHIs so hard to contain, only now it is extending into AI operations.

For practitioners

• Define a pre-release approval gate for frontier models Require documented benchmarking, red-team results, and sign-off before any model is connected to tools, data, or production workflows. Treat release approval as a control point with explicit owners and escalation paths.
• Separate model validation from tool delegation Make sure a model that passes evaluation does not automatically inherit access to APIs, internal data, or execution environments. Use least privilege for model-connected services and review every delegated capability.
• Inventory models that bypass voluntary review Identify which internal or third-party models are outside any expected external testing path and record the residual risk they introduce. This is essential where business teams can procure or deploy models without central approval.
• Align AI governance with identity controls Tie model lifecycle approvals to access policies, service accounts, secret handling, and logging so that release decisions cannot be detached from runtime privilege. That reduces the chance of post-approval expansion into sensitive systems.

Key takeaways

• Frontier AI governance is shifting from internal review to external scrutiny, but voluntary testing still leaves coverage gaps.
• The real control problem is not model quality alone. It is whether approval, delegation, and runtime access are governed together.
• Security teams should build release gates, inventory bypass paths, and tie model approval to identity controls before frontier AI reaches production.

Key terms

• Frontier AI model: A frontier AI model is a high-capability system near the cutting edge of what the market can deploy. In governance terms, it requires stronger assurance because its behaviour, misuse potential, and downstream impact can exceed the assumptions used for ordinary software release.
• Pre-release evaluation: Pre-release evaluation is the process of testing a model before it is made public or connected to operational systems. It typically combines benchmarking, red teaming, and risk review so that release decisions are based on evidence rather than vendor assurance alone.
• Model Governance: Model governance is the set of controls that decides which foundation models can be used for which agent types and use cases. It links platform choice to security policy, because the model selection influences data exposure, tool behaviour, and the risk profile of the resulting agent.
• Delegated Access: Delegated access is permission granted to one identity to act on behalf of another user, service, or system. In NHI environments, this usually appears in OAuth-connected apps and automation tooling. It is powerful, but it must be tightly scoped and reviewed because it can persist long after the original business need ends.

What's in the full analysis

Swarmnetics' full analysis covers the operational and policy detail this post intentionally leaves for the source:

• How the proposed pre-release evaluation process would work across agencies and trusted partners
• The implications of a 30-day private testing window for frontier model release timing and assurance
• What the benchmarking process may mean for developers that want to remain in voluntary review programs
• How state-level AI laws could interact with federal expectations in a patchwork compliance environment

👉 Swarmnetics' full article covers the agency roles, benchmarking process, and policy trade-offs in more detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity lifecycle control to the broader security programmes they already run.