How should organisations govern model provenance and tool access together?

Why provenance and tool access belong in the same governance model

Model provenance and tool access only look separate if you treat the model as static software and the tools as a runtime detail. In practice, the model artefact, its prompt templates, retrieval corpus and connected plugins form one operating chain. Governance has to cover who introduced each component, who may change it, and which policy authorises its use.

This matters because the trust decision is not “is the model approved?” or “is the tool approved?”; it is whether the combined system can be trusted to execute with the intended data, instructions and reach. If provenance is weak, you may not know what you actually deployed. If tool access is weak, you may know the model but not what it can do.

For organisations building an AI control plane, the most useful mental model is end-to-end change control. The published model, the retrieved context and the invoked plugin should all sit inside the same traceable approval path, so provenance evidence and runtime authorisation can be reviewed together rather than by separate teams with different records.

What has to be owned, authorised and traceable

Governance should assign a clear owner for each artefact class: model files, system prompts, retrieval corpora, tool connectors, policy rules and deployment configurations. Ownership is not just an administrative label, it is the basis for change approval, rollback and incident attribution. If no named owner can explain why an artefact exists, it is already outside effective governance.

Authorisation needs to extend beyond the model repository into the actions the system can take at runtime. A plugin that can read tickets, a retrieval index that can surface sensitive documents, or a connector that can trigger downstream workflows all widen the effective authority of the AI system. That authority should be explicitly granted, bounded and reviewable, not assumed because the interface is convenient.

Traceability is the other half of the control. Teams should be able to answer which artefact version was used, which policy allowed it, which data sources were in scope, and which tools were reachable at the time of execution. A governance record that stops at model registration leaves a blind spot where the highest-risk decisions often occur.

How to govern the supply chain and runtime together

The practical pattern is to tie provenance checks to authorisation checks. A model or prompt should not move into production unless its origin, integrity and approval state are known, and the tools it can invoke are constrained to the minimum required set. That creates a single control path for admission, change and usage.

For broader AI supply-chain hygiene, AI Supply Chain Security and AI-BOM Guide is the most direct internal reference because it covers model provenance, AI-BOM content and credential containment together. When organisations need to compare access models for people, workloads and AI agents, the Authorisation Models Guide helps translate policy intent into enforceable access boundaries.

At the external control level, SLSA supports provenance and integrity for software artefacts, which is useful when AI components are packaged, built or promoted through controlled pipelines. For organisations that need a broader AI governance wrapper, ISO/IEC 42001:2023 AI Management System Standard provides the management-system discipline that keeps approvals, accountability and continual improvement linked to AI deployment.

Where governance usually breaks down in practice

One common failure is provenance without authority. Teams may verify that a model came from the right source, yet still allow broad tool access because the runtime layer was treated as an engineering detail. That creates a system that is traceable but overpowered.

The opposite failure is authority without provenance. A plugin policy may be tight, but if the underlying prompt, retrieval corpus or model version changes without review, the organisation is no longer governing the actual system in use. In that case, the approval record becomes stale while the live behaviour changes underneath it.

Another recurring issue is split ownership. Security, ML engineering, platform engineering and product teams each hold part of the picture, but no one owns the combined control outcome. The result is duplicated review for low-risk changes and missed review for high-impact ones, especially when tool scopes expand quietly over time.

Risk and Threat Considerations

Weak provenance and broad tool access create a compound exposure: an untrusted or altered artefact can be paired with excessive runtime authority, turning a governance gap into operational abuse. The risk is highest when prompt, model or retrieval changes are possible without the same approval standard as tool permissions.

Failure mechanism: an attacker or careless insider can introduce or swap an artefact, or expand tool reach, without a single control point noticing that the deployed system has changed in both content and authority. Once the model can call tools or surface sensitive context, compromise of one layer can cascade into the other.

Impact: organisations can end up with invisible policy drift, unauthorised data access, malicious workflow execution, or outputs that appear legitimate but are produced by a system whose provenance and authority no longer match the approved design.

Practitioner Guidance

What to prioritise: treat provenance review and tool authorisation as one release gate. If you cannot say who approved the artefact and who approved the reachable tools, the deployment is not ready.

What to verify: confirm that each production model, prompt bundle, retrieval source and plugin has an owner, a version history and an access policy. The minimum bar is that change evidence and runtime permissions can be reconciled after the fact.

Decision rule: if a component can influence model behaviour or expand the system’s action space, govern it with the same seriousness as code that ships to production. Do not exempt prompts, corpora or connectors because they are “configuration”.

Practitioner takeaway: the control objective is not merely to approve a model or to restrict a tool, but to ensure the approved artefact and the allowed authority remain bound together throughout the system’s lifecycle.

Related resources from NHI Mgmt Group

• How should security teams govern non-human identities that have persistent access?
• How should security teams govern API keys used for generative AI access?
• How should organisations govern enterprise and privileged access together?
• How should organisations govern physical and cyber access together?