Fine-tuned model provenance needs graph verification, not just signing

At a glance

What this is: This is an analysis of why signed AI artifacts can still conceal unverified training inputs, and why provenance for fine-tuned models has to be verified as a graph rather than as a single object.

Why it matters: It matters because IAM, NHI and AI governance teams need assurance over what was consumed to build an AI system, not just whether the final artifact was signed after the fact.

Context

A signed artifact proves integrity at the point of verification, but it does not prove that the build was assembled from approved inputs. In AI and model supply chains, that gap matters because provenance, lineage and consumption history are governance controls in their own right, not just documentation.

Fine-tuned models are especially difficult because the output inherits risk from multiple upstream artifacts, such as a base model and a dataset. The security problem is not only tampering after build, but unverified substitution before and during assembly.

Key questions

Q: How should teams prove what a fine-tuned model was built from?

A: Teams should prove composition by recording every consumed input by digest at build time and attaching a signed provenance attestation to the output. The attestation should name the base model, dataset and build parameters, while verification should recurse through each upstream artifact to confirm its own required attestations before the model is approved.

Q: Why do signed AI artifacts still leave a governance gap?

A: Because a signature proves integrity of the final artifact, not that the artifact was assembled from approved or policy-compliant inputs. If the build process does not bind source digests into the provenance record, an unverified dataset or base model can flow into production while the output still verifies cleanly.

Q: How should teams validate third-party model provenance before deployment?

A: They should validate provenance as part of a formal intake process that includes source review, conversion review, and trigger testing. The goal is to confirm not only where the model came from but also what behaviour it can still execute after transformation.

Q: What is the difference between an SBOM and a provenance attestation?

A: An SBOM describes what should be in the model lineage, while a provenance attestation is a cryptographic statement about what the build process actually consumed. The SBOM gives visibility into relationships such as training inputs; the attestation gives proof that those relationships were real at build time.

Technical breakdown

Why single-artifact attestations stop short

A signed attestation answers a narrow question: is this artifact the same one that was scanned, approved or stored? That is integrity, not provenance. In an AI pipeline, the final model can be authentic while still being assembled from inputs that were never validated together. The model registry may hold the output, the base model and the dataset, but without explicit dependency links the security control only covers each object in isolation. That leaves a governance blind spot where the build is trusted, yet the composition is not. See the SLSA and Build Provenance Guide.

Practical implication: Treat final-model signatures as necessary evidence, not as proof of safe composition.

Why fine-tuning creates a provenance graph

Fine-tuning is not a linear chain of one input becoming one output. It is a graph, because the resulting model depends on multiple upstream artifacts, each with its own history, controls and trust status. The important object is the relationship between them: which base model and which dataset were consumed, and at what digests. Standards such as SLSA and in-toto can express build provenance, while transparency logs help preserve ordering and auditability. The technical requirement is to bind those input relationships directly into the output attestation so they can be verified recursively. See the CI/CD Pipeline Identity Security Guide.

Practical implication: Record resolved dependencies by digest at build time so downstream verification can follow the full model lineage.

What SBOMs record and what they do not

An SPDX 3 SBOM can describe lineage by listing training inputs and dependency relationships such as TRAINED_ON or GENERATED_FROM. That makes it useful as a manifest of composition and dependency structure. But an SBOM is descriptive, not evidentiary. It does not prove that the pipeline actually consumed those exact inputs, nor that each input met policy requirements at the time of use. The stronger control is a signed provenance attestation from the build system, because it is a cryptographic statement about what was actually consumed during execution.

Practical implication: Use SBOMs for lineage visibility and signed provenance for proof of consumption and policy compliance.

Threat narrative

Attacker objective: To get a trusted-looking model into production even when its upstream training inputs were not approved or verified.

1. Entry occurs when a pipeline or operator substitutes an unapproved base model or dataset before fine-tuning begins, while the final artifact still appears valid after signing and scanning.
2. Credential or provenance abuse occurs when the build process fails to bind consumed inputs to their digests, allowing the wrong source artifacts to flow into the output without breaking the final signature.
3. Impact is that the organisation ships a model whose attestations verify in isolation even though its upstream composition was never proven or policy-checked.

Breaches seen in the wild

• Massive Docker Hub Secrets Leak: 10,000+ Docker Hub container images expose hardcoded secrets and authentication keys.
• Secrets in Docker Hub images (RWTH Aachen study): A 2023 RWTH Aachen study found secrets in 8.5% of container images, and 275,269 internet hosts still using the leaked private keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Artifact signatures are not provenance proofs. The article correctly separates integrity from composition, and that distinction matters across AI supply chains as much as it does in software delivery. A signed model can still hide unverified upstream data, inherited bias or policy failures if the build record does not bind inputs by digest. Practitioners should treat the final signature as one control layer, not the whole assurance story.

Fine-tuning creates a provenance graph, not a linear artefact trail. That is the central operational insight here, and it is why isolated attestation checks do not scale to model composition. The governance problem is not only whether an output is signed, but whether every upstream dependency was consumed as intended and at the right version. Teams need graph-aware verification to make provenance meaningful in production.

SBOM lineage helps, but it does not replace build-time proof. The article is right to position SBOMs as descriptive and attestations as evidentiary, because many programmes still collapse those two roles into one. An SBOM can tell you what should have been used; a provenance attestation tells you what was actually used. Provenance debt: when teams document model lineage without proving consumed inputs, they accumulate an assurance gap that only appears at audit or incident time.

Identity controls now extend into model assembly pipelines. Fine-tuning workflows are becoming a governance boundary for AI systems, which means the trust model around builders, registries and signing keys matters as much as the model itself. Where the article touches on signed attestations and registry workflows, the intersection with CI/CD pipeline identity security is real: build identities must be able to prove what they consumed, not just publish what they produced. Practitioners should align pipeline identity, registry trust and provenance policy as one control surface.

Recursive verification is the missing control plane. The article exposes a tooling gap that many organisations still assume is already solved. Verifying one signed object is easy; verifying the chain or graph behind it requires orchestration, policy logic and recursion across registries and attestations. That is the point where model governance becomes an operational discipline rather than a documentation exercise, and practitioners should design for graph verification rather than after-the-fact review.

From our research library:

• 7% of security leaders admit they do not know how often their AI systems are making autonomous changes to infrastructure, according to the 2026 Infrastructure Identity Survey.
• Read next: SLSA and Build Provenance Guide

What this signals

Provenance debt: model programmes that stop at artifact signing create a hidden assurance gap, because lineage documentation and lineage proof are not the same control. As AI systems move from single models to assembled pipelines, graph verification becomes the practical boundary between auditable trust and paper compliance.

The governance model for AI supply chains now has to include build-time identity, digest pinning and recursive verification as first-class controls. That shifts the operational question from whether a model was signed to whether the pipeline can prove the exact inputs it consumed.

For practitioners

• Record consumed inputs by digest Capture the base model and dataset digests before fine-tuning starts, and write them into the build provenance so downstream verification can match what was actually used.
• Bind provenance to the build identity Ensure the pipeline identity that runs fine-tuning is the same identity that signs the resulting attestation, so the provenance record is attributable and auditable.
• Verify the upstream attestation chain recursively Check the final model, then resolve each resolved dependency and confirm its required import or scan attestations before promotion to production.
• Use SBOMs as lineage manifests, not evidence Keep SPDX 3 relationships for dependency visibility, but require signed provenance attestations for any control decision that depends on actual consumption history.

Key takeaways

• A fine-tuned model can be legitimately signed and still hide unverified upstream inputs if the build record does not bind the inputs by digest.
• The article’s core evidence is that isolated attestation checks do not follow the model graph, so a substitution in the base model or dataset can remain invisible.
• Provenance becomes enforceable only when build-time input capture, signed attestation and recursive verification are treated as one control system.

Related NHI Mgmt Group resources: SLSA and Build Provenance Guide · CI/CD Pipeline Identity Security Guide · AI Governance Guide

Key terms

• Provenance Attestation: A provenance attestation is evidence that software was produced through a known pipeline or workflow. It helps verify origin, but it does not guarantee that the workflow was free of compromise, poisoned caches, or attacker-controlled inputs.
• Resolved Dependency: A resolved dependency is a specific input artifact identified by immutable digest at build time. It matters because registry tags can move, but digests pin the exact model, dataset or package that was consumed when the output was created.
• Provenance Graph: A provenance graph is the relationship map between an output artifact and every upstream input, transform and attestation that contributed to it. In fine-tuning workflows, it is the only structure that can prove how multiple inputs flowed into one released model.
• Model Lineage: Model lineage is the traceable record of what data, code, training runs, evaluations, and approvals produced a deployed AI model. It is the trust chain for machine learning operations, because it lets security and risk teams verify provenance, investigate changes, and support rollback or audit requirements.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity and secrets management for practitioners who need to connect identity controls to modern build and AI supply chains. It is designed for teams that have to turn governance into operational assurance across production systems.