AI Model Provenance and Watermarking Guide: Model Signing and Lineage

Most organisations cannot prove that the model running in production is the one they evaluated, or show where a piece of AI-generated content came from once it has left their systems. Those are two different provenance problems. The first is about models: where weights came from, what they were derived from, and whether they were changed on the way to deployment. The second is about outputs: marking AI-generated text, images, audio and video so they can be recognised later. This guide covers model provenance and lineage, model signing with the OpenSSF model signing specification, AI content watermarking including SynthID, what watermarks cannot do, provenance metadata, and the EU AI Act marking duty. It is for ML platform engineers, AI security teams and governance leads. For the wider picture see the AI Supply Chain and AI-BOM Guide.

Key takeaways

• Model provenance and content provenance are separate controls. Signing and lineage records protect the models you consume and ship; watermarking and provenance metadata label the content those models produce.
• Sign models like software, verify them like software. OpenSSF model signing hashes every file in a model directory and signs the manifest, so any changed file fails verification at deployment.
• A signature proves who and what, not that a model is safe. Verification shows a model has not been tampered with since signing by a known identity; it says nothing about poisoned training data or a malicious signer.
• Watermarks are a signal, not proof. NIST reports that "most current watermarking schemes have been consistently found vulnerable to removal", that watermarks can be spoofed, and that text watermarks fail on low-entropy text.
• The EU AI Act now requires machine-readable marking. Article 50(2) applies from 2 August 2026, and systems already on the market before then must comply by 2 December 2026.

Two provenance problems

NIST AI 100-4 defines provenance data tracking as "Recording information about the origins and history for digital content" and stresses that "None of these techniques offer comprehensive solutions on their own." Plan for layers.

Model provenance and model lineage

AI model provenance is the verifiable record of where a model came from and what happened to it. Model lineage is the chain behind it: base model, fine-tuning runs, merged adapters, quantisation and conversion steps, each with its inputs. The OpenSSF model signing project starts from the gap that, for models downloaded from public hubs, "there is no means to trace the provenance of the model back to its creator" and no indication that an uploaded model "matches what was intended during training."

A usable lineage record for each model version holds:

• Identity: model name, version, and the digest of every file (weights, tokenizer, configuration).
• Origin: source hub or vendor, licence, and the signer identity if the publisher signed it.
• Parents: the digest of the base model and of each adapter or merged component.
• Process: training or fine-tuning job ID, code commit, data set versions and digests, build environment.
• Transformations: conversion to another format, quantisation or pruning, each producing new digests that must link back to the source.
• Approvals: evaluation results and who approved promotion to production.

These records belong in the AI-BOM. What this guide adds is that each link should be signed, so the chain can be verified rather than trusted. Build-system provenance for the code around the model follows the same pattern; see the SLSA and Build Provenance Guide.

Model fingerprinting

Model fingerprinting means deriving an identifier for a model. Two kinds are in use and should not be confused:

• Cryptographic fingerprints: digests of the model files, as computed by the model_signing digest command. They change if a single byte changes, which makes them right for integrity checks and wrong for recognising a lightly modified copy.
• Similarity fingerprints: identifiers designed to stay similar when content changes slightly. C2PA calls this kind of identifier a soft binding, one that is "not statistically unique, such as a fingerprint", and states that "a soft binding shall not be used as a hard binding". Use them to find likely copies or derivatives, never as proof of integrity.

Model signing with OpenSSF model signing

OpenSSF model signing (OMS) v1.0 was announced on 4 April 2025 by the OpenSSF AI/ML working group as "a library and CLI for signing and verification of ML models" supporting "any type of model format and models of any size." It is developed in the sigstore/model-transparency repository.

• What is signed: the tool hashes the files in a model directory (it currently supports "only files and file shards") and builds an in-toto statement whose subjects are "a list of (file path, digest) pairs". Signed models therefore cover weights, tokenizer and configuration together, not one file.
• Signature format: a Sigstore bundle in JSON, holding a DSSE envelope with the statement, its signature and the verification material. The format is the same whichever key type is used.
• Signing identities: Sigstore keyless signing is the default, using an OpenID Connect flow to obtain a short-lived signing certificate tied to an identity such as a CI workload. Self-signed certificates, public and private key pairs, and PKCS #11 hardware keys are also supported.
• Verification: the verifier must state the expected identity and issuer (or key) explicitly; the project notes there are "no sensible defaults", and the identity and provider used at signing "must be reused during verification". A passing check shows the model "hasn't been tampered with after training" by anyone other than that signer.

The same lessons as code signing apply: the signing identity is the control, and a stolen publishing credential lets an attacker sign or publish whatever they like. The LiteLLM PyPI package breach, where a leaked publishing token let attackers ship a malicious release of an AI library, is the pattern to design against. See the Code Signing Security Guide.

Model provenance verification in practice

The OpenSSF proposal is to sign during training and check signatures "each time the model is used", with verification points at hub upload, deployment and use as input to another model. In an enterprise that becomes:

1. Intake: when a third-party model is admitted, verify the publisher's signature if one exists, record digests either way, and re-sign with your own intake identity so internal systems verify one trust root.
2. Training and fine-tuning: the pipeline's workload identity signs each output model and records the parent digests, so lineage is attested by the system that produced it.
3. Registry: store the signature bundle next to the model; reject uploads without one.
4. Deployment gate: the serving platform verifies the signature against an allowlist of signer identities before loading weights, and refuses to load on failure.
5. Runtime: re-verify on each load or restart, not only at first deployment, so swapped weights on disk are caught.

Signing does not detect a model that was poisoned before it was signed. Pair it with evaluation and the controls in the AI Data Poisoning and Backdoors Guide.

AI content watermarking

NIST AI 100-4 describes digital watermarking as "embedding information into content". The vocabulary matters when comparing claims:

• Overt watermarks: "can be perceived directly by the senses", such as a visible label or logo. Easy to understand, easy to crop out.
• Covert (invisible) watermarks: "machine-readable watermarks involving subtle perturbations of the content", found only by a detector. This is what most machine readable watermarking means.
• Robust versus fragile: robust methods "are designed to withstand certain types of attacks or modifications"; fragile ones "become invalid in the face of any changes", which suits tamper detection rather than labelling.

SynthID and LLM watermarking

Google DeepMind's SynthID embeds imperceptible watermarks in AI-generated images, audio, text and video. For images and video, Google states the watermark is "designed to stand up to modifications like cropping, adding filters, changing frame rates, or lossy compression"; these are the vendor's own claims. Google also runs a SynthID Detector portal for checking content.

SynthID Text shows how AI text watermarking and LLM watermarking work in general. It is a logits processor that adjusts token probabilities during generation using a pseudorandom function seeded by secret keys, and Google has released it as open source for developers. Detection is statistical: "Watermark detection is probabilistic", and the Bayesian detector returns watermarked, not watermarked or uncertain, with thresholds you tune for false positives and negatives. Google lists its limits plainly: it "is less effective on factual responses", confidence "can be greatly reduced when an AI-generated text is thoroughly rewritten" or translated, and it "is not designed to directly stop motivated adversaries".

Model watermarking

Model watermarking is a different idea: embedding a signal in a model's weights or behaviour so its owner can later show that a deployed model is derived from theirs. It addresses model theft and licence enforcement, not content labelling, and it remains largely a research area. Treat any vendor claim of robust model watermarking as something to test, and rely on signing and lineage records for your own models.

Watermark robustness, removal, spoofing and forgery

Defenders should plan as if watermarks will sometimes be missing on AI content and sometimes present on content that is not AI-generated. NIST AI 100-4 sets out the limits:

Watermark robustness is always relative to a threat model: survival of compression and cropping is not resistance to a deliberate attacker. NIST's answer is "a defense-in-depth approach" for high-risk uses: combine watermarks, signed metadata, fingerprint lookup and human review. Watermark detection keys are also a target: Google advises that each watermarking configuration "should be stored securely and privately", because whoever holds the keys can detect, and potentially imitate, the watermark.

Content provenance and provenance metadata

Where watermarking hides a signal in the content, provenance metadata attaches a signed history to it. The Coalition for Content Provenance and Authenticity (C2PA) specification (version 2.2, May 2025) defines the structure: assertions are statements about the asset, a claim is "A digitally signed and tamper-evident data structure that references a set of assertions", and claims and their signatures form manifests held in a manifest store "embedded into an asset or be external to its asset". Hard bindings are cryptographic hashes of the asset; soft bindings, such as an invisible watermark or fingerprint, help recover a manifest after metadata has been stripped. Ingredient assertions carry the manifests of source assets, so a composite keeps its chain, and trust rests on the signer's X.509 certificate and the C2PA Trust List.

Two limits matter. NIST notes that "Metadata on its own is neither tamper-evident nor reliably attributable", which is why C2PA signs it. And C2PA itself avoids "value judgments about whether a given set of provenance data is 'good' or 'bad'": it gives trust signals, not a verdict on truth.

The Content Authenticity Initiative (CAI) is "a cross-industry community" that promotes adoption; the C2PA provides the "end-to-end open technical standards", and the CAI notes that Content Credentials "may combine watermarking, secure metadata, and digital fingerprinting". For using content credentials against impersonation and deepfakes, see the Deepfake and AI Impersonation Guide.

EU AI Act Article 50: marking AI-generated content

• Article 50(2): providers of systems generating synthetic audio, image, video or text must ensure outputs "are marked in a machine-readable format and detectable as artificially generated".
• Article 50(4): deployers of deep fakes "shall disclose that the content has been artificially generated or manipulated."
• Dates: under Article 113 the Regulation "shall apply from 2 August 2026". The Digital Omnibus on AI added Article 111(4): providers whose systems were "placed on the market before 2 August 2026" must comply with Article 50(2) "by 2 December 2026."

Machine-readable marking in practice means a covert watermark, signed provenance metadata, or both, with a published way to check them. See the EU AI Act Compliance Guide.

Where identity fits

• Signing identities are non-human identities: the CI workload that signs models needs its own identity, least privilege and monitoring. Keyless signing removes the stored key but makes the OIDC trust configuration the control.
• Watermarking keys are secrets: store them in a secrets manager or HSM, restrict detection access, and rotate on exposure.
• Identity based watermarking: a watermark or manifest can carry an identifier for the provider, model or account that generated content. That supports attribution and abuse investigation, but it can link content to individuals, so it needs a privacy assessment and should usually identify the system or tenant rather than a person.

Implementation roadmap

1. Inventory (weeks 1 to 3): list models in production with source, digests and owners; record which generative features produce external content.
2. Sign in audit mode (weeks 3 to 6): sign every new model from the training pipeline with a workload identity; deployment logs verification failures but does not block.
3. Enforce at the gate (weeks 6 to 10): once false failures are fixed, block unsigned or unverified models in production; keep a documented break-glass override with expiry. Roll back by returning the gate to audit mode, not by removing signing.
4. Lineage (weeks 8 to 14): add parent digests and data set versions to signed attestations and the AI-BOM.
5. Output marking (in parallel): for each generative feature, choose watermarking, signed metadata or both; test detection on edited, compressed and translated samples before claiming coverage.
6. Review: re-test watermark detection whenever models or post-processing change.

Decision guidance

• Keyless or managed keys: keyless suits teams with strong CI workload identity and public models; private PKI suits regulated internal estates that need their own trust root and revocation.
• Watermark or metadata: watermarks survive some re-encoding but are probabilistic; signed metadata is exact and attributable but easily stripped. Content leaving your control benefits from both.
• Build or buy marking: if you use a hosted model, check what marking the provider already applies before adding your own; if you serve open-weight models, you are the provider and the marking duty may fall on you.
• Small versus large organisations: small teams should start with digests and a deployment check; large estates need a signed registry, admission policy and lineage across many teams.

Detection and measurement

• Log every verification result with model digest and signer identity, and alert on failures, on models loaded outside the registry, and on new signer identities.
• Monitor the signing identity like any privileged NHI: unusual signing volume or signing from a new pipeline is an incident signal.

Common failure patterns

• Signing only the weights file: tokenizer or configuration is swapped instead. Sign the whole directory.
• Verifying without pinning the signer: any valid signature is accepted, so an attacker's own signature passes.
• Treating "no watermark" as human-made: removal is routine, and some content never carried a signal.
• Trusting a detected watermark blindly: spoofing can make genuine content look AI-generated.
• Lineage lost at conversion: a quantised or converted model gets new digests with no signed link to its source.

Framework mapping

Practitioner checklist

• Record digests, source and signer for every production model.
• Sign models in the training pipeline with a dedicated workload identity, covering the whole directory.
• Verify against a pinned signer allowlist at deployment and on every load.
• Keep signed lineage linking each fine-tune, adapter and conversion to its parents and data sets.
• Choose watermarking, signed metadata or both for each generative feature, and test detection on edited samples.
• Never treat watermark absence or presence as proof; combine signals for high-risk decisions.
• Protect signing identities and watermark keys as privileged secrets.
• Confirm EU AI Act Article 50 applicability and the 2 December 2026 deadline for systems already on the market.

Standards and references

• NIST: AI 100-4 Reducing Risks Posed by Synthetic Content
• C2PA: C2PA Technical Specification 2.2
• OpenSSF: Launch of Model Signing v1.0
• Sigstore: model-transparency (model signing library and CLI)
• European Commission AI Act Service Desk: Article 50 Transparency obligations
• Google: SynthID Text
• Google DeepMind: SynthID
• Content Authenticity Initiative: How it works

This guide summarises the EU AI Act transparency obligations as they relate to provenance and watermarking and is not legal advice. Related NHI Mgmt Group resources: AI Supply Chain and AI-BOM Guide · Deepfake and AI Impersonation Guide · SLSA and Build Provenance Guide · Code Signing Security Guide · EU AI Act Compliance Guide