Model Registry Compromise Playbook

A compromised model registry lets an attacker change which model your production systems load, while every serving endpoint still asks for a name and version you approved. Model registries hold trained weights, serialised model files, version metadata, promotion aliases such as "production" or "champion", evaluation results and the credentials that training jobs, pipelines and serving platforms use to push and pull. The urgent questions are which model versions can still be trusted, whether a tampered file can run code when it is loaded, which credentials must be rotated, and how to restore known-good models without redeploying the attacker's content. This playbook is for ML platform, MLOps, security operations and incident response teams running a self-hosted or managed model registry. It is vendor-neutral and assumes the common architecture: a metadata service, an artefact store in object storage, and API tokens or service accounts for automation. Preventive design is in Securing the model registry itself and the Model and dataset registry security guide.

Key takeaways

Freeze promotion before anything else. Stop alias and stage changes, new version registration and deletions so the attacker cannot swap production models while you investigate.
Never load a suspect model to inspect it. Pickle-based formats can execute arbitrary code during loading, so examine suspect files in an isolated, network-less sandbox or with static scanners only.
Trust hashes and signatures, not version labels. A version number or alias is a pointer in a database; a signed manifest of file digests is evidence.
The registry concentrates machine credentials. Training job tokens, serving pull credentials, object storage keys, database passwords and webhook secrets all need rotation.
Rebuild from lineage, not from the registry. Re-register models from verified training runs and signed artefacts, and re-check the datasets those runs used.

Scope: what a model registry compromise covers

A model registry sits between training and serving. Compromise of any of its parts can change what a production system predicts or generates, or turn model loading into code execution on serving hosts.

Entry point	Typical cause	What the attacker can do	Mapping
Registry administrator or user account	Phished user, reused password, SSO not enforced on the registry UI or API	Register new versions, move production aliases, change permissions, delete history	T1078 Valid Accounts
Automation token or service account	Token leaked from a notebook, training script, CI variable or public repository	Push a tampered model under a trusted name; read private models	AML.T0010.003 AI Supply Chain Compromise: Model
Artefact store (object storage bucket)	Over-broad bucket policy or leaked storage key	Overwrite model files directly, outside the registry audit trail	T1565.001 Stored Data Manipulation
Metadata database	Exposed database, weak credentials	Repoint versions to attacker files, alter lineage and evaluation records	T1565.001
Registry server or host	Unpatched service, exposed admin endpoint	Everything above, plus credential theft from configuration	T1552.001 Credentials In Files
Upstream model import	Importing public base models without verification	Introduce a backdoored or code-carrying model that is then fine-tuned and promoted	AML.T0058 Publish Poisoned Models, AML.T0018.000 Poison AI Model

If the tampering began in training data or the training pipeline, run the CI/CD Pipeline Compromise Playbook and the AI agent supply chain incidents guide alongside this one. A poisoned training run registers a bad model with valid credentials, so it looks like normal publication.

How the compromise shows up

Alias or stage moved without an approval: the production alias points at a version no approved promotion request produced.
Versions with no lineage: a registered version has no linked training run, dataset reference or evaluation report, or its run identifier does not exist in the experiment tracker.
File digests that changed: the hash of a stored model file no longer matches the digest recorded at registration or in a signature manifest.
Unexpected identities: registrations from human accounts into production namespaces, or automation tokens used from outside training and CI network ranges.
Behaviour drift in serving: sudden changes in prediction distributions, refusal rates or output content, picked up by model monitoring. See the ML model evaluation and monitoring guide.
Host-level signals on serving nodes: new outbound connections, child processes or file writes at model load time, which point to code embedded in a serialised model (T1059.006 Python).
External signals: a leaked token report, a provider advisory or a researcher disclosure. The Hugging Face API token exposure of 2023 showed how leaked hub tokens can open organisations' model repositories to anyone who finds them.

Triage: the first hour

Freeze writes: remove register, update, delete and alias permissions from every identity except named responders. If the registry has no read-only switch, revoke automation tokens and restrict the artefact store bucket policy to read-only.
Snapshot the state: export the audit log, every registered model with its versions, aliases, tags, source run identifiers, artefact URIs and timestamps, plus users, roles and tokens. Hash the exports.
Preserve storage: enable or confirm object versioning and take a snapshot of the artefact store and the metadata database, so files changed outside the API can be found later.
Pause automatic deployment: stop serving platforms and pipelines that redeploy on alias change. Pin each production endpoint to the exact model file digest it is running now, or to the last digest you can verify.
Stop loading unverified models: block new loads of affected models in serving, batch scoring and notebooks until integrity is established.

Proving which models you can still trust

Model files are binary and, as the OWASP Top 10 for LLM Applications 2025 notes under LLM03 Supply Chain, static inspection offers little assurance about hidden behaviour. Trust has to come from evidence created outside the attacker's reach.

Evidence you hold	How to use it	Confidence
Model signatures from an identity the attacker did not control	Verify each version's signature bundle, then recompute file hashes and compare with the signed list	High, if signing keys or identities sat outside the registry and pipeline
Digests recorded in deployment manifests in version control	Compare the stored file digest with the digest that was reviewed and deployed	Medium to high, depends on repository integrity
Training run records with output hashes	Match registered files to runs and their recorded outputs	Medium; tracker records may be in the attacker's reach
Evaluation reports	Re-run the evaluation suite on the stored file and compare with the recorded scores	Medium; a targeted backdoor can pass general evaluations
Version labels and timestamps only	Treat as unverified	Low

Signature format: the OpenSSF model signing project produces a Sigstore bundle containing an in-toto statement whose subjects are file path and digest pairs, so verification checks the signer identity and then recomputes every file hash. See the Model Signing and OpenSSF Model Signing Guide.
Serialisation risk: the Python documentation warns that malicious pickle data can execute arbitrary code during unpickling and that data which could have been tampered with should never be unpickled. Inspect suspect pickle-based files only in a disposable sandbox without credentials or network, and prefer tensor-only formats such as safetensors for rebuilt models.
Backdoors without code: weights can be altered to embed a trigger without any executable payload (AML.T0018.000 Poison AI Model). Hash comparison catches this when you have a trusted digest; behavioural tests alone may not.
Signing keys in the blast radius: if keys were held in the registry, in pipeline variables an attacker could read, or used by a compromised service account, signatures from the exposure window stop being evidence.

Containment by priority

Remove attacker access: disable unknown accounts, revoke every token created or used in the suspect window, and cut administrators to named responders.
Repoint production to verified versions: roll serving endpoints back to the last verified file digest, with two-person approval for each alias change.
Quarantine suspect versions: move them to a restricted namespace or bucket rather than deleting them, so evidence survives and nothing can load them.
Isolate serving hosts that loaded suspect files: treat them as potentially compromised hosts, capture memory and disk where practical, and rebuild them.
Close side doors: disable webhooks, replication to other registries and upstream import jobs that were added or changed.
Protect the logs: confirm registry, storage and identity provider logs stream to a store that registry administrators cannot alter.

Credential rotation inventory

Credential	Where it lives	Rotation action	Better pattern afterwards
Training and pipeline push tokens	CI variables, training job configuration, notebooks	Revoke all; issue per-pipeline tokens scoped to one model namespace	Workload identity federation from the pipeline, no long-lived token
Serving pull credentials	Serving platform configuration, Kubernetes secrets	Revoke and redeploy endpoint by endpoint	Read-only workload identity per serving environment
Data scientist personal tokens	Workstations, notebook environments, shell history	Revoke all; reissue with expiry	SSO-backed short-lived tokens
Admin accounts and API keys	Registry, identity provider	Reset, re-enrol phishing-resistant MFA, revoke sessions	SSO only; break-glass account held in a vault
Artefact store keys and bucket policies	Registry server configuration	Rotate keys; review and tighten bucket policy; review storage access logs	Registry-only identity; no human write access to the bucket
Metadata database credentials	Registry configuration, environment variables	Rotate; review database audit logs	Secrets manager injection at runtime
Model signing keys (if exposed)	KMS, HSM, CI	Rotate, re-sign verified models, update verification policy	Keyless signing or KMS-held keys outside the registry
Secrets inside model packages	Bundled configuration, inference code, environment files	Rotate every secret found in any package an attacker could read	No secrets in model artefacts

Read access is enough to cause harm: private models are intellectual property, and packages often carry API keys for downstream services. The xAI API key leak of 2025 shows how one exposed key can open private models. See the If your secrets manager is compromised playbook if registry credentials were held there, and the Workload Identity Federation Guide for the replacement pattern.

Eradication and rebuild

Rebuild or patch the registry service: for a self-hosted registry, rebuild hosts from a trusted image on a supported release and restore configuration from a reviewed export. For a managed registry, ask the provider whether their side was involved and what logs they hold.
Re-register models from verified lineage: take model files from signed artefacts or re-run training from verified code and datasets on a trusted environment, sign the output and register it in a fresh or reviewed namespace.
Check the inputs: if the attacker could write to training datasets or feature stores, verify dataset hashes and versions before retraining. OWASP LLM04 Data and Model Poisoning covers the integrity risks at each training stage.
Re-check imported base models: re-verify every third-party model imported during the window against its publisher's digest or signature, and replace pickle-based imports with tensor-only formats where available.
Re-baseline policy: restore approval gates on production aliases, enable object versioning and deletion protection on the artefact store, and require signature verification at serving load time.
Hunt for follow-on activity: review what serving and batch hosts did after loading suspect versions: outbound connections, cloud API calls with their workload identity and new credentials. See the Kubernetes Cluster Compromise Playbook where serving runs on clusters.

Recovery, testing and rollback

Staged unfreeze: re-enable registration for one pipeline at a time with its new federated identity, confirm it signs output and that serving verifies the signature, then widen.
Negative tests: try to load an unsigned model and a model whose file hash does not match its signature; both must be rejected by the serving platform.
Behavioural regression: run the full evaluation suite, including any trigger and safety tests you hold, on every restored production version before reopening traffic.
Rollback point: keep the list of verified digests per production model as the known-good set, so a bad recovery step can be reverted by pinning to it.

Small and large organisations

Decision	Small team, one registry	Large estate, many teams and environments
Scope of re-verification	Re-verify or rebuild every production model	Risk tier first: customer-facing, high-impact decision and agent-facing models before internal ones
Serving credential rotation	Rotate all at once	Rotate per environment with a tracking list, moving to federation as you go
Downstream notice	Tell internal users which versions to discard	Publish an internal advisory listing affected versions, digests and verified replacements

Evidence and notification

Preserve: registry and storage audit logs, version and alias exports before and after containment, quarantined model files, database and bucket snapshots, serving host images for any host that loaded a suspect file, and identity provider sign-ins.
Integrity of evidence: hash exports and keep a chain of custody, in line with NIST CSF 2.0 RS.AN-07 (incident data and metadata are collected, and their integrity and provenance are preserved).
Notification: if a tampered model made or influenced decisions about people, or if personal data in training sets or packages was exposed, privacy and sector rules may apply. Legal duties vary by jurisdiction; see the Cyber Incident Reporting Requirements Map. Coordinate with affected customers and providers as NIST SP 800-61 Rev. 3 describes (CSF 2.0 RS.MA-01).

Detection and measurement

Share of production model versions whose file digests verify against a signature from an approved identity (target 100%).
Production alias changes without a linked approval record (target zero; each one investigated).
Long-lived registry tokens remaining after migration to federation (target zero).
Registered versions with no linked training run or dataset reference (target zero).
Pickle-based model files loaded in production serving (NHI Mgmt Group recommendation: target zero, with documented exceptions).
Revoked credentials seen in use after rotation (target zero).

Framework mapping

Area	NIST SP 800-53 Rev. 5	NIST CSF 2.0	ATT&CK / ATLAS addressed
Model integrity and signatures	SI-7	RS.AN-03	AML.T0010.003, AML.T0018.000
Provenance of models and data	SR-4	RC.RP-02	AML.T0058
Registry credentials and service accounts	IA-5, AC-6, AC-2	RS.MI-02	T1078, T1552.001
Artefact store and metadata tampering	SI-7, SI-4	RS.MI-01	T1565.001
Audit log protection	AU-9	RS.AN-07	T1685 Disable or Modify Tools
Incident handling and recovery	IR-4, CP-10	RS.MA-01	AML.T0011.000 Unsafe AI Artifacts

NIST AI 100-2 E2025 provides the shared taxonomy of adversarial machine learning attacks, including poisoning, that analysts can use when describing what was done to a model.

Common failure patterns

Loading the suspect model to "see what it does": on a normal workstation or serving node this can run the attacker's code with your credentials.
Trusting the version history: the attacker who could write models could also edit metadata, lineage and evaluation records.
Deleting suspect versions first: evidence is lost and you cannot show which predictions used which model.
Rotating registry tokens but not storage keys: files in the bucket stay writable outside the registry's audit trail.
Restoring from the same poisoned base model: retraining on a backdoored upstream model or poisoned dataset brings the problem straight back.
Forgetting offline consumers: batch scoring jobs, edge deployments and notebook caches keep using a copied model long after the registry is clean.

Practitioner checklist

Freeze registration, alias changes and deletions, and pause automatic deployment.
Export audit logs, version and alias maps, accounts and tokens, and snapshot storage and the database before changing anything.
Pin production endpoints to verified file digests and block loading of unverified models.
Inspect suspect files only in an isolated sandbox with no credentials or network.
Verify every production version against signatures from identities outside the blast radius.
Revoke and reissue push tokens, serving credentials, admin keys, storage keys and database passwords.
Rebuild models from verified lineage, re-check datasets and imported base models, and sign the output.
Require signature verification at load time and track the metrics above.

If your model registry is compromised: containment, credential rotation and rebuild playbook