Join our Newsletter — 33% off our NHI Course
Home› Guides› AI Training and Fine-Tuning Pipeline Security: Controls from…
Guide AI Platforms

AI Training and Fine-Tuning Pipeline Security: Controls from Data to Deployed Model

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 15 November 2026 16 min read
On this page

A training or fine-tuning pipeline turns data, code and compute into a model that will later run in production, so anyone who can influence any stage of that pipeline can change what the model does without touching the serving system. Most organisations secure the model registry and the inference endpoint, then leave the stages in between (data snapshots, training images, distributed jobs, checkpoints, evaluation and promotion) with build-server levels of trust and none of the build-server controls. This guide treats the pipeline as a build system: it defines a gate and an evidence record for each stage, from data intake to the deployed artefact, and maps those gates to NIST SP 800-218A, SLSA and MITRE ATLAS. It is written for ML platform engineers, security architects and assurance teams who own or approve pre-training, continued training or fine-tuning pipelines. For the broader lifecycle overview, see the MLOps Security Guide; for method-level detail on adapters and safety regression, see the Fine-Tuning and LoRA Security Guide.

Key takeaways

  • Every stage needs a gate and a record. NIST SP 800-218A asks producers to document the entire training process, track provenance of models, derivatives, libraries and pipelines, and generate attestations of training dataset integrity; a pipeline without per-stage evidence cannot prove what it built.
  • Distributed training is a trusted-network protocol. PyTorch states that its distributed primitives (c10d, RPC, TCPStore) have no authorisation and send messages unencrypted, so anyone on the training network can execute code as the training user.
  • Checkpoints are executable state. PyTorch says loading a distributed checkpoint may execute arbitrary code and that the weights_only protection does not apply, so only checkpoints your own infrastructure produced and stored should ever be resumed.
  • The training job is a privileged non-human identity. It reads the most sensitive data and writes the artefact production will run; give it one scoped, short-lived identity and keep humans away from weights by default, as SP 800-218A suggests.
  • Promotion is the control that matters most. Evaluate the exact artefact, verify its signature and provenance, and require an approver who did not run the training job before anything reaches the registry's production stage.

The pipeline as a build system

A model pipeline has the same shape as a software build: inputs (datasets, base models, code, container images), a build step (training) and an output artefact (weights or an adapter). The SLSA Build track gives useful vocabulary. At Build L1 provenance exists but is "trivial to bypass or forge"; at L2 a hosted platform generates and signs provenance; at L3 the platform isolates runs so they cannot influence one another and keeps signing secrets out of reach of user-defined steps. Applied to training, that means the orchestration platform, not the training script, should record which dataset version, base model digest, image digest and code commit produced each checkpoint, and sign that record. See the SLSA and Build Provenance Guide and the in-toto Attestations Guide.

The difference from ordinary software is that the output cannot be reviewed line by line. A poisoned dataset or a tampered checkpoint produces weights that look like any other weights. Integrity therefore has to be proven from the inputs and the process, and behaviour has to be tested afterwards.

Stage-by-stage gates and evidence

The table is the core of this guide: one gate per stage, the evidence that gate should produce, and the main threat it addresses. ATLAS IDs were checked against ATLAS 2026.10.

StageGate before the next stageEvidence to keepMain threat (MITRE ATLAS / ATT&CK)
1. Data intakeSource on an allowlist; licence and lawful basis recorded; secret and PII scan passedSource register, scan reports, dataset hashAML.T0010.002 Data Repository; AML.T0020 Training Data Poisoning
2. Data preparationFiltering, deduplication and anomaly checks run on an immutable snapshot; changes reviewedTransformation log, snapshot digest, reviewerAML.T0020 Training Data Poisoning
3. Base model and dependency intakeBase model verified by hash or signature; safe format; packages pinned by hash from a private mirrorBase model digest, AI-BOM entry, lock fileAML.T0010.003 Model Repository; AML.T0010.001 Software; T1195.002
4. Training environmentImage built in CI and pinned by digest; job network isolated; one scoped job identityImage digest and SBOM, network policy, role definitionAML.T0012 Valid Accounts; T1190 Exploit Public-Facing Application
5. Training runRun launched only by the orchestrator from approved inputs; checkpoints written to a store only the job can writeSigned run provenance (inputs, code commit, hyperparameters, image)AML.T0018.000 Poison AI Model
6. Checkpoint handlingCheckpoints hashed on write; resume only from the job's own store; old checkpoints expiredCheckpoint manifest with hashes and retentionAML.T0018 Manipulate AI Model; AML.T0035 AI Artifact Collection
7. EvaluationHeld-out evaluation set access-controlled and versioned; safety, task and regression tests on the exact artefactEvaluation report bound to the artefact digestAML.T0018 Manipulate AI Model
8. Signing and promotionArtefact signed; provenance verified; separate approver; registry version immutableSignature bundle, approval recordAML.T0010.003 Model Repository
9. DeploymentServing verifies signature and loads only registered versions; rollback criteria definedDeployment log, verification resultAML.T0011 User Execution

What NIST SP 800-218A asks for at each stage

NIST SP 800-218A, final since July 2024, is the SSDF Community Profile for generative AI and dual-use foundation models. It adds AI-specific recommendations to SSDF tasks. The ones that bear directly on the pipeline are summarised below; read the publication for exact wording, and see the NIST SP 800-218A SSDF Community Profile for Generative AI Guide for the full profile.

SP 800-218A itemPipeline stageWhat it asks for (paraphrased)
PW.3.1 (R1, R2)Data intake and preparationVerify provenance and integrity of training, fine-tuning and alignment data before use; apply anomaly detection, filtering and sanitisation
PW.3.2Data intakeTrack provenance of all training, testing, fine-tuning and aligning data and document data whose provenance is unknown
PW.4.4 (R1, R2)Base model and dependency intakeVerify and scan acquired models, datasets, reward models, adaptation layers and configurations before use
PO.5.1 (R2, R3, R4, R6)Training environmentKeep sensitive data in approved environments, apply least privilege to pipelines and registries, monitor training activity and registry changes, and monitor for plaintext secrets
PS.1.1 and PS.1.3Training run and checkpointsKeep models, weights, pipelines and reward models in secure storage; minimise direct access; store reward models separately; keep weights apart from training data; add encryption, hashes, signatures, multiparty authorisation or air-gapping by risk
PS.3.1 and PS.3.2Whole pipelineVersion the tools used for dataset creation and training, document the entire training process, and track provenance of models, derivatives, libraries and pipelines
PW.8.2 (R1, R2)EvaluationTest models for vulnerabilities and retest when retrained or when new data sources are added
PS.2.1 (R1, R2)Signing and promotionProvide hashes or signatures for the model and its components, and sign model changes
RV.2.2 (R2)DeploymentDefine criteria for stopping use of a model or rolling back to a previous version

Securing the training environment

Distributed training traffic

PyTorch's security policy is direct: PyTorch Distributed features "are intended for internal communication only", none of the primitives include an authorisation protocol, messages are sent unencrypted, and anybody with network access can execute arbitrary code with the privileges of the user running PyTorch. Treat the training fabric as a closed segment. Put workers in a dedicated subnet or namespace with network policy that allows only peer-to-peer traffic between job members, keep rendezvous endpoints off shared networks, and never expose them across tenants. The same reasoning applies to cluster control planes: the ShadowRay 2024 campaign used internet-reachable Ray job interfaces to run code on AI clusters and take cloud credentials, AI tokens and compute. See the Ray and AI Compute Cluster Security Guide and GPU and AI Cluster Multi Tenancy Security.

Managed training network isolation

The major cloud ML platforms each document a way to stop training jobs reaching the internet or data outside an approved boundary. Platforms are listed alphabetically; each is a customer-side setting.

PlatformSettingWhat the documentation says it doesOperational consequence
Amazon SageMaker AIEnableNetworkIsolation on CreateTrainingJob (optionally with VpcConfig)Containers cannot make outbound calls to any service, including Amazon S3, and no AWS credentials are made available to the container runtime; SageMaker AI moves input and output data outside the container using the execution role; multi-instance traffic is limited to training peersTraining code reads and writes local paths; dependencies must be in the image
Google Cloud Gemini Enterprise Agent PlatformVPC Service Controls perimeterTraining data for AutoML or custom models stays inside the perimeter, and default internet access from the Google-managed environment is blockedPublic PyPI installs are not possible; use custom containers or install packages from Artifact Registry
Microsoft Azure Machine LearningWorkspace managed virtual network, isolation mode allow_only_approved_outboundOutbound traffic from compute instances, clusters, serverless compute and managed online endpoints is blocked except approved service tags, FQDNs and private endpoints, with data exfiltration protection enabled automaticallyMicrosoft notes that added outbound rules, especially FQDN rules, are not guaranteed exfiltration protection; once set, the isolation mode cannot be switched or disabled

As of November 2026. For platform-specific depth see the Amazon SageMaker Security Guide, the Google Vertex AI Security Guide and the Azure Machine Learning Security Guide.

Images, dependencies and secrets

  • Build images in CI, not on the cluster: training images should be built from a reviewed Dockerfile, scanned, signed and referenced by digest in the job spec. A tag such as :latest lets the image change between runs without a record.
  • Pin Python dependencies by hash: the PyTorch torchtriton attack showed how a public package with the same name as an internal dependency was preferred by pip and stole SSH keys and environment variables from nightly users. Use a private mirror and hash-pinned lock files.
  • No secrets in job specs or notebooks: SP 800-218A PO.5.1 calls for monitoring environments for plaintext secrets. Inject short-lived credentials at runtime through workload identity instead.

Data, checkpoints and the training run

Data controls (source allowlists, provenance, poisoning defences) are covered in depth in the CISA AI Data Security Guidance Guide and the AI Data Poisoning and Backdoors Guide. The pipeline-specific points are these.

  • Train from snapshots, not live locations: point jobs at an immutable, versioned dataset snapshot whose digest is recorded in run provenance. A job that reads a mutable bucket prefix can train on data that changed after review.
  • Scan for secrets before training: the 12,000 Live Secrets in LLM Training Data 2025 case found 11,908 working credentials in a Common Crawl snapshot used for model training. Revoke anything found; removing it from the dataset does not undo the exposure.
  • Protect checkpoints like releases: PyTorch states that distributed checkpoints are meant to be restored only from storage you control, that loading one "may execute arbitrary code", and that the weights_only protection of torch.load does not apply. Write checkpoints to a location only the job identity can write, hash them on write, and never resume from a checkpoint copied in from elsewhere.
  • Keep reward models and evaluation sets separate: SP 800-218A recommends storing reward models separately from models and data. Apply the same to held-out evaluation sets: if the training identity can read or write them, results can be contaminated or gamed.
  • Expire what you no longer need: intermediate checkpoints are full copies of model weights. Set retention so they do not become an unmanaged store of valuable artefacts. See Securing AI Model Weights.

Where identity fits

A training pipeline typically holds several non-human identities with unusual reach: the orchestrator's service account, the training job's cloud role, the hub token used to pull base models, the experiment tracker API key and the registry credential used to publish. Each should be distinct, scoped to its stage and short-lived.

IdentityShould be able toShould not be able to
Data ingestion pipelineWrite new raw data to a landing zoneApprove snapshots or read training outputs
Training job roleRead one approved snapshot, write checkpoints and outputs to stagingWrite training data, overwrite registry versions, reach the internet
Model hub or registry pull tokenRead allowlisted base modelsPublish or delete models
Promotion identitySign and move a verified artefact to production stageLaunch training jobs
Human engineersLaunch jobs through the orchestrator, read metricsDownload production weights directly (SP 800-218A PS.1.1 suggests considering blocking human access to weights)

Use cloud-native workload identity or federation rather than static keys in job definitions. See the Workload Identity Federation Guide and the CI/CD Pipeline Identity Security Guide.

How to implement

  1. Inventory pipelines and identities (weeks 1 to 2): list every training and fine-tuning pipeline, where it runs, its data sources, base models, images and the identities it uses. Record them in the AI-BOM. See the AI Supply Chain and AI-BOM Guide.
  2. Close the network (weeks 2 to 4): turn on managed network isolation or equivalent policy for one pilot pipeline, move dependencies into images or a private mirror, and confirm the job still completes. Roll back by reverting the job definition, not by opening the network permanently.
  3. Split identities (weeks 3 to 6): replace shared credentials with one role per stage, remove standing human access to weights, and log every assumption of the training role.
  4. Add provenance (weeks 4 to 8): have the orchestrator record dataset digest, base model digest, image digest, code commit and hyperparameters for every run and sign the record. Start in record-only mode, then make missing provenance a promotion blocker.
  5. Gate promotion (weeks 6 to 10): require a signed artefact, verified provenance, an evaluation report bound to the artefact digest and an approver who did not run the job. Serving verifies the signature before load. See the Model Signing and OpenSSF Model Signing Guide.
  6. Extend and rehearse: roll the pattern to remaining pipelines, then rehearse a tampered-checkpoint or poisoned-snapshot scenario. The NIST, NSA and CISA Key Practices for Incident Readiness in AI Training and Evaluation Guide covers response preparation.

Decision guidance

  • Managed fine-tuning API versus self-managed training: a provider-hosted fine-tuning service removes the cluster, network and checkpoint stages from your scope but leaves data handling, job permissions and who can deploy or call the custom model. Self-managed training gives full control and full responsibility for every gate above.
  • Small teams: start with the three highest-value gates: immutable dataset snapshots, network-isolated jobs with one scoped role, and signed promotion. Manual approval is acceptable at low volume.
  • Large or frontier-scale programmes: automate provenance at SLSA Build L3-style isolation, separate training and promotion duties formally, and apply SP 800-218A's multiparty authorisation and air-gapping options to the most valuable weights.
  • Continuous or online training: treat each data refresh as a new input that triggers re-evaluation, as SP 800-218A PW.8.2 expects when new data sources are added.

Detection and measurement

Log job submissions (who, which inputs, which image), training role assumptions, reads of evaluation sets, writes to checkpoint stores, registry stage transitions and signature verification failures at serving. Alert on jobs launched outside the orchestrator, checkpoint writes from identities other than the job, and promotions without a matching provenance record.

  • Provenance coverage: share of production models with a signed run record linking dataset, base model, image and code digests.
  • Isolated-job ratio: share of training jobs running with network isolation or an approved-outbound boundary.
  • Standing access to weights: number of human identities able to read production weights directly (target near zero).
  • Unverified loads: serving loads that skipped or failed signature verification (target zero).
  • Re-evaluation latency: time between a data source change and a completed evaluation of the retrained model.

Framework mapping

Control objectiveNIST SP 800-53 Rev. 5NIST CSF 2.0SP 800-218A
Record data, model and pipeline provenanceSR-4 Provenance, CM-8ID.AM-08PW.3.2, PS.3.2
Isolate training networksSC-7 Boundary ProtectionPR.IR-01PO.5.1
Least privilege for job and promotion identitiesAC-6 Least PrivilegePR.AA-05PO.5.1, PS.1.1
Integrity of checkpoints and released modelsSI-7, SI-7(6), SC-28PR.DS-01PS.1.3, PS.2.1
Baseline training images and configurationCM-2 Baseline ConfigurationPR.PS-01PS.3.1
Evaluate before promotion and after retrainingSA-11 Developer Testing and EvaluationPR.PS-06PW.8.2
Monitor training activity and registry changesAU-12, SI-4DE.CM-09PO.5.1, PO.5.3

Common failure patterns

  • Open training fabric: distributed workers or cluster dashboards reachable from shared or public networks, despite PyTorch and Ray both assuming a trusted network, as the ShadowRay campaign showed.
  • Resuming from a borrowed checkpoint: a checkpoint copied from a shared drive or another team's bucket was resumed, although distributed checkpoint loading can execute code.
  • Training on a moving target: jobs read a live bucket prefix, so the data trained on was not the data reviewed, and no digest exists to prove otherwise.
  • One role for everything: the same service account ingests data, trains, writes to the registry and promotes, so one leaked key gives control of the whole chain.
  • Evaluation set inside the blast radius: the training identity can write the held-out set, so evaluation results cannot be trusted.
  • Promotion by tag: a registry stage was changed by name without signature or provenance checks, so a swapped artefact would pass.

Practitioner checklist

  • Define a gate and an evidence record for each pipeline stage, from data intake to deployment.
  • Train only from immutable, versioned dataset snapshots whose digests are recorded in run provenance.
  • Scan training data for secrets and personal data, and revoke any credential found.
  • Run training jobs network-isolated, with dependencies baked into digest-pinned images or a private mirror.
  • Keep distributed training traffic and rendezvous endpoints on a closed segment limited to job peers.
  • Write checkpoints to a store only the job can write, hash them, expire old ones, and never resume from external checkpoints.
  • Give each stage its own short-lived non-human identity and remove standing human access to production weights.
  • Bind evaluation reports to the artefact digest and keep evaluation sets out of the training identity's reach.
  • Promote only signed artefacts with verified provenance, approved by someone who did not run the job.
  • Define rollback criteria and rehearse a tampered checkpoint or poisoned snapshot scenario.

Standards and references

Related NHI Mgmt Group resources: MLOps Security Guide · Fine-Tuning and LoRA Security Guide · If your model registry is compromised · LLM Quantisation Explained

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 15 November 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.