Why do ML pipelines create hidden identity risk?

Why This Matters for Security Teams

ML pipelines are not just data workflows. They are production control paths that move code, data, models, and artifacts through multiple environments, often with automation that outlives the original experiment. That creates identity risk because service accounts, API keys, notebook credentials, and deployment roles tend to accumulate privileges as teams optimise for speed. Current guidance from the NIST Cybersecurity Framework 2.0 supports managing this as a governance and exposure problem, not only a technical one.

The practical issue is visibility. Security teams often focus on the model or the data set, while the real risk sits in who can retrain, approve, promote, or serve that model. A pipeline may span developer laptops, notebook environments, object storage, model registries, CI systems, and inference endpoints, each with a different identity model. If those identities are not mapped and reviewed as a chain, privilege creep becomes normal. In practice, many security teams encounter excessive pipeline access only after a leaked token, unintended model change, or cross-environment abuse has already occurred, rather than through intentional identity design.

How It Works in Practice

A secure ML pipeline treats every stage as an identity boundary. The notebook identity should not be the same as the training job identity, and the training job identity should not be reused for deployment unless there is a deliberate control decision. Secrets should be short-lived where possible, scoped to the minimum resource set, and rotated automatically. For shared platforms, federation and workload identity are usually preferable to embedded credentials because they reduce secret sprawl and improve revocation.

Operationally, the pipeline should enforce separate permissions for reading training data, writing artifacts, approving models, and publishing endpoints. This is where identity governance matters most: a model registry or orchestrator may appear to be a platform component, but it is also a privilege broker. If the registry can approve downstream deployment, then compromise of that identity becomes a release-path issue, not just an application issue. NIST guidance on identity and access control, alongside model governance practices, aligns with this boundary-based approach.

• Use distinct identities for human users, automation jobs, and serving components.
• Prefer ephemeral tokens and workload identity over static keys in notebooks and scripts.
• Separate read, write, approve, and deploy permissions across pipeline stages.
• Log identity context for every model action, including retraining, promotion, and rollback.
• Review inherited permissions after every platform migration or orchestration change.

Where possible, pair pipeline controls with attack-path thinking from MITRE ATT&CK and AI-specific threat modeling from MITRE ATLAS. These help teams see how valid accounts, token theft, or compromise of a build agent can become model tampering or service abuse. These controls tend to break down when teams centralise ML operations on one shared service account because the same credential then covers development, training, validation, and production promotion.

Common Variations and Edge Cases

Tighter identity control often increases operational overhead, requiring organisations to balance pipeline velocity against auditability and revocation speed. That tradeoff becomes sharper in research environments, where ad hoc experimentation is common and researchers expect rapid access to storage, compute, and notebooks. Best practice is evolving, but the direction is clear: temporary access should be time-bound, and production-grade privileges should not be inherited from experimentation accounts without review.

There are also edge cases where standard IAM patterns are not enough. Air-gapped training environments, multi-tenant platforms, and third-party managed MLOps services can limit the use of federation or fine-grained token issuance. In those settings, compensating controls such as network segmentation, stronger approval gates, and detailed artifact provenance become more important. If the organisation uses agentic AI systems inside the pipeline, the identity problem expands further because the agent may have execution authority to trigger jobs, query data, or modify prompts. That intersection is where OWASP guidance for LLM and agentic systems becomes especially relevant, although current guidance suggests controls should be adapted to the actual autonomy level rather than assumed from the tooling label.

For regulated environments, the most important question is not whether a pipeline has credentials, but whether it can prove who authorised each step and whether that access still matches the stated business need.

Related resources from NHI Mgmt Group

• Why do cloud AI platforms create hidden identity risk?
• Why do CI/CD pipelines create non-human identity risk?
• Why do hidden application identities create risk for identity-first security programmes?
• Why do identity provider migrations often create hidden governance risk?