AI security must span training, deployment, and runtime controls

At a glance

What this is: This explainer argues that AI security must cover the full model lifecycle, with the central finding that cloud controls alone do not govern training, deployment, or runtime AI risks.

Why it matters: It matters because IAM, data security, and AI governance teams need controls that follow the model from data ingestion to inference, especially where prompts, tools, and agent actions create new access paths.

Context

AI security is the discipline of protecting models, training data, inference paths, and the decisions those systems make. The problem is that many programmes still treat AI as another cloud workload, even though prompts, embeddings, model registries, tool calls, and inference logs create control gaps that standard cloud security was not designed to govern.

The lifecycle matters because compromise in training can persist into deployment, while runtime abuse can turn a safe-looking model into a data-access or action-execution channel. For identity and access teams, the key question is not only who can reach the infrastructure, but what the model can access, delegate, or leak once it is running.

Key questions

Q: How should security teams govern AI use when the same model creates different risk in different contexts?

A: Security teams should classify AI use at runtime based on identity, purpose, and data sensitivity, then apply policy that matches the specific context. The same model can be acceptable for one role and high risk for another, so static allow or block rules are too blunt. Governance works when enforcement reflects the live business situation, not just model approval.

Q: Why do AI security programmes need runtime controls instead of only pre-deployment review?

A: Because many AI risks emerge only after the system starts acting on real data, tools, and prompts. Pre-deployment review can validate design intent, but runtime controls determine whether the system stays constrained when conditions change. That is where abuse, drift, and unsafe delegation usually appear.

Q: What breaks when AI agents are given broad standing access?

A: Broad standing access breaks governance because the agent can move from one task to another without a fresh authorization check. That creates a control gap between intended scope and actual runtime behaviour. The result is weak accountability, limited containment, and audit trails that show activity without explaining why the activity was allowed.

Q: How do teams know whether AI governance is actually working?

A: Look for evidence that every AI interaction can be traced end to end, from identity and intent to output and enforcement. If auditors can ask for a transaction and receive a complete record in hours, not weeks, the programme is producing usable control evidence rather than just documentation.

Technical breakdown

Why training data integrity shapes downstream model behaviour

Training is where AI systems learn the logic that later appears in production. If data poisoning, sensitive-data leakage, or weak provenance enters the dataset, the resulting behaviour is embedded in the weights and may survive deployment controls. That is why checkpoints, feature stores, and training corpora need integrity verification and lineage, not just storage permissions. In AI security, the training set is not passive input. It is part of the control plane for model behaviour and trust.

Practical implication: Treat dataset provenance and checkpoint integrity as security controls, not ML housekeeping.

How deployment expands the AI attack surface

Deployment moves the security problem from one controlled pipeline into registries, CI/CD, inference endpoints, storage, and secrets. The article’s core point is that over-broad IAM, shared tokens, and weak segregation of duties create the conditions for artifact exposure, replacement, or uncontrolled model access. In practical terms, model artifacts behave like high-value binaries. They require version integrity, scoped access, and continuous posture management across the AI estate, including model registries and vector stores.

Practical implication: Harden registry access, artifact integrity, and secret handling as part of AI deployment governance.

Why runtime controls must inspect prompts, tools, and actions

Runtime risk is different from traditional application abuse because the system can interpret natural language, retrieve context, and call tools in a single flow. Prompt injection, jailbreaks, indirect prompt attacks, and over-privileged agents can all steer the model toward data leakage or unsafe actions. AI runtime security therefore needs policy enforcement at the prompt, context, tool, and action layers, plus telemetry that shows what the system accessed and why. Without that traceability, incidents become difficult to contain or explain.

Practical implication: Enforce deny and allow decisions at runtime for prompts, retrieval, and tool calls.

Threat narrative

Attacker objective: The attacker aims to make the AI system reveal sensitive data, execute unsafe actions, or behave in a way that undermines governance and trust.

1. Entry occurs when malicious instructions arrive through prompts, retrieved content, documents, or other indirect inputs that the model trusts during inference.
2. Credential or privilege abuse follows when the model or agent has broad access to data sources, tools, or cloud actions and can be steered into using them.
3. Impact appears as data exfiltration, unsafe tool execution, manipulated decisions, or production workflows that act outside their intended boundaries.

Breaches seen in the wild

• DPD chatbot incident 2024: A customer got DPD's AI chatbot to swear and mock its owner; DPD blamed an error after a system update and disabled the AI element.
• Chevrolet chatbot $1 Tahoe 2023: Users made a Chevrolet dealer's ChatGPT chatbot agree to sell a 2024 Tahoe for $1 and write code; no sale was honoured and the bot was taken offline.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI security now has to be treated as a lifecycle governance problem, not a point product problem. Training, deployment, and runtime each expose a different control surface, and each one fails differently. Security teams that only harden hosting infrastructure will still miss poisoned data, exposed artifacts, and unsafe live actions. The programme implication is simple: governance must follow the model from lineage to inference.

Prompt injection is the runtime expression of a broader trust-boundary failure. Once a model can consume retrieved context and tool outputs, the security question becomes which inputs are trusted to trigger action. That is where AI security intersects directly with IAM and NHI governance, because over-privileged agents and weak tool scopes turn model output into access execution. The practitioner lesson is to govern trust boundaries, not just content.

AI governance only works when policy becomes enforceable production behaviour. Documentation, approvals, and review boards do not control a model if there is no traceability from policy to prompt, tool call, and downstream action. This is the governance gap the article exposes most clearly. The field is moving toward evidence-backed control planes, and teams that cannot prove enforcement will struggle to defend their AI programme.

Model provenance is becoming a security asset class of its own. The article’s focus on checkpoints, datasets, registries, and feature stores reflects a broader shift: security must be able to prove what went into a model before it can trust what comes out. That changes procurement, development, and third-party intake decisions. Practitioners should treat provenance as a prerequisite for controlled deployment.

From our research library:

• 13% of organisations suffered breaches of their AI models, according to IBM's Cost of a Data Breach 2025 report.

What this signals

Model security debt: organisations that treat AI as an extension of cloud posture will accumulate hidden risk in datasets, registries, prompts, and agent permissions. The governance burden shifts from infrastructure inventories to lifecycle inventories, and teams that cannot trace model lineage will struggle to prove control at audit or incident time.

The practical boundary is no longer model versus application. It is whether the organisation can enforce least privilege, data minimisation, and runtime policy at the point where the model consumes context and takes action. That is the control model security teams should prepare for now.

For IAM and PAM programmes, the main implication is that access review alone is not enough. AI systems can request, combine, and use access dynamically, so issuance policy, scoped tool permissions, and traceable execution paths become the decisive controls.

For practitioners

• Map AI controls to the full model lifecycle Separate training, deployment, and runtime controls in your operating model so provenance, access, and enforcement are owned at the right stage.
• Inventory model assets and adjacent data stores Track models, endpoints, registries, feature stores, vector databases, and inference logs as first-class security assets, not hidden dependencies.
• Constrain access to training pipelines and artifacts Use least privilege, immutable audit trails, and cryptographic integrity checks for datasets, checkpoints, and model registries.
• Enforce runtime deny and allow policies Apply policy checks to prompts, retrieved context, tool calls, and high-impact actions such as exports or identity changes.
• Bind governance approvals to production controls Make model onboarding, data handling rules, and agent permissions produce enforceable settings and evidence in the live environment.

Key takeaways

• AI security fails when organisations rely on infrastructure hardening alone and leave training data, model artifacts, prompts, and tool calls outside the control model.
• The article’s central evidence is that risks differ across training, deployment, and runtime, so governance must be lifecycle-specific rather than document-only.
• Practitioners should enforce provenance, least privilege, and runtime policy where the model actually consumes data and takes action.

Key terms

• AI Security Platform: An AI security platform governs how people and agents use AI systems across prompts, responses, files, and tool calls. It goes beyond traditional content filtering by adding intent-aware policy, runtime enforcement, and audit linkage so the organisation can control both the conversation and the action that follows.
• Model Provenance: Model provenance is the evidence chain showing where an AI artefact came from, how it was modified, and whether the version in use is the one that was approved. For AI security teams, provenance is the control that turns trust from assumption into verification.
• Runtime Enforcement: Runtime enforcement is the practice of blocking malicious behaviour while software is running, rather than only detecting it after the fact. It monitors process activity, network actions, and privilege changes so a live attack can be interrupted at the point of execution.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course, the industry's only accredited NHI security programme. It helps security and identity teams build the governance foundations needed for machine identities, access control, and runtime oversight.