How should security teams implement AI risk management across training, inference, and AI agent workflows?

Why This Matters for Security Teams

AI risk management has to cover model training, inference, and agent workflows because risk changes at each stage. Training introduces data poisoning, provenance, and model lineage issues. Inference introduces prompt injection, data leakage, and unsafe outputs. Agent workflows add tool use, autonomy, and chained actions that can amplify small mistakes into operational incidents. NIST’s NIST AI Risk Management Framework is useful here because it treats AI risk as a lifecycle discipline rather than a point-in-time approval.

NHIMG’s research on agentic systems shows why this matters operationally: the OWASP Agentic Applications Top 10 highlights risks that do not stay confined to the model boundary, especially when agents can call tools, access data, and take actions on behalf of users. The practical mistake is assuming a single control set can protect every phase equally. Security teams need different guardrails for dataset governance, model evaluation, runtime authorization, and agent supervision. The same identity, logging, and policy concepts matter throughout, but the enforcement point changes.

NHIMG’s 2024 ESG Report: Managing Non-Human Identities found that 72% of organisations have experienced or suspect a breach of non-human identities, which is a strong reminder that AI systems inherit the same weak points as other machine identities when governance is incomplete. In practice, many security teams discover AI exposure only after a model, pipeline, or agent has already touched sensitive data or executed an unintended action, rather than through intentional lifecycle review.

How It Works in Practice

A workable program starts by mapping each AI lifecycle stage to a control objective. During training, teams should verify dataset provenance, restrict who can modify corpora, retain lineage metadata, and test for poisoning or memorization risks. During inference, the focus shifts to request filtering, output controls, secret redaction, abuse detection, and monitoring for prompt injection. For agents, the control model becomes stricter: every tool call, data access, and side effect should be authorized at runtime, not assumed from a one-time role assignment.

That is why policy-as-code matters. Current guidance suggests using centralized policies that can evaluate context in real time, including user identity, data sensitivity, model type, tool risk, and execution purpose. This is where CSA MAESTRO agentic AI threat modeling framework and the OWASP Top 10 for Agentic Applications 2026 are especially helpful, because they force teams to think about tool access, memory, and autonomy as first-class risks. For workload identity, many organisations are moving toward short-lived, cryptographic identity for workloads rather than static secrets, using patterns such as SPIFFE or OIDC-based federation. That lets security teams tie every model or agent action to a verifiable identity and revoke access quickly.

• Define approved training data sources and model owners before any fine-tuning begins.
• Use short-lived credentials and scoped tokens for inference and agent tool access.
• Log prompts, outputs, tool calls, and policy decisions with enough detail for audit and incident response.
• Continuously test for drift, jailbreaks, and data exfiltration across staging and production.

These controls tend to break down in highly dynamic multi-agent environments because one agent can inherit trust from another and chain actions faster than static reviews can keep up.

Common Variations and Edge Cases

Tighter lifecycle controls often increase engineering overhead, so organisations have to balance speed of deployment against assurance, especially when AI is embedded in customer-facing systems or internal automation. Best practice is evolving on how much to centralize versus delegate, particularly for agent workflows where business teams want flexibility but security teams need predictable guardrails.

One common edge case is retrieval-augmented generation: the model may be stable, but the retrieved data changes constantly, so inference risk looks more like data governance than model governance. Another is multi-agent orchestration, where one agent prepares context and another executes actions. In those setups, trust boundaries can blur quickly, and the most important control may be separating read-only reasoning from write-capable execution. NIST’s NIST AI 600-1 Generative AI Profile and the MITRE ATLAS adversarial AI threat matrix help teams distinguish model misuse from broader adversarial behaviour, while NHIMG’s Top 10 NHI Issues remains useful when agents rely on long-lived secrets or over-permissioned service identities.

There is no universal standard for this yet, but the safest pattern is to treat every AI component as a governed workload with its own identity, policy envelope, and review cadence.

Related resources from NHI Mgmt Group

• How should security teams implement an AI risk management framework across discovery, policy, and monitoring?
• How should security teams implement a third-party risk management policy across SaaS, cloud, and AI tools?
• How should security teams unify human and AI agent risk management across the workforce?
• How should security teams implement AI agent credential management?