Why do AI pilots often stall when teams try to scale them?

Why This Matters for Security Teams

AI pilots do not usually fail because the model is unusable. They stall when the organisation tries to turn an experimental workflow into a governed service, but the surrounding controls were never designed to travel with the use case. The result is duplicated review, unclear ownership, and inconsistent access decisions that make every new deployment feel like a fresh exception rather than a repeatable pattern.

That matters because scaling AI increases the number of identities, secrets, approvals, and data paths that must be managed at once. If governance only appears after the pilot proves value, teams discover that the hardest part is not inference quality but operational trust. The NIST Cybersecurity Framework 2.0 is useful here because it emphasises outcomes such as governance, risk management, and repeatable control execution rather than one-time signoff.

NHIMG research on the Ultimate Guide to NHIs shows why this pattern is so brittle: once non-human access becomes fragmented, scaling stops being a technical issue and becomes an operational one. In practice, many security teams encounter their first serious AI control failure only after a pilot has already been approved for production, rather than through intentional governance design.

How It Works in Practice

Scalable AI governance starts by treating the pilot as a production candidate from day one, even if the model itself remains in test. That means defining the owner, the data boundary, the intended tools, the secrets model, and the review path before the use case is allowed to expand. For AI systems with autonomous or agent-like behaviour, the control problem is more than app approval: it includes workload identity, runtime authorisation, and fast revocation when the task ends.

Current guidance suggests using repeatable control patterns instead of bespoke reviews. A practical stack usually includes:

• Workload identity for the agent or service, so access is tied to what the workload is, not who last touched it.
• Just-in-time credential issuance with short TTLs, so secrets are created per task and revoked when the workflow completes.
• Policy checks at runtime, not just at deployment, so high-risk actions can be blocked based on context.
• Central logging of prompts, tool calls, and access events, so review does not depend on manual reconstruction after the fact.

For agentic systems, this is where the distinction between static IAM and intent-aware access becomes important. A pre-approved role is often too coarse for an AI agent that can chain tools or change tactics mid-task. Emerging practices such as policy-as-code and runtime authorisation align better with the operational reality described in the LLMjacking research, where compromised non-human identities are abused rapidly once exposed. Security teams also need to assume that “approved” does not mean “safe indefinitely”; the control must still be re-evaluated when the task, dataset, or toolchain changes. These controls tend to break down when multiple teams manage the same AI service through separate IAM, MLOps, and application approval paths because no single owner can enforce the full lifecycle.

Common Variations and Edge Cases

Tighter governance often increases delivery overhead, requiring organisations to balance speed against review depth. That tradeoff becomes visible in low-risk internal pilots, where a heavy approval process can slow learning, and in regulated or customer-facing workflows, where lighter controls create downstream exposure. Best practice is evolving, but there is no universal standard for how much governance is “enough” at pilot stage.

One common edge case is the team that already has strong application security but weak non-human identity discipline. In those environments, the pilot appears controlled until the first shared API key, embedded token, or connector credential is reused across multiple experiments. Another edge case is the multi-team platform model, where data science, product, and security each believe someone else owns the control boundary. That is where pilots stall: not because the model cannot scale, but because the organisation cannot answer who can approve, who can revoke, and who can audit.

NHIMG analysis of the DeepSeek breach shows how quickly exposed credentials and unsafe data handling can turn into a broad operational problem, not just a one-off incident. The lesson for scaling AI is straightforward: if the governance model cannot survive team handoffs, secret rotation, and new tool integrations, the pilot will remain a pilot no matter how promising the model is.

Related resources from NHI Mgmt Group

• Why do governed AI gateways often cost more than teams expect at scale?
• Why do identity and access programmes often stall when teams treat them as isolated tooling projects?
• Where do local scanner programmes fail in practice when teams try to scale them across regulated environments?
• Why do DevSecOps programmes often stall when teams try to move from policy to practice?