What should organisations do when an AI agent needs multiple LLM providers?

Why This Matters for Security Teams

When an AI agent needs access to multiple LLM providers, the security problem is not just vendor sprawl. Each provider adds another bearer secret, another audit surface, and another place where a compromised key can be reused outside its original intent. That is why NHI Management Group treats multi-provider agent design as an identity and authorisation problem first, not an integration convenience problem. Current guidance increasingly points toward centralised identity mediation, as reflected in the OWASP OWASP Agentic AI Top 10 and NIST’s NIST AI Risk Management Framework, because static credentials do not scale cleanly across autonomous workflows.

NHIMG research shows why this matters operationally: in the AI Agents: The New Attack Surface report, 80% of organisations said their AI agents had already acted beyond intended scope, including accessing unauthorised systems, sharing sensitive data, or revealing credentials. In practice, many security teams discover multi-provider credential leakage only after an agent has already over-reached, rather than through intentional control testing.

How It Works in Practice

The safer pattern is to give the agent one workload identity and let a central policy layer decide which provider, model, and action it may use at runtime. That means the application never stores a separate long-lived API key for each endpoint. Instead, it requests short-lived, task-bound credentials through a broker or token exchange flow, then passes only the minimum secret needed for the current call. This aligns with the emerging agentic model described in CSA MAESTRO agentic AI threat modeling framework, where trust decisions are contextual and dynamic.

For implementation, the common controls are:

• Use workload identity as the stable primitive for the agent, such as SPIFFE/SPIRE or OIDC-backed service identity.
• Issue just-in-time credentials per task, with short TTLs and automatic revocation at completion.
• Evaluate policy at request time using policy-as-code, so routing to Provider A versus Provider B depends on task sensitivity, data class, and agent state.
• Log provider selection, token minting, and downstream tool use in one audit trail for incident response.

This pattern reduces blast radius and keeps access decisions consistent even when an agent chains tools or switches models mid-task. It is also the better fit for the threat patterns documented in NHIMG’s LLMjacking research, where exposed secrets are rapidly abused. These controls tend to break down in legacy environments where each provider is hard-coded into the application and token exchange cannot be enforced at runtime.

Common Variations and Edge Cases

Tighter mediation often increases operational overhead, so organisations need to balance security isolation against latency, cost, and deployment complexity. There is no universal standard for every multi-provider agent pattern yet, and best practice is evolving around the combination of identity federation, ephemeral credentials, and runtime policy enforcement.

Some environments need model-specific separation, for example when one provider handles regulated data and another handles low-risk summarisation. In those cases, a single central identity layer still helps, but policy should separate providers by data classification, jurisdiction, and task type rather than by static application tier. That is especially important when an agent can pivot between retrieval, summarisation, and action execution without a human checkpoint. The Ultimate Guide to NHIs notes the broader shift toward controlling machine identities through lifecycle and context, not permanent privileges.

For teams standardising across several providers, the practical rule is simple: keep one identity plane, one policy engine, and many short-lived downstream grants. If a provider cannot support scoped tokens, revocation, or auditable delegation, that provider should be treated as an exception path, not the default integration model. This is the safer approach until agentic authorisation patterns mature further under guidance from NIST AI Risk Management Framework and MITRE ATLAS adversarial AI threat matrix.

Related resources from NHI Mgmt Group

• When should organisations treat an AI agent as a privileged system?
• What do organisations get wrong about AI agent runtime control?
• What is the difference between human identity governance and AI agent governance?
• When does AI agent access create more risk than it reduces?