What is the difference between attached provider keys and AI gateway-managed provider keys?

Why This Matters for Security Teams

Attached provider keys and ai gateway-managed provider keys look similar at the edge, but they create very different governance and breach consequences. With attached keys, the customer retains control of the upstream provider account and its secrets, which means rotation, revocation, and billing ownership stay tied to the customer. With gateway-managed keys, the operator becomes the trust and control point, so the customer must evaluate the operator’s isolation, logging, and tenant separation instead of protecting a provider secret directly.

That distinction matters because secrets exposure is still one of the fastest paths to abuse. NHIMG research on the State of Secrets in AppSec shows organisations are managing an average of 6 distinct secrets manager instances, a sign of fragmentation that weakens central control. When provider keys are attached, that fragmentation can extend into AI usage, billing, and incident response. Security teams should also align the model with NIST Cybersecurity Framework 2.0 expectations for asset visibility, access control, and recovery.

In practice, many security teams discover the real ownership model only after an overage bill, a leaked credential, or an incident review has already exposed the gap.

How It Works in Practice

In an attached-provider-key design, the gateway accepts a customer gateway access key and uses a customer-supplied provider secret behind the scenes to call the upstream AI service. That means the customer controls the provider account, the quota, and often the audit trail. The gateway is acting as a policy and routing layer, not as the owner of the inference relationship. This is useful when an organisation needs chargeback, direct vendor contracting, or strict data residency decisions tied to its own cloud account.

In an AI gateway-managed model, the operator holds the provider key and exposes a managed service to the customer without requiring the customer to store or present the upstream secret. The customer governs usage through the gateway contract and policy, while the operator handles provider rotation and direct provider access. This can reduce secret sprawl, but it also shifts trust: the customer must rely on the operator’s controls for isolation, metering, logging, and abuse prevention. For broader secrets governance, the NHI Lifecycle Management Guide is a useful reference point for assignment, rotation, and retirement discipline.

• Use attached keys when customer-owned billing, compliance evidence, or upstream contract control is required.
• Use gateway-managed keys when you want to remove provider secrets from customer environments and centralise rotation.
• Require separate controls for each model: key inventory, revocation playbooks, usage alerts, and abuse thresholds.
• Verify whether the gateway logs raw prompts, responses, and token usage, since that affects both privacy and incident forensics.

Current guidance suggests treating either model as a governance decision, not just an integration choice, because the security boundary changes depending on who can rotate, revoke, and observe the upstream provider credential. These controls tend to break down in multi-tenant environments where billing, routing, and secret ownership are split across different teams, because no single party can prove end-to-end accountability.

Common Variations and Edge Cases

Tighter key custody often increases operational overhead, requiring organisations to balance direct account control against simpler managed operations. The tradeoff is especially visible when teams want customer-owned inference accounts but also want the operator to manage everything else. In those cases, a hybrid model sometimes emerges: the customer keeps the provider account, while the gateway enforces policy and request mediation. Best practice is evolving here, and there is no universal standard for how much of the trust chain should sit with the gateway versus the customer.

Edge cases usually appear around subcontractors, regional deployments, and shared service models. For example, if a reseller or platform team holds the provider key, the end customer may believe it owns the inference contract when it does not. That can complicate incident response, export controls, and retention commitments. Security teams should also compare the model with the control intent in NIST SP 800-53 Rev. 5, especially around access enforcement, auditability, and configuration management.

For threat analysis and common NHI failure modes, the Top 10 NHI Issues remains relevant, particularly where secret sprawl, weak rotation, and unclear ownership intersect. Attached keys fail when customers cannot operationally protect the upstream secret they technically own; gateway-managed keys fail when customers assume the gateway also owns the risk they still inherit contractually.

Related resources from NHI Mgmt Group

• What is the difference between keeping AI gateway analytics in customer-owned object storage and running a managed logging database in the provider cloud?
• How should security teams handle provider keys for AI gateway traffic without putting them in application code or policy files?
• Why do attached provider keys reduce risk compared with storing AI credentials directly in traffic policy?
• What is the difference between a direct model integration and a multi-provider AI gateway?