Attached provider keys are customer supplied credentials linked to a gateway access key, so the customer controls the upstream account used for inference. AI gateway-managed provider keys are controlled by the gateway operator and require no customer-supplied provider secret. The difference matters for governance, billing ownership, rotation responsibility, and trust boundaries.
Why Attached and Gateway-Managed Keys Are Operationally Different
These two models answer a practical governance question: who owns the upstream provider relationship, and who carries the secret-management burden. Attached provider keys preserve customer control over the inference account, which can be important for billing separation, procurement, and auditability. Gateway-managed keys reduce customer secret handling, but they also shift trust to the gateway operator and make that operator part of the access and rotation model. For teams comparing the models, the key issue is not just convenience, but where accountability and failure responsibility sit.
When a gateway brokers access to external model providers, the credential model becomes part of the security boundary rather than a background implementation detail. That matters because provider keys can be overprivileged, misrotated, or reused across environments, and the consequences often show up later as unexpected spend, broken inference paths, or unclear incident ownership. In practice, many security teams notice the governance gap only after the first rotation failure or billing dispute has already exposed it.
How the Two Key Models Work in Practice
With attached provider keys, the customer supplies a provider secret to the gateway, and the gateway uses that secret when it calls the upstream model service. The customer therefore owns the lifecycle of that credential, including creation, storage, rotation, revocation, and any policy that limits which workloads can use it. This model is usually preferred when the organisation wants direct control over the upstream tenant, a clean billing line, or the ability to prove exactly which account was used for inference.
With ai gateway-managed provider keys, the gateway operator holds the upstream secret and the customer does not provide one. The customer may still control which applications can call the gateway, but they no longer control the provider account itself. That can simplify onboarding and reduce secret sprawl, especially where many applications need access and the provider relationship is meant to be abstracted behind the gateway.
The difference shows up in three operational areas:
- Rotation: attached keys require the customer to coordinate provider-secret updates with the gateway; gateway-managed keys centralise that responsibility.
- Trust boundary: attached keys keep the upstream provider account within the customer’s control; managed keys extend trust to the gateway operator.
- Failure handling: attached keys fail when customer-controlled credentials expire or are revoked; managed keys fail when the gateway operator’s provisioning, policy, or provider relationship breaks.
For teams applying this model in production, the control question is whether the gateway is a pass-through for customer-owned access or an intermediate custodian of access. That distinction drives logging, approval workflow, billing attribution, and the evidence needed during audit or incident review. The NIST Cybersecurity Framework 2.0 is useful here because it frames ownership, governance, and recovery as system-level concerns rather than isolated technical settings. Where the gateway also brokers broader access control decisions, teams should treat the secret model as part of the access architecture, not just a deployment choice. This guidance breaks down when the gateway operator cannot clearly document who owns the upstream provider account or how revocation is enforced.
Where the Model Choice Becomes a Governance Trade-off
Tighter customer control often increases operational overhead, while gateway-managed keys reduce friction but introduce a stronger dependency on the operator. Organisations need to balance autonomy against abstraction, especially when the same gateway serves multiple business units or environments.
One common variation is hybrid use: some teams attach provider keys for regulated workloads and use gateway-managed keys for lower-risk internal use cases. That can be sensible, but only if the organisation clearly distinguishes which workloads require direct customer ownership and which can accept operator custody. Another edge case appears when a gateway supports both modes at once; without clear policy, teams may assume they have the same billing, revocation, or audit properties when they do not.
There is no universal consensus that one model is always superior. The right choice depends on whether the priority is customer-held control, reduced secret handling, or simplified central administration. In practice, the model should be selected by the ownership boundary first, and by convenience second. Where the provider key is attached, the customer must be prepared to prove its rotation and scope; where the key is managed by the gateway, the customer must be prepared to trust the operator’s controls and incident response process. The NIST SP 800-53 Rev 5 Security and Privacy Controls is relevant when evaluating how authentication, access enforcement, and control ownership are documented across that boundary.
Risk and Threat Considerations
The main risk is misplaced trust in who actually controls the upstream credential and what that means for exposure if the credential is abused, over-scoped, or not revoked cleanly. Attached keys concentrate risk in customer-managed secret handling, while gateway-managed keys concentrate risk in the operator’s custody and operational discipline.
Failure mechanism: In the attached model, a leaked or stale provider secret can be reused outside the intended gateway path if scope and rotation are weak. In the managed model, a compromised or misconfigured gateway can become the single point through which many workloads inherit upstream access, and a customer may have limited visibility into the provider-side control state.
Impact: The result can be unauthorised inference spend, loss of tenant separation, failed revocation, delayed incident containment, or disputes over which party is responsible for a security or billing event.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
CIS Controls v8 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| CIS Controls v8 | 5 — Account Management | Covers lifecycle ownership of provider credentials and access paths. |
| Recommendation — Apply CIS Control 5 to define ownership, provisioning, and revocation for provider keys. | ||
| NIST CSF 2.0 | GV.RM — Risk Management Strategy | Fits the governance trade-off between customer custody and operator custody. |
| ID.AM — Asset Management | Provider keys and gateway custody are assets that need clear inventory and ownership. | |
| PR.AA — Identity Management, Authentication, and Access Control | Key models directly affect upstream authentication and access enforcement. | |
| Recommendation — Use GV.RM to set the key-custody model that matches your risk tolerance and accountability. Maintain an inventory of who owns each provider key and where it is used. Enforce PR.AA so upstream authentication follows the intended custody model. | ||
Practitioner Guidance
What to verify: Confirm who can create, rotate, revoke, and audit the upstream provider credential, and verify that the answer matches the commercial and security model you think you bought. If the gateway says it is “managing” keys, check whether that means operator custody, delegated customer custody, or a split arrangement with different evidence requirements.
Decision rule: Use attached provider keys when the customer must retain clear upstream ownership, billing attribution, or provider-side accountability. Use gateway-managed keys when reducing secret sprawl and simplifying onboarding matters more than direct custody, but only if the operator can document rotation, revocation, and incident handling with enough clarity to support trust.
Practitioner takeaway: The real difference is not technical convenience but where the security and audit burden lands when something goes wrong, so teams should choose the model that matches their evidence and accountability needs before they choose the one that is easiest to deploy.
Related resources from NHI Mgmt Group
- What is the difference between keeping AI gateway analytics in customer-owned object storage and running a managed logging database in the provider cloud?
- What breaks when AI provider keys are left in internet-reachable gateway policy instead of attached to a managed access key?
- What is the difference between a direct model integration and a multi-provider AI gateway?
- What is the difference between routing AI requests through a gateway and integrating each provider directly?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 7, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
