Security teams should keep agent credentials in an encrypted vault that sits outside the model and the prompt path. The strongest pattern is per-user token storage, encryption at rest with KMS, and TLS in transit, so the LLM never sees the secret. At execution time, the runtime injects the token only for the action being performed, then returns the result without persisting the credential in model memory.
Where should agent credentials live?
Agent credentials should be treated as secret material, not model context. The safe pattern is to store them in an external encrypted vault and let the runtime retrieve them only when an action needs to execute. That keeps the model blind to the secret itself, reduces prompt leakage risk, and preserves a clear boundary between reasoning and authorization.
The practical distinction is between a system that can use a credential and a model that can inspect it. If the credential passes through the prompt path, it becomes available to the model and to anything that can be logged, replayed, or exfiltrated from that path. Keeping the secret outside the model also makes rotation, revocation, and audit much easier to manage centrally.
For teams standardising this pattern, Secrets Management Guide is the clearest foundation for centralising secrets, using dynamic secrets where possible, and moving toward secretless access patterns. The broader credential lifecycle issues are also covered in API Key Management Guide, which is useful when the credential in question is a scoped token or API key rather than a traditional password.
Why per-user token storage is stronger than shared agent secrets
Per-user token storage limits blast radius and gives the runtime a more precise authority model. Instead of one shared credential that every workflow can reuse, each user or delegated context gets a distinct token, so a compromise, logout, or revocation affects only that scope. Encryption at rest with KMS and TLS in transit protect the vault and the transport, but the real gain comes from scoping and separation, not encryption alone.
This is where the design becomes operationally important. Shared secrets are harder to revoke safely, harder to attribute, and easier to misuse across tools or agents. Per-user or per-session tokens make it possible to bind an action to a specific user intent, which is especially valuable when the agent can trigger external systems or financial, administrative, or customer-impacting workflows. The token should exist only long enough to complete the action and then be discarded.
For credential rotation and expiry discipline, Guide to NHI Rotation Challenges explains why long-lived credentials become brittle at scale. If you are standardising on token-based delegation, RFC 8693: OAuth 2.0 Token Exchange is the relevant delegation pattern to study because it supports on-behalf-of flows without handing the original secret to the model.
How runtime injection keeps the model out of the secret path
The key control is just-in-time injection at execution time. The model can propose an action, but the runtime fetches the credential from the vault, attaches it to the specific call, and immediately clears it from transient memory after use. That means the model sees the outcome of the action, not the secret itself, and the credential never becomes part of the conversation history or model memory.
That separation matters because many leaks happen through convenience paths: logs, prompts, traces, debugging output, and copied examples. If the secret is ever embedded in the prompt, the protection boundary has already been weakened. If the runtime handles injection, you can still enforce least privilege, session expiry, and revocation without teaching the model the credential value.
For implementation patterns, Secrets Management Buyer’s Guide helps teams evaluate vault capabilities such as dynamic issuance, rotation, and access policy. For the underlying access pattern, RFC 6749: The OAuth 2.0 Authorization Framework is useful when the runtime is obtaining a scoped token rather than retrieving a static password.
Risk and Threat Considerations
When agent credentials are exposed to the model path, they can leak through prompts, logs, traces, memory reuse, or downstream tooling that was never intended to hold secret material. The risk is amplified when the same credential is shared across users or environments, because a single exposure can widen into full account misuse.
Failure mechanism: The model or an adjacent system ingests the secret as text, then the credential becomes available to prompt inspection, output replay, telemetry capture, or malicious prompt injection that tries to elicit or exfiltrate it.
Impact: Attackers can reuse the credential for unauthorized API calls, privilege escalation, data access, or persistent abuse until the secret is rotated and all dependent sessions are invalidated.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 and OWASP API Security Top 10 address the attack and risk surface, while NIST SP 800-53 Rev 5 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | IA-5 — Authenticator Management | Credentials here must be stored, rotated, and revoked safely. |
| IA-9 — Service Identification and Authentication | Agent runtimes and services authenticate with non-human credentials. | |
| AC-6 — Least Privilege | Scoped tokens reduce blast radius if a credential is exposed. | |
| Recommendation — Manage agent tokens as authenticators and rotate or revoke them promptly. Use service-oriented authentication controls for runtime-issued agent tokens. Scope each agent token to the minimum access needed for the task. | ||
| OWASP Non-Human Identity Top 10 | NHI-02 — Secret Leakage | The question is specifically about keeping agent secrets out of the model path. |
| NHI-07 — Long-Lived Secrets | Runtime-injected credentials should avoid unnecessary persistence. | |
| NHI-05 — Overprivileged NHI | Per-user tokens and scoped execution directly limit excess access. | |
| Recommendation — Store agent credentials outside prompts and models to prevent leakage. Prefer short-lived credentials and retire them immediately after use. Reduce agent permissions so a leaked token cannot do broad damage. | ||
| OWASP API Security Top 10 | API2 — Broken Authentication | Agent tokens are used to authenticate calls made by the runtime. |
| API5 — Broken Function Level Authorization | Runtime-injected tokens must only authorize the intended action. | |
| Recommendation — Protect token issuance and validation so injected credentials remain trustworthy. Bind each token to the specific function or operation the agent may invoke. | ||
Practitioner Guidance
What to prioritise: Decide first whether the agent actually needs a reusable secret at all. In many cases, a short-lived delegated token, exchanged at runtime and bound to one action, is safer than storing a long-lived credential that the agent can reuse.
What to verify: Confirm that the model never receives the credential in prompts, tool arguments, examples, logs, or retrieval content. Also verify that the vault path, token exchange path, and execution path are separate enough that a debug trace cannot reconstruct the secret.
Common mistake: Teams often encrypt a credential and still expose it by passing the decrypted value through the model boundary. Encryption helps, but it does not compensate for poor placement. The control only works when the model is excluded from the secret path entirely.
Practitioner takeaway: The safest design is not “hide the password better”, it is “never let the model possess the password in the first place.” Keep secrets in a vault, inject them only at execution time, and make the runtime, not the model, the point of authority.
Related resources from NHI Mgmt Group
- How should security teams govern AI agent tool calls without exposing credentials?
- How should security teams debug JWTs without exposing live credentials?
- How should security teams implement credential access for browser-based AI agents without exposing secrets to the model?
- How should security teams phase agent rollouts before exposing them to real users and data?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 30, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
