Storing credentials in the LLM context puts secrets inside the reasoning loop, where they can leak through prompts, logs, or traces. Storing them in the action runtime keeps the model on an intent-only path, so it can request work without ever seeing the credential itself. That separation sharply improves containment and reviewability.
Why the placement changes the security model
Where credentials live is not just an implementation detail, it changes who can observe them, when they are exposed, and how far a failure can travel. In the LLM context, the credential becomes part of the model-facing conversation space, which is usually broader, less deterministic, and more likely to be copied into traces or logs. In the action runtime, the model stays at the intent layer and a separate execution boundary handles the secret.
That separation matters because the reasoning surface and the execution surface have different trust properties. The LLM can decide what should happen, but the runtime should decide how it happens using approved inputs, scoped credentials, and auditable calls. Keeping those roles apart makes the credential easier to govern and reduces the chance that a prompt, retry, or model output ever turns into credential exposure.
What happens when the credential enters the LLM context
Putting a credential into the LLM context makes it available to anything that can influence or inspect that context, including prompt injection, accidental echoing, tool output, debugging traces, or retention in conversation history. The risk is not limited to outright theft. Even a temporary appearance of a token, key, or password inside the prompt window can create secondary leakage paths that are difficult to reason about after the fact.
The other problem is that the model may treat the credential as just another text object, not as protected authentication material. That increases the odds of unsafe summarisation, inadvertent reproduction, or misuse in a downstream tool call. NHIMG’s Guide to the Secret Sprawl Challenge is useful background here because the same exposure pattern appears whenever secrets are allowed to proliferate beyond the system that should hold them.
What the action runtime protects that the model should never see
The action runtime is the better place for credentials because it can enforce bounded execution without revealing the secret to the model itself. The model issues an intent, such as “call this API” or “run this action,” and the runtime resolves identity, injects the credential, applies policy, and returns only the result that the model needs. That keeps the secret out of the prompt loop and limits it to the component that actually performs the work.
This also improves reviewability. Security teams can inspect the action boundary, the credential scope, the approved destinations, and the audit trail without having to reconstruct whether the model ever handled the secret. NHIMG’s Secrets Management Guide fits this design choice because it frames the operational shift toward secretless or secret-minimising patterns rather than prompt-based handling. In practice, the runtime should be the only place where credential material is injected, unwrapped, or exchanged.
Why this separation improves containment, but does not remove responsibility
Intent-only architecture reduces blast radius, but it does not make the system safe by default. If the runtime is overprivileged, uses long-lived credentials, or forwards tokens too broadly, a compromise of the action layer can still become a high-impact event. The security gain comes from reducing exposure to the model plus making the runtime the narrow, governed choke point for access.
That is why the design should be paired with short-lived credentials, narrow scopes, explicit allowlists, and logging at the action boundary rather than in the prompt. NHIMG’s Guide to NHI Rotation Challenges is relevant because the same lifecycle pressure applies when the runtime holds machine credentials that must expire, rotate, and be recoverable without manual drift. The core architectural decision is not “where can we hide the secret,” but “where can we confine the authority that uses it.”
Risk and Threat Considerations
Credentials in the LLM context are exposed to a much wider attack and leakage surface than credentials held by the action runtime. Once a secret appears in the reasoning loop, it can be copied into logs, surfaced by prompt injection, replayed in traces, or retained in places that are hard to sanitize consistently.
Failure mechanism: The model or surrounding tooling treats secret material as ordinary context, so disclosure can occur through prompt manipulation, debugging output, conversation persistence, or downstream tool misuse.
Impact: An exposed credential can enable unauthorized actions, widen the blast radius of a single agent session, and make containment or revocation harder because the secret may already have propagated outside the runtime boundary.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10, OWASP Agentic AI Top 10 and OWASP API Security Top 10 address the attack and risk surface, while NIST SP 800-53 Rev 5 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-02 — Secret Leakage | Secret leakage is central when credentials are exposed in the LLM context. |
| NHI-07 — Long-Lived Secrets | The runtime model depends on secret lifecycle and short-lived credential handling. | |
| NHI-05 — Overprivileged NHI | Runtime-held credentials can still become dangerous if they are scoped too broadly. | |
| Recommendation — Keep credentials out of model context and confine secret use to the runtime boundary. Prefer short-lived credentials and rotate any runtime-held secrets aggressively. Scope runtime credentials narrowly and remove unused access paths. | ||
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | The question turns on separating model intent from executable authority. |
| Recommendation — Separate agent intent from execution authority and enforce policy at the runtime boundary. | ||
| NIST SP 800-53 Rev 5 | IA-9 — Service Identification and Authentication | Runtime credential handling concerns machine-to-machine authentication. |
| AC-6 — Least Privilege | The runtime should use narrowly scoped access to limit blast radius. | |
| Recommendation — Authenticate the runtime service separately and keep its credentials outside the model. Limit runtime permissions to the minimum required for each action. | ||
| OWASP API Security Top 10 | API2 — Broken Authentication | Credentials passed through the wrong layer can weaken authentication and increase abuse risk. |
| API5 — Broken Function Level Authorization | The runtime must enforce what actions the credential may perform. | |
| Recommendation — Keep authentication material in the execution layer and never expose it to prompts. Enforce function-level authorization before the runtime executes any tool call. | ||
Practitioner Guidance
What to verify: Confirm that the model never receives raw credentials, even transiently, and that any token exchange happens only inside the runtime boundary. If the credential can be observed in a prompt, it is already in the wrong place.
Decision rule: If the model needs to choose an action, let it pass intent and parameters only; if the credential is needed to execute that action, inject it after policy checks inside the runtime. Keep the model untrusted for secret handling even when the workflow feels internal.
What good looks like: The prompt contains instructions, not secrets; the runtime holds the authority; and the audit trail shows action selection, credential use, and result delivery as separate events.
Practitioner takeaway: The safest pattern is not secret hiding, it is secret removal from the reasoning path, with the action runtime acting as the only trusted place where credential-bearing authority is exercised.
Related resources from NHI Mgmt Group
- What is the difference between storing secrets in docker-compose files and injecting them at runtime from a secrets manager?
- What is the difference between storing CI/CD secrets in pipeline variables and retrieving them from a central secrets manager at runtime?
- What is the difference between storing secrets in a vault and exposing them at runtime through an environment layer?
- What is the difference between storing credentials in a vault and sharing them through insecure channels?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 30, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
