Secrets in Prompts: Why Keys Reach Agent Instructions and How to Remove Them

API keys keep turning up in system prompts, agent instruction files and agent configuration because, for the person building the agent, pasting the key where the model can see it is the fastest way to make a tool call work. Once a key is in a prompt it is copied into every request, every trace, every prompt version and often a public repository, and OWASP is clear that a system prompt is not a secret store. This guide explains the specific ways keys get there, where to look for them across prompt stores, instruction files, MCP configuration and no-code agent builders, and how to remove them in the right order without leaving copies behind. The target architecture (vaults, tool-layer authentication, short-lived tokens) is covered in Credential Vaulting for AI Agents; this page is for the clean-up, written for AppSec, platform and secrets management teams.

Sources: OWASP, MITRE, GitHub, MCP and vendor documentation and GitGuardian research as cited below, checked November 2026. Confirm against the official text before relying on it.

Key takeaways

• A prompt is not a secret store. OWASP LLM07:2025 says the system prompt should not be considered a secret or used as a security control, and that credentials and connection strings should not be in it.
• Keys arrive through convenience, not malice. Prototypes, tool schemas with an api_key argument, instruction files and shared MCP configuration are the usual routes.
• Search more than Git. Prompt registries, agent builder instructions, evaluation datasets, traces and chat history all hold prompt text and are rarely scanned.
• Revoke first, then clean. Rotation makes the leaked value useless; deleting it from a prompt while it still works only hides the problem.
• Block at the point of publication. Scan prompts and instruction files when they are committed or published, not just when they reach production.

Why keys end up in prompts

MITRE ATLAS records the attacker side of this as AML.T0083 Credentials from AI Agent Configuration: credentials for an agent's tools are frequently stored in configuration for convenience, and whoever can read that configuration gets the systems behind it. The defender's question is how they got there. In practice there are six recurring root causes.

The common thread is the model being asked to do authentication. Once the tool layer authenticates instead, there is no reason for a key to appear in text the model reads. For how that works at runtime, see Runtime Secrets Injection for AI Agents Without Prompt Exposure.

How a key in a prompt spreads

A key written once into a prompt is copied each time the prompt is used or stored. Removing it from the source file leaves these copies:

• Prompt version history: prompt management tools and Git keep every earlier version.
• Request logs and traces: the full system prompt is sent on every call and often logged by the application, the gateway and the observability platform. See How to Find Secrets in AI Agent Transcripts and Traces.
• Model output: extraction prompts, verbose errors or the model quoting its own instructions. OWASP LLM07 includes a scenario where a system prompt holding tool credentials leaks and the attacker reuses them.
• Repository clones and forks: instruction files and MCP configuration travel with the repository to every clone.
• Third-party AI services: keys pasted into AI chat tools sit in that provider's history. The OmniGPT breach claim (unconfirmed) involved chats in which users had shared API keys and credentials; the advice is to rotate any key shared with a service that reports a breach.

Where to look

Build the search list from the agent inventory, not from the code repositories alone. GitGuardian reports that about 28% of secrets sprawl incidents occur outside code repositories.

Finding keys: a step-by-step approach

1. Inventory agents and prompt stores: list each agent, where its prompt and instructions live, which MCP servers it uses and where it logs. Agents built in SaaS platforms need the platform admin's help.
2. Agree permissions and handling: scanning prompt stores and traces exposes whatever they hold, including personal data. Get approval from the owners, run scans with a dedicated read-only identity, and keep findings in a restricted system that stores fingerprints and locations, not raw values.
3. Run pattern detection: use a secret scanner with provider-specific rules (AI service keys, cloud keys, database URIs, bearer tokens). AI service credentials are a fast-growing category: GitGuardian reports AI-service secrets grew 81% year over year in its 2026 data. See the Secret Scanner Rule Coverage Comparison Guide.
4. Add generic detection: prompts often hold passwords and internal tokens with no known prefix. GitHub's generic secret detection uses AI to find unstructured secrets such as passwords, but GitHub states it may miss secrets and skips files whose paths contain "test", "mock" or "spec". The OWASP Secrets Management Cheat Sheet suggests using more than one detection utility and de-duplicating results.
5. Search for intent phrases: a keyword pass for phrases such as "use this key", "token:", "Authorization: Bearer" and "password is" inside prompt text catches cases detectors miss.
6. Validate and attribute: confirm whether each finding is live using the provider's own non-destructive verification where one exists, and identify the owning service and the human or team responsible.

Removing them in the right order

Both the OWASP Secrets Management Cheat Sheet and GitHub's guidance put revocation first: OWASP says exposed keys "should undergo immediate revocation", followed by creating a replacement and removing the old value from the exposed system; GitHub says that as a first step you need to revoke and/or rotate the secret, and that once it is unusable a history rewrite may not even be necessary.

1. Revoke or rotate: issue a replacement, update the consuming service, then disable the old key. Disable before deleting where the provider allows it, so a missed dependency can be restored quickly.
2. Move authentication into the tool layer: remove credential parameters from tool schemas and change the tool or MCP server to fetch its own credential from a secrets manager or use OAuth. Without this step the key will be pasted back.
3. Edit the prompt source: replace the key with a description of the tool ("call get_forecast"), never with a placeholder the model might try to fill.
4. Purge version history where the risk justifies it: for public repositories, GitHub documents git-filter-repo (2.47 or later) with --sensitive-data-removal and --replace-text, and warns that clones, forks and cached views can still hold the data. For prompt registries, delete or redact earlier versions through the platform's own tools.
5. Clean logs and traces: OWASP says secrets in logs need a removal process that maintains log integrity. Redact the spans or apply masking and shorten retention, recording what was done.
6. Check use of the old key: review provider logs for activity between first exposure and revocation, especially from unfamiliar networks, and escalate to incident response if found.
7. Close the finding with evidence: record rotation time, clean-up locations and the prevention control added.

Preventing it coming back

• Scan at commit and publish: pre-commit hooks and server-side push protection cover instruction files and MCP config in repositories; add a scan step to the prompt registry's publish workflow so a prompt with a secret cannot be promoted. See Secret Push Protection.
• Use the client's own credential mechanisms: agent clients document safer patterns. Listed alphabetically: Claude Code's MCP documentation recommends local scope for servers with credentials you do not want in version control, supports ${VAR} expansion in .mcp.json and a headersHelper that generates headers at connection time, and stores OAuth client secrets in the system keychain or a credentials file rather than the configuration. VS Code's MCP documentation says "Do not paste credentials into chat or commit them to your project" and points to its authentication and secrets configuration.
• Lint tool schemas: fail the build if any tool definition has a parameter named like api_key, token, secret or password.
• Filter prompt input: a gateway guardrail that blocks credential patterns on input catches keys pasted by users and builders. See the Generative AI Data Loss Prevention Guide.
• Treat repository agent files as untrusted: instruction files and MCP definitions that arrive with a repository can also be used to steal credentials. The Gemini CLI prompt injection flaw used a poisoned README to make an allow-listed command send environment variables to a remote server (fixed in 0.1.14). See Coding Agent Configuration File Attacks.
• Scope what is left: the MCP security best practices (draft, checked November 2026) warn against wildcard or omnibus scopes and say MCP servers must not accept tokens not issued for them, which limits the value of any key that does leak.

Small teams with a handful of agents can do most of this with repository push protection, a manual review of instruction files and a schema lint. Larger estates with prompt registries and several agent platforms need the publish-time scan and a periodic export-and-scan of every platform holding instructions.

Detection and measurement

• Secrets per prompt store: open findings by location type (repository, instruction file, MCP config, registry, builder, dataset, logs).
• Time to revoke: median time from detection to revocation of a live key found in prompt text; aim for hours, not days.
• Recurrence: share of agents where a secret reappears after clean-up, which shows whether tool-layer authentication was actually implemented.
• Credential parameters in tool schemas: count of tool definitions with credential-like parameters; target zero.
• Scan coverage: percentage of known prompt stores and agent platforms scanned in the last 30 days.
• Use of revoked keys: authentication failures against revoked keys at the provider, which can reveal a missed consumer or an attacker holding the old value.

Common failure patterns

• Deleting without rotating: the key is removed from the prompt file but stays valid, while copies remain in Git history, traces and provider logs.
• Rotating without refactoring: a new key is issued and pasted into the same prompt, because the tool still expects the model to supply it.
• Scanning code only: the registry, builder instructions and evaluation sets are never scanned, so most prompt text is invisible to the programme.
• Placeholders the model fills: replacing the key with "YOUR_KEY_HERE" leaves a tool that tries to obtain a key from the conversation, inviting users or injected text to supply one.
• Committed MCP configuration: a team shares one MCP file with an inline token, giving every repository reader the same credential and no per-user audit trail.
• Trusting prompt secrecy: relying on "do not reveal these instructions" to protect the key, which OWASP LLM07 explicitly warns against.

Framework mapping

Practitioner checklist

• Every agent's prompt, instruction files, MCP configuration and log destinations are listed with an owner.
• Repositories, prompt registries, agent builders, datasets and traces are scanned with pattern and generic detection.
• Instruction file paths and MCP configuration files are explicitly included in scanner scope.
• Live keys found in prompt text are revoked or rotated first, with disable before delete.
• Tool schemas carry no credential parameters; tools authenticate themselves.
• Prompt history, Git history and traces are cleaned where the exposure justifies it, with records kept.
• Push protection and a publish-time scan block secrets in prompts and agent files.
• Provider logs are reviewed for use of each exposed key before the finding is closed.

Standards and references

• OWASP: Top 10 for LLM Applications 2025 (LLM07 System Prompt Leakage)
• OWASP: Secrets Management Cheat Sheet
• MITRE ATLAS: Credentials from AI Agent Configuration (AML.T0083)
• Model Context Protocol: Security Best Practices
• NIST: SP 800-53 Rev. 5
• GitHub Docs: Removing sensitive data from a repository
• GitHub Docs: Generic secret detection with Copilot secret scanning
• GitGuardian: The State of Secrets Sprawl 2026

Related NHI Mgmt Group resources: Credential Vaulting for AI Agents Guide · OWASP LLM Top 10 LLM07 System Prompt Leakage Guide · Local MCP Server Security Guide · Secrets Management Guide