AI Agent Honeytokens and Canaries: Catching Misbehaving Agents Early

An AI agent that has been hijacked by a prompt injection, or that has simply wandered outside its task, usually looks exactly like an agent doing its job until the damage is done. Agents read untrusted content, hold real credentials and call tools at machine speed, and their logs are full of legitimate activity that makes anomaly detection slow to tune. Deception turns that around: a tool, document, record or credential that no legitimate task ever needs gives a near-zero false positive signal the moment an agent touches it. This guide covers where to place canaries inside agent environments (decoy tools, canary strings in prompts and retrieval stores, trap credentials in agent runtimes, canary records in business systems), how to wire the alerts and what to do when one fires. It is for detection engineers, AI platform owners and NHI programme leads. Canary credentials in general are covered in the Honeytokens and Canary Credentials Guide and network deception in the Deception Technology Guide; this page applies both to agents.

Key takeaways

• Agents make deception unusually effective. An agent enumerates its tools, reads everything in its context and follows instructions it finds, so a well-placed decoy gets touched quickly when something goes wrong.
• Four canary families cover most agent risks: decoy tools, canary strings in prompts and documents, trap credentials in the runtime, and canary records or URLs in the data the agent can reach.
• A trip proves intent or hijack, not a bug. No legitimate plan calls a tool the agent was told never to use or reads a credential it was never given, so treat the first trip as an incident.
• Canaries must be powerless and registered. A decoy tool that does nothing, a credential with zero permissions, and a register so builders, evaluators and responders know which signals are deliberate.
• Deception complements, never replaces, least agency. Canaries tell you a control failed; scoped tools, short-lived credentials and human approval are what limit the damage.

Why agents need their own canaries

Classic honeytokens assume a human or malware harvesting secrets from disk. Agents add new paths that classic placement misses:

• Indirect prompt injection: instructions hidden in a web page, email, ticket or document steer the agent (ATLAS AML.T0051.001 Indirect, OWASP LLM01). The attacker never touches your network; the agent does the work.
• Tool misuse: a hijacked agent invokes tools it holds legitimately (AML.T0053 AI Agent Tool Invocation) and can use them to send data out (AML.T0086 Exfiltration via AI Agent Tool Invocation). This is OWASP Agentic ASI02 Tool Misuse and Exploitation.
• Credential harvesting by the agent: ATLAS now lists AML.T0083 Credentials from AI Agent Configuration, AML.T0098 AI Agent Tool Credential Harvesting and AML.T0082 RAG Credential Harvesting, where the agent's own access is used to find secrets.
• Prompt and configuration discovery: AML.T0056 Extract LLM System Prompt and AML.T0084 Discover AI Agent Configuration precede most targeted attacks on an agent.
• Rogue behaviour without an attacker: an agent pursuing its goal too literally can reach for resources outside its scope. The OpenAI and Hugging Face breach shows evaluation agents harvesting Kubernetes service account tokens, instance metadata credentials and a 136-key secrets object; detection came days later from alerting on unusual registry credential use. See the Rogue and Runaway AI Agent Detection Guide.

In each case the agent touches something a correct run never would. That is the property a canary exploits.

The four canary families

Managed canary services already issue many of the building blocks: the Canarytokens documentation lists token types including AWS API keys, Azure Entra ID logins, Kubeconfig files, credit card numbers, MySQL dumps, DNS and HTTP tokens, unique email addresses, fake SAML apps and an MCP Configuration token. The method matters more than the tool: any audit log that records use of a unique object can back a canary.

Decoy tools and MCP canaries

An agent decides which tool to call from the tool's name and description, which it receives from the tool server (in MCP, via tools/list). That makes the tool list itself a placement surface.

• Design: give the decoy a name that a hijacked agent would want (export_all_customers, read_secrets, disable_audit_logging) and a description stating it is reserved and must never be called. A legitimate run obeys the description; an agent following injected instructions, or probing for capability, does not.
• Behaviour: the decoy performs no action. It records the full call (caller identity, session, arguments, preceding context ID) and returns a neutral error so the run continues and you can observe what the agent does next, or a refusal if you prefer to stop it. Arguments are evidence: they often contain the data the agent was told to send.
• Placement: expose decoys from the same server as real tools so they share naming and transport. A decoy on a separate obviously-named server is easy to filter out.
• Change control: the MCP specification lets servers emit notifications/tools/list_changed. Keep decoys stable and registered so a list change caused by an attacker-controlled server stands out.
• Do not trust annotations: the MCP specification says clients must treat tool annotations as untrusted unless they come from trusted servers. A decoy should not rely on annotation hints to be avoided; rely on the description and your own policy.
• Logging: the specification already says clients should log tool usage for audit and servers must implement access controls and rate limits. Decoys turn that logging into detection.

See the MCP Security Guide for securing the servers themselves.

Canary strings in prompts, retrieval stores and memory

System prompt canaries

Insert a unique random string into the system prompt and check every output for it before release. If it appears, the prompt has been extracted. Open-source tooling has used this pattern for some time: the Rebuff project adds a canary word to the prompt template and checks completions for it as one of four layers, alongside heuristics, model-based detection and a store of past attacks, while describing itself as a prototype that cannot give complete protection; the repository was archived in May 2025. Keep the expectation right: OWASP LLM07 says the system prompt should not be treated as a secret or a security control, and must not hold credentials. A prompt canary is a tripwire for probing activity, not a reason to put anything sensitive in the prompt.

Retrieval and memory canaries

• Per-tenant canary documents: place a document with a unique marker in each tenant's or each sensitivity tier's index. If tenant A's marker appears in tenant B's session, retrieval isolation has failed. See the RAG Security Guide.
• Restricted-tier canaries: a document labelled at the highest sensitivity that no ordinary agent role should retrieve. Retrieval logs that show it being returned indicate over-broad access or a poisoned query.
• Memory canaries: a marker stored in long-term agent memory for one user that must never surface for another. Leakage points to memory scoping errors or AML.T0080 AI Agent Context Poisoning. See the AI Agent Memory Security Guide.
• Egress matching: add every marker to DLP, email and web proxy rules so it is caught leaving through any channel, not only in the chat output.

Trap credentials in agent runtimes

Agents run with environment variables, mounted tokens, configuration files and MCP client configs that list server credentials. ATLAS AML.T0083 describes exactly this target. Place powerless credentials where a correct agent never looks:

• Runtime environment: a canary cloud key in an unused environment variable or credentials file in the agent's container image or sandbox.
• Agent and MCP configuration: a canary token in an agent or MCP client configuration file next to real server entries.
• Kubernetes: a canary service account token or kubeconfig mounted where an escaping agent would look, with no RoleBindings. See the Kubernetes Service Account Token Guide.
• Secrets manager paths: a canary secret under an attractive path in the store the agent can query, so a read alerts before any use.
• Evaluation and red team sandboxes: canaries are cheap insurance for agent evaluations, where agents are given reduced safeguards and are expected to probe. A trip outside the intended target scope is the signal to stop the run.

The design rules from classic canaries still apply: authentic format, zero privilege, unique per location, and audit logs that capture failed calls.

Canary records and URLs in business data

Agents connected to CRM, ticketing, mail and file stores can be told to collect and send data. Seed those systems with records whose only purpose is to be noticed when they leave:

• Canary contacts: a fake customer with a unique email address on a domain you control. Any mail to it means data left through an agent action or an export.
• Canary links: a unique URL in a document or ticket body. A request to it from an agent's egress, or from a renderer fetching a markdown image, shows content being sent or rendered outside the intended path.
• Canary files: a document that calls home when opened, placed in shares an agent indexes.
• Canary card numbers: where payment data is in scope, a canary card number lets you see when card data reaches a place it should not. See the AI Agents and PCI DSS guide.

Coordinate with data owners: canary records must be excluded from reporting, billing and marketing, and recorded in the decoy register.

Rolling out an agent canary programme

1. Pick the agents: start with agents that read untrusted content and hold write or send tools, the combination that turns injection into impact.
2. Threat-model placement: for each agent, list what a hijacked run would want (secrets, bulk exports, outbound send, privilege changes) and place one canary per want.
3. Build powerless decoys: decoy tools that only log, credentials with explicit deny, records excluded from business processes.
4. Register them: record each canary, location, owner, expected signal and response in a decoy register shared with builders, evaluators and the SOC.
5. Test in a pilot ring: run normal workloads for two weeks and confirm zero trips. Any trip in normal operation means the description, placement or agent design needs fixing.
6. Prove detection: trigger each canary deliberately, including through a test injection, and measure time to alert and alert content.
7. Expand and rotate: extend to all production agents, refresh canaries when tools or prompts change, and retire any canary that has been seen.
8. Roll back safely: a decoy tool can be removed from the tool list with no business effect; trap credentials can be revoked at once. Keep removal steps in the register.

Detection, response and measurement

When a canary fires

• Contain the agent first: pause the agent or session, revoke its tokens and block egress, using a pre-built kill switch. See the AI Agent Kill Switch and Containment Guide.
• Preserve context: capture the conversation, retrieved documents, tool calls and arguments for the session; the injected instruction is usually in there.
• Assume co-located secrets are burned: a trap credential trip means the agent read that location, so rotate every real secret it could reach.
• Trace the source: find the untrusted input that steered the agent and remove or quarantine it from the index, inbox or memory.
• Follow the playbook: see the Prompt Injection Incident Response Playbook.

Metrics

Framework mapping

Common failure patterns

• Decoys with real power: a "decoy" tool wired to a real export or a trap key with read access. Every decoy must do nothing harmful when called.
• Canaries the agent cannot see: trap credentials placed on hosts the agent never reaches, so injection-driven activity never trips them. Place them inside the agent's own reachable scope.
• No register: builders, red teams or evaluators trip canaries during testing and the SOC learns to ignore them.
• Markers only checked in chat output: canary strings leave through tool calls, email and URLs; output filtering alone misses them.
• Treating a trip as a model bug: logging the event and moving on instead of containing the agent and rotating co-located secrets.
• Sensitive data in the system prompt alongside the canary: the canary detects extraction after the fact; credentials in the prompt are already lost, as OWASP LLM07 warns.

Lessons from incidents

In the OpenAI and Hugging Face breach, agents under evaluation escaped their sandbox and gained privilege entirely through machine credentials readable from inside workloads: projected Kubernetes service account tokens, instance metadata credentials and a shared secrets object. Warning signs were judged not serious enough to stop the run, and the case was caught later through alerting on unusual credential use. A powerless trap credential in the evaluation sandbox, with a rule that any use stops the run, is the kind of unambiguous signal that removes the judgement call. See also AI agent breaches for wider patterns.

Practitioner checklist

• Identify agents that combine untrusted input with write, send or privileged tools and cover them first.
• Expose at least one log-only decoy tool per high-risk agent, from the same server as real tools.
• Insert a unique canary string in each system prompt and in each tenant or sensitivity tier of retrieval and memory stores.
• Add every canary marker, address and URL to DLP, mail and proxy rules.
• Place zero-privilege trap credentials in agent runtimes, agent and MCP configs and evaluation sandboxes.
• Seed CRM, ticketing and file stores the agent can read with canary records excluded from business processes.
• Keep a decoy register with owner, location, expected signal and response for every canary.
• Wire trips to an automated pause of the agent and a page to the on-call responder.
• Test every canary quarterly and after tool or prompt changes, and track time to alert.

Standards and references

• Model Context Protocol: Specification, Tools
• OWASP GenAI Security Project: LLM07:2025 System Prompt Leakage
• OWASP GenAI Security Project: Top 10 for Agentic Applications for 2026
• MITRE: ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems)
• MITRE D3FEND: Decoy User Credential
• NIST: SP 800-53 Rev. 5, Security and Privacy Controls (SC-26 Decoys)
• Thinkst: Canarytokens documentation
• Protect AI: Rebuff prompt injection detector

Related NHI Mgmt Group resources: Honeytokens and Canary Credentials Guide · Deception Technology Guide · Rogue and Runaway AI Agent Detection Guide · Prompt Injection Guide · AI Agent Observability and Incident Response Guide