AI agent prompt injection exposed full system prompts in testing

At a glance

What this is: This is a report on how an AI pentesting engine bypassed an agent guardrail and extracted sensitive configuration details from a production-style AI assistant.

Why it matters: It matters because AI agent governance now depends on controlling what the agent can reveal and call, not just blocking obvious malicious prompts.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.
• 92% agree governing AI agents is critical to enterprise security, yet only 44% have implemented any policies to do so.

👉 Read Escape's analysis of how Cascade exposed AI agent system prompts

Context

AI agent guardrails often fail on intent rather than syntax. A request that is blocked when phrased directly can still succeed when it is reframed as a legitimate-sounding question, which makes prompt injection a governance problem as much as a model-safety problem. For IAM and NHI teams, the sensitive issue is not only what the agent answers, but what system instructions, tool metadata, and session context it can disclose.

This case shows how an authenticated assistant can become an information leak even without code exploitation or privilege escalation. Once the system prompt is exposed, an attacker gains a map of the agent’s tools, constraints, and session details, which is enough to target downstream actions with far less guesswork. That is a typical failure mode for early agent deployments, not an edge case.

Key questions

Q: How should security teams handle prompt injection in AI systems?

A: Treat prompt injection as an authorisation problem, not only a content problem. Validate user input, monitor for override patterns, and block any prompt that tries to change model policy, exfiltrate data, or trigger sensitive actions. The safest design is one where untrusted prompts cannot directly reach code execution, production data, or privileged tools without policy checks.

Q: Why do conversational AI systems create new identity and access risks?

A: Because they can combine data retrieval, decision-making, and execution in a single interaction. That collapses the gap between information access and business action, which traditional IAM and security tools were not built to manage. The result is higher exposure when the system can modify records or disclose sensitive guest data.

Q: What do organisations get wrong about AI guardrails?

A: Many teams assume a policy filter alone can prevent harmful output, but adversarial prompting shows that language models can be steered around obvious controls. The common mistake is treating guardrails as a static filter list instead of a system of content separation, monitoring, and authorisation boundaries.

Q: How can organisations reduce blast radius when an AI tool is compromised?

A: Limit the tool's scope, separate high-risk functions from general collaboration data, and make revocation fast enough to matter. Pair least privilege with short-lived tokens, clear ownership, and logging that links the agent, the user, and the downstream system. Containment only works when those paths are visible.

Technical breakdown

Why guardrails fail when the same request is reframed

Prompt-injection guardrails usually inspect the text for obvious malicious intent, but they are weak against socially plausible wording. If the model or wrapper classifies a request as legitimate research, documentation, or troubleshooting, it may pass through even when the underlying goal is to exfiltrate instructions or context. This is a classic semantic-evasion problem: the attack does not defeat the model, it defeats the policy layer that interprets the request. In agent systems, that means the boundary between user text and trusted instructions is thinner than many teams assume.

Practical implication: classify requests by intent and risk, not just keywords, before they reach the agent.

Why system prompt disclosure is an access-control failure

The system prompt is effectively the agent’s operating policy and tool map. When it leaks, the attacker learns which tools exist, how they are triggered, and what formatting or session constraints shape the conversation. That shifts the attack from guesswork to targeted abuse. In identity terms, this is a control-plane disclosure problem: the agent’s governing instructions become sensitive security material because they directly shape privilege use and downstream authorization decisions.

Practical implication: treat system prompts, tool manifests, and session metadata as sensitive configuration assets.

How output controls and least-privilege prompting reduce exposure

Input-side defenses are necessary but not sufficient because they miss reframed prompts and indirect leakage paths. Output-side filtering is the last line of defence for blocking system instructions, secrets-like material, and tool metadata from leaving the model. Least-privilege prompting reduces the blast radius by removing anything from the prompt that the agent does not strictly need to complete its task. For NHI and agentic AI governance, the goal is to make leakage less useful even when prevention fails.

Practical implication: minimise prompt contents and add output filtering for instruction, tool, and session disclosure.

NHI Mgmt Group analysis

AI prompt injection is now a governance problem, not just a model-safety problem. The bypass worked because the agent trusted a plausible register more than a malicious string, which means policy enforcement sits at the same risk layer as the model itself. Security teams need to treat reframing resistance, not keyword blocking, as the real test of an agent control. For practitioners, that means governance has to inspect how the agent reasons about intent, not only what it returns.

System prompt disclosure is a control-plane leak, not a harmless debug artifact. Once an attacker sees tool names, call conditions, and session metadata, they can aim directly at the agent’s trust boundaries. That is especially relevant for NHI governance because AI agents increasingly behave like identities with privileges, tool access, and session state. Practitioners should classify prompt content and agent metadata as sensitive identity material, not operational noise.

Least-privilege prompting is becoming a core NHI control for agentic AI. The fewer instructions, tool hints, and context fragments the agent carries, the less useful any disclosure becomes. This is a practical extension of zero standing privilege thinking into AI systems, where the safest prompt is the one that exposes the least governable surface. Teams should design prompts as if some portion will eventually leak.

Semantic filtering needs to sit alongside zero trust controls for AI workflows. Zero trust assumes every request is untrusted until verified, but agent deployments often stop at the input boundary and miss the output boundary. The lesson for identity programmes is that continuous verification must apply to what an agent is allowed to reveal, not only what it is allowed to do. Practitioners should fold agent output controls into the same governance model they use for privileged human and machine access.

Named concept: prompt reframing bypass. This is the specific failure mode where an attacker rephrases the same forbidden objective in a socially acceptable form and the guardrail reclassifies it as safe. It explains why static jailbreak lists age quickly and why testing must emulate adversarial reasoning. Practitioners should assume any rule that depends on phrasing alone will eventually be bypassed.

What this signals

Agentic AI is moving from experimentation into routine production use, but governance maturity is not keeping pace. The immediate signal for security teams is that agent visibility, output control, and delegated authority now need to be treated as a single governance problem rather than separate teams’ responsibilities. That matters most where assistants can call tools or access internal data.

Prompt reframing bypass: the control failure is not a broken blocklist, it is a policy layer that cannot reliably distinguish legitimate phrasing from malicious intent. Teams should assume that adversarial testing must become continuous, just as it is for phishing or application attack surfaces.

The broader programme implication is that AI agent review needs the same kind of lifecycle discipline that IAM brought to human and machine identities. If the agent can see tools, sessions, and configuration state, then those artefacts need ownership, review, and removal paths just like any other privileged asset.

For practitioners

• Implement semantic intent classification Screen agent prompts for objective and risk, not just banned words, so a harmless-looking rephrase does not slip past the guardrail.
• Restrict system prompt contents Remove tool inventories, call conditions, and session metadata from the prompt unless the agent truly needs them to complete the task.
• Add output-side disclosure filtering Block the agent from returning system instructions, tool maps, secret-like values, and conversation identifiers even when the input passes review.
• Test agents with reframing attacks Run repeated adversarial tests that change tone, context, and wording between attempts, because one blocked prompt does not prove the control holds.

Key takeaways

• AI agent guardrails can fail when attackers rephrase the same objective in a trustworthy tone, which makes intent classification more important than keyword blocking.
• System prompt disclosure gives attackers a direct map of tools, rules, and session context, turning a conversation leak into an access problem.
• Least-privilege prompting, output filtering, and repeated adversarial testing are the practical controls that reduce the blast radius when prevention fails.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads — causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• System Prompt: The system prompt is the core instruction set that defines an LLM’s behaviour, boundaries, and response style during a session. When attackers influence or override it, they are not merely changing text. They are trying to change the model’s governing authority.
• Least-privilege prompting: Least-privilege prompting is the practice of keeping an AI agent’s instructions, context, and tool hints as minimal as possible. The goal is to reduce the value of any leak and limit what the agent can be coerced into revealing or using during a session.
• Output-side filtering: Output-side filtering is the control layer that checks what an AI system is about to return before it reaches the user. It is used to block disclosure of sensitive instructions, metadata, or secret-like content that input controls may miss when the request is framed innocently.

What's in the full article

Escape's full research covers the operational detail this post intentionally leaves for the source:

• Step-by-step reconstruction of the prompt-injection bypass path and the exact pretext shift that defeated the guardrail
• Lab screenshots showing the synthetic system prompt, tool list, and session metadata disclosure in context
• Practical notes on how the testing approach reproduced the bypass against a comparable synthetic prompt
• Discussion of how the vendor interpreted the fix and what changed in the tested workflow

👉 The full Escape article covers the bypass sequence, leaked context, and the remediation discussion in more detail.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and agentic AI identity. It helps security practitioners connect access control, lifecycle discipline, and operational oversight across modern identity programmes.