What breaks when untrusted issue or pull request content is inserted directly into an AI prompt?

The core control breaks are prompt integrity and action trust. Malicious text can be interpreted as instructions, not data, which can redirect the agent toward editing issues, running commands, or publishing sensitive values. Sanitisation helps, but the safer pattern is to keep untrusted content out of prompts whenever possible and constrain the tools the agent can invoke.

Why This Matters for Security Teams

When untrusted issue or pull request content is copied into a prompt, the model no longer sees a clean separation between instructions and data. That turns ordinary collaboration text into a potential control surface for prompt injection, tool misuse, and unintended disclosure. The risk is not limited to clever wording. It also includes hidden instructions, copied secrets, and malicious content designed to steer the agent toward a harmful next step.

This matters because the security failure is often downstream, not immediate. An LLM may summarize, classify, open links, or trigger actions based on content that was never meant to be operational input. The safer framing is to treat repository content like any other untrusted input and enforce prompt boundaries, tool restrictions, and explicit approval points, consistent with guidance in the NIST Cybersecurity Framework 2.0 and NHIMG research on AI-exposed secrets in The State of Secrets in AppSec.

In practice, many security teams discover the weakness only after a bot has already quoted, forwarded, or acted on content that was supposed to be inert review text.

How It Works in Practice

The core problem is that prompt construction often collapses three different trust levels into one string: system instructions, trusted task context, and untrusted repository content. If issue text or pull request comments are inserted verbatim, the model can misread embedded directives such as “ignore prior instructions,” “call this webhook,” or “paste the last token you saw.” Even when the model does not comply perfectly, the presence of the instruction can distort output and increase the chance that downstream tooling follows the wrong branch.

Current guidance suggests using a layered design instead of raw concatenation. Keep untrusted content outside the instruction channel, label it explicitly as data, and pass only the minimum necessary excerpt into the model. Where the agent can take action, constrain tool access and require runtime checks before each high-impact step. That includes allowlisted actions, scoped credentials, and human review for destructive operations. For repositories and pull request automation, the prompt should not be the place where trust is established. Trust should come from policy, identity, and task-specific limits, not from the text itself.

Practical controls usually include:

• Separating system prompts from issue or PR content and never blending them into one free-form block.
• Filtering or redacting secrets before any text reaches the model, especially tokens, keys, and session values.
• Using tool gating so the agent can only read, summarize, or suggest unless a higher-risk action is explicitly approved.
• Logging the exact content supplied to the model so teams can reconstruct prompt injection attempts during review.

This pattern aligns with NHIMG findings on secret exposure in code workflows and the kinds of failures described in the DeepSeek breach, where embedded sensitive data and exposed records compounded the blast radius. These controls tend to break down when teams let the model directly operate on long, messy pull request threads because the relevant instruction and the attacker’s payload become indistinguishable.

Common Variations and Edge Cases

Tighter prompt filtering often increases friction for developers, requiring organisations to balance faster automation against stronger content handling and review. The tradeoff is especially visible in large pull requests, security triage bots, and code review assistants that rely on full context to be useful.

There is no universal standard for this yet, but best practice is evolving toward least-privilege prompt design: only the minimum issue or PR text needed for the task, plus explicit separation between user content and agent instructions. In some workflows, summarisation is safe enough if the agent never gets tool access. In others, even read-only prompting is risky because the model may reproduce secrets, follow hidden instructions, or generate a false sense of confidence. That is why sanitisation should be treated as a backstop, not the primary defence.

Edge cases include markdown comments that look benign but contain encoded instructions, copy-pasted shell snippets that are actually execution payloads, and automated PR reviewers that can be nudged into writing harmful follow-up comments. If the agent can edit repositories, open tickets, or trigger CI, the trust boundary must extend beyond the prompt itself and into the action layer. Current guidance suggests treating all externally supplied repository text as hostile until it has been validated, minimized, and isolated from execution paths.

Related resources from NHI Mgmt Group

• What breaks when untrusted pull request content is executed in a workflow?
• What breaks when AI agents can call tools after reading untrusted content?
• What breaks when an AI assistant can access private data and untrusted content at the same time?
• What breaks when AI code review tools are allowed to analyse untrusted pull requests?