Burp AI shows how prompt structure changes PoC speed and accuracy

At a glance

What this is: PortSwigger found that prompt structure materially changes Burp AI’s speed, request count, and accuracy when generating functional proof-of-concepts.

Why it matters: Pentesters and security teams evaluating AI-assisted testing need to understand that prompt quality can shape both cost and reliability, especially when validating web vulnerabilities.

By the numbers:

• SSTI was resolved in 44 seconds with the free-form prompt style.

Context

Burp AI is being used here as a testing assistant inside a controlled pentest workflow, so the governance question is not whether AI can think, but how much context it needs to stay useful. In practice, AI-assisted security testing still depends on the quality of the task framing, the target selection, and the stop conditions the tester provides.

The article sits at the intersection of application security and AI-assisted workflow design. For IAM and NHI practitioners, the relevance is indirect but real: as security teams adopt AI copilots, the same issues of context, scoping, and guardrails will shape how effectively those tools support identity testing, access review, and exploit validation.

Key questions

Q: How should security teams prompt AI assistants for web vulnerability testing?

A: Use a short objective, the exact target element, the suspected flaw, and a stop condition. The best prompts keep the assistant focused on live HTTP evidence instead of forcing it to infer too much from templates or overly rigid structure. That reduces wasted requests, limits hallucinations, and improves the chance of a usable proof-of-concept.

Q: Why do rigid prompt templates sometimes perform worse than free-form prompts?

A: Rigid templates can add friction between the analyst’s intent and the model’s next action. When the model spends effort following structure rather than interpreting the live application behaviour, it may take more requests, drift outside guidance, or invent unnecessary steps. Free-form prompts often preserve the security context more effectively.

Q: What signs show that an AI testing assistant is losing context?

A: Common signs include extra requests after confirmation, payloads that do not match the observed response, and output that branches into irrelevant attacks. If the assistant starts proposing content outside the stated guidance, the prompt has likely become too broad or too prescriptive for the task.

Q: How do teams control risk when using AI for exploitation testing?

A: They control it by defining scope, limiting the target surface, and using an explicit stop rule once the test goal is reached. That keeps the assistant within the engagement boundary and prevents it from continuing to probe after the relevant evidence has already been obtained.

Technical breakdown

How prompt structure changes Burp AI behaviour

Burp AI is sensitive to the shape of the instruction it receives because the model is being used to drive iterative security testing, not to answer a static question. Free-form prompts gave the tool a clearer path from objective to validation criteria, while heavily structured JSON or templated prompts added friction and, in some cases, encouraged extra steps that were not needed. In this setting, the model’s usefulness depends less on syntactic regularity and more on whether the prompt preserves the tester’s intent, the target parameter, and the stopping rule. Practical implication: optimise for task clarity, not schema rigidity.

Practical implication: keep the instruction concise enough that the tool can preserve intent without inventing extra work.

Why hallucinations increase when context is over-specified

The article shows a familiar LLM pattern: when the prompt carries too much structure or redundant detail, the model can lose focus and begin generating content outside the intended test path. In security testing, that means the assistant may continue probing after confirmation, propose payloads that do not fit the observed response, or drift into irrelevant branches. The issue is not that structure is always bad, but that excessive scaffolding can compete with the actual evidence in the HTTP exchange. Practical implication: treat context as a control surface, and tune it so the live request/response evidence remains dominant.

Practical implication: reduce prompt clutter when the assistant starts drifting from the observed application behaviour.

How human guidance acts as the control layer

Burp AI performed best when the pentester supplied a focused objective, a clear target element, and an explicit stop condition. That combination functioned as the effective control layer, keeping the assistant aligned with the engagement boundary and preventing unnecessary requests. This is a useful pattern for any AI-assisted security workflow: the model can accelerate analysis, but the human still defines the scope, the success criteria, and the point at which exploration should stop. Practical implication: design prompts as operating instructions, not just as feature requests.

Practical implication: put scope and stop conditions into every AI-assisted test to constrain over-testing.

NHI Mgmt Group analysis

Prompt quality has become a security control, not a convenience feature. In AI-assisted testing, the way a task is framed materially changes whether the assistant stays aligned with the evidence or drifts into extra probes and hallucinations. That makes prompt design part of the control plane for security work, especially where request count, cost, and stop conditions matter. Practitioners should treat instruction quality as an operational safeguard, not an optional productivity tweak.

Free-form guidance outperformed rigid structure because it preserved attack context. The strongest result in the article was not a more elaborate schema, but a prompt that kept the objective, target, and stop rule visible to the model. That suggests the model is better at handling semantically clear context than at following heavy formatting when the task is iterative exploitation. Security teams should optimise for the model’s ability to remain evidence-driven.

Burp AI illustrates a broader AI governance lesson for security teams. AI copilots do not remove the need for expert judgment, they amplify whichever assumptions the operator encodes into the prompt. In identity and access workflows, that same pattern will apply when teams use AI to triage anomalies, test access paths, or review sensitive entitlements. The practical conclusion is to govern AI assistance as a bounded workflow, not as an autonomous tester.

Context discipline is the real differentiator in AI-assisted offensive testing. The article’s examples show that a short, highly specific brief can outperform richer templates because it keeps the model anchored to the live scenario. That mirrors a wider governance problem across agentic systems: the more authority a system gets to explore, the more important it becomes to define scope, stop rules, and evidence thresholds. Practitioners should build those constraints first, then automate.

From our research library:

• Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
• Read next: Top 10 NHI Issues

What this signals

Context is the governing variable in AI-assisted pentesting. The article shows that the same model can behave like a disciplined assistant or a noisy one depending on how tightly the operator defines the target, the scenario, and the stop condition. That should push teams to treat prompt quality as part of their testing methodology, not as an afterthought.

Prompt calibration is now part of operational AI governance. As more security workflows adopt assistants, the question is no longer whether the model can generate output, but whether the workflow can preserve intent, evidence, and control boundaries. For practitioners, the useful lesson is to design prompts the way they design access: narrowly, explicitly, and with clear termination criteria.

For practitioners

• Define a concise test objective State the target parameter, suspected vulnerability, and confirmation criteria in one focused instruction so the AI can stay on-task.
• Provide just enough system context Include the system summary, expected flow, and prior observations, but avoid adding redundant structure that can pull the model away from live evidence.
• Use highlighted inputs to anchor analysis Mark the exact parameter, cookie, or path segment you want the assistant to inspect so it does not generalise beyond the intended attack surface.
• Set an explicit stop condition Tell the assistant to stop once a vulnerability is confirmed so it does not continue testing after the proof point has already been reached.

Key takeaways

• Prompt design materially affects whether an AI-assisted tester reaches a valid proof-of-concept quickly or burns requests on irrelevant exploration.
• The article’s tests show that free-form, evidence-led prompts can outperform more rigid structures when the goal is to confirm a web vulnerability.
• Security teams should govern AI-assisted testing with clear scope, explicit stop conditions, and enough context to keep the model anchored to live application behaviour.

Key terms

• Threshold Calibration: Threshold calibration is the process of setting alert floors and sensitivity levels so a monitoring system flags risk at the right time. In digital asset compliance, weak calibration often appears when indirect flows are tolerated at much higher values than direct flows.
• Proof Of Concept Exploit: A proof of concept exploit is working code or request logic that demonstrates a vulnerability can be triggered in practice. It does not always equal mass exploitation, but once released publicly it often lowers the barrier for attackers, speeds up scanning, and increases the urgency of patching and monitoring.
• Hallucination: An AI-generated response that is fluent and plausible but incorrect, unsupported, or fabricated. For identity and governance teams, hallucination is a control issue because users may act on it as if it were trusted system output, especially when the chatbot sits inside an operational workflow.
• Stop Condition: A stop condition is the explicit rule that tells a tool to halt once a defined result has been reached. In AI-assisted testing, it prevents over-execution, reduces noise, and limits cost. It is a governance control as much as an operational instruction because it constrains continued action after confirmation.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity controls to the wider security workflows that AI tools increasingly support.