Notifications
Clear all
Topic starter
01/08/2026 10:49 pm
TL;DR: Free-form prompts produced the fastest and most reliable functional PoCs in Burp AI tests, with SSTI resolved in 44 seconds and insecure deserialization in 6 requests, while more structured prompt formats consumed more requests and introduced hallucinations, according to PortSwigger. The practical lesson is that AI-assisted testing still depends on human context engineering, not just tool capability.
NHIMG editorial — based on content published by PortSwigger: Functional PoCs in less than a minute? Julen Garrido Estévez puts Burp AI to the test
By the numbers:
- When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes - and as quickly as 9 minutes in some cases.
Questions worth separating out
Q: How should security teams structure AI-assisted testing prompts to get reliable results?
A: Use a short, repeatable prompt that names the objective, the specific parameter or object under test, the expected vulnerability class, and the stop condition.
Q: Why do AI tools drift when prompts contain too much or too little context?
A: Too little context forces the model to guess, which increases generic payloads and wasted requests.
Q: What breaks when stop conditions are not built into AI security workflows?
A: The model keeps testing after it has already confirmed the issue, which increases cost, adds noise, and can create unnecessary follow-up traffic.
Practitioner guidance
- Standardise prompt templates for security automation Define a minimum prompt pattern with objective, target element, expected outcome, and stop condition so AI-assisted testing stays reproducible and bounded.
- Separate system context from test scenario Keep broader application context in a short note block and put the exact parameter, header, or cookie under test in a single scenario line.
- Enforce stop-on-confirmation rules Require the workflow to stop immediately after a confirmed vulnerability, then log the request sequence and evidence for manual review.
What's in the full article
PortSwigger's full guest post covers the operational detail this post intentionally leaves for the source:
- The exact prompt templates used for free-form, structured, and JSON-driven testing across SSTI and deserialisation cases.
- The request counts, timings, and estimated credit consumption behind each test run, which are useful if you need to benchmark AI-assisted workflows.
- The step-by-step observations from the Web Security Academy labs, including how the tool behaved once it confirmed a vulnerability.
- The author's notes on context, highlights, and stop conditions, which are the most implementation-relevant parts for practitioners.
👉 Read PortSwigger's analysis of how Burp AI performs in PoC generation →
Burp AI prompt calibration: what it means for pentest workflows?
Explore further
01/08/2026 10:57 pm
Prompt quality is becoming a security control, not just a usability detail. This article shows that the model’s effectiveness depends on how precisely the practitioner constrains context, scenario, and stopping conditions. In practice, that means AI-assisted testing behaves less like a black box and more like a governed workflow with control points. For identity and access teams, the parallel is obvious: if delegated runtime decisions are not scoped, the system expands its own attack surface.
A question worth separating out:
Q: How should teams decide whether AI-assisted PoC generation is safe to use in production testing?
A: Use it only when the workflow is bounded, logged, and human-reviewed. If the task can change privilege state, touch authenticated sessions, or generate exploit payloads, a clear approval and review step is necessary before execution. Speed is useful only when the guardrails are explicit.
👉 Read our full editorial: Burp AI shows how prompt structure changes PoC speed and accuracy
