Notifications
Clear all
Topic starter
11/08/2026 3:19 pm
TL;DR: Known vulnerabilities were missed at only 10% by GPT-5.5, down from 40% for GPT-5 and 18% for Opus 4.6, while also improving login speed and persistence decisions across real penetration-testing workflows, according to Xbow. The finding matters because AI-assisted testing is moving from isolated model evaluation to operational security work where failure modes and escalation paths now shape results.
NHIMG editorial — based on content published by Xbow: GPT-5.5: Mythos-Like Hacking, Open to All
By the numbers:
- GPT-5 missed 40% of vulnerabilities.
- Opus 4.6 reduced that to 18%.
- GPT-5.5 brings it down further to just 10%.
Questions worth separating out
Q: What breaks when AI security tools rely on model benchmarks alone?
A: Benchmark-only decisions break when teams assume capability scores translate directly into operational security.
Q: Why do AI agents complicate access governance more than ordinary automation?
A: AI agents complicate access governance because they can branch at runtime, wait on external services, and continue later with the same operational context.
Q: How do teams know if AI-assisted pentesting is actually working?
A: Look for higher-quality findings, faster triage, and fewer unresolved false positives, not just more output.
Practitioner guidance
- Define agent execution boundaries Limit which systems AI-assisted security workflows can touch, what credentials they may use, and when they must stop.
- Instrument login and retry behaviour Log every login attempt, failure, pivot, and retry so you can see whether the agent is wasting cycles or discovering a valid path.
- Treat source code access as amplification Assume that code visibility will improve AI-assisted discovery rather than simply validating what you already know.
What's in the full article
Xbow's full post covers the operational detail this post intentionally leaves for the source:
- Benchmark design details, including how vulnerable applications are frozen and replayed across model runs.
- Comparative workflow observations for GPT-5, Opus 4.6, and GPT-5.5 across black box and white box tasks.
- Logging-in and failure-handling behaviour that explains why the model feels faster in practice.
- How XBOW uses different models in different parts of its pentesting stack.
👉 Read Xbow's analysis of GPT-5.5 performance in offensive security workflows →
GPT-5.5 and offensive security workflows: what changed for teams?
Explore further
11/08/2026 3:31 pm
AI-assisted offensive testing is becoming a governance problem, not just a tooling problem. Once a model is embedded in a workflow that can authenticate, navigate, and continue across multiple steps, it becomes a governed operational component. That changes the control conversation from model accuracy alone to scoping, logging, privilege, and task termination boundaries. Practitioners should treat these systems like other high-trust non-human workflows, with clear ownership and review.
A question worth separating out:
Q: What should security teams do when AI tools can log into live systems?
A: They should place those tools under the same access governance used for other privileged non-human workflows. That means task-scoped credentials, explicit target allowlists, logging, and termination rules when the tool drifts from the intended path. Live access should never be broader than the test objective requires.
👉 Read our full editorial: GPT-5.5 raises the bar for offensive AI security workflows
