The exploit race in web3 is turning security into automation

At a glance

What this is: This is Quantstamp’s analysis of how automated exploitation in web3 is changing both attacker economics and defender workflows.

Why it matters: It matters to security and identity practitioners because agentic systems, trust boundaries, and tool governance now affect how teams validate code, scope access, and avoid over-relying on AI output.

By the numbers:

• In 2025 alone, an estimated $3.4B was stolen through crypto exploits.
• Prompt design alone can reduce false positives by over 60% in controlled evaluations.
• Crypto lost more than $240 million across 29 incidents in July 2026, up roughly 216% from June.

👉 Read Quantstamp's analysis of the exploit race in web3 security

Context

Web3 security is defined by direct financial incentive, which makes exploitation faster to industrialise than in many conventional software environments. When attackers can test at scale and only execute a proven path, defenders need workflows that are repeatable, measurable, and difficult to game. That is why this article sits at the intersection of application security, automation, and the governance of AI-driven security tools.

The identity angle is real even though the article is not primarily about IAM. Security agents depend on scoped context, tool permissions, and reproducible decision-making, which makes them a governance problem as much as a technical one. For identity and security teams, the lesson is that AI assistance only becomes useful when its access, outputs, and verification steps are controlled like any other high-risk system.

This is a typical pattern for emerging security automation: the tool becomes valuable only after teams stop treating it as a chatbot and start treating it as part of a controlled operational workflow.

Key questions

Q: How should security teams use AI agents for vulnerability discovery without over-trusting them?

A: Treat AI agents as a repeatable screening layer, not as proof of security. Use targeted prompts, a fixed workflow, and a verification step that requires traces, tests, or reproduction before a finding is accepted. Human reviewers should handle ambiguous cases, exploitability judgments, and design-level reasoning.

Q: Why do AI security agents need benchmark-driven governance?

A: Because performance varies by prompt design, context handling, and the evaluator itself. Without benchmark discipline, teams can mistake pattern matches for real assurance and buy tools that look better than they are. Reproducible metrics are the only way to compare precision, recall, and operational noise.

Q: What breaks when vulnerability detection is done with raw prompting alone?

A: Raw prompting produces unstable results, inconsistent false positives, and invisible false negatives. The model is forced to choose scope, evidence, and stopping criteria at once, so output quality changes with small wording shifts. That makes it unsuitable as a security control unless wrapped in a structured workflow.

Q: How accountable are teams for AI-assisted security findings?

A: Teams remain accountable for the evidence behind any AI-assisted finding, just as they are for any other security control. If an agent cannot reproduce a claim, explain its basis, or show the relevant code path, the organisation should treat the output as advisory only and not as a decision record.

Technical breakdown

Why raw prompting fails in vulnerability discovery

A single prompt forces the model to decide the scope, the hypothesis, the evidence threshold, and the stopping point all at once. In vulnerability work, that creates unstable results because small changes in wording or context can swing outputs from no issues to critical findings. In practice, broad prompts are less reliable than targeted checks, and the model often produces findings that sound plausible but are not exploitable. The core technical problem is not intelligence alone, but lack of process discipline around search, validation, and reproducibility.

Practical implication: replace open-ended prompts with targeted checks and a standard evaluation rubric.

How multi-pass agent workflows create usable assurance

An agent becomes more than a prompt when it separates discovery, verification, and reporting into distinct stages. One pass can surface suspicious patterns, another can test preconditions or call paths, and a final pass can validate whether the issue is real. This reduces false confidence because the system must justify claims instead of stopping at pattern recognition. The same logic applies to retrieval, memory, and tool use: each pass should narrow the search space and preserve only the context needed for the next decision.

Practical implication: design workflows that force verification before a finding reaches developers.

Why benchmarking agent output matters as much as model choice

The article makes clear that agent performance depends on workflow design, protocol complexity, and evaluator quality. If a benchmark treats a finding as correct simply because it resembles ground truth, scores become inflated and comparisons stop being meaningful. That matters because the choice is not just between models, but between precision-oriented and recall-oriented workflows. For security teams, the technical lesson is to measure false positives, false negatives, and reproducibility against the actual code paths that matter, not against loose pattern matches.

Practical implication: benchmark agents on reproducible evidence, not on similarity scores or vendor claims.

Threat narrative

Attacker objective: The attacker objective is to identify a provable exploit path quickly enough to extract value before defenders can patch or intervene.

1. Entry begins with automated vulnerability search, where agents can test large numbers of code paths and contract states at falling cost.
2. Escalation occurs when attackers move from suspicion to proof, using simulations and local state variations to identify a proven exploit path.
3. Impact is direct financial theft on-chain, because successful exploitation converts code flaws into immediate monetary loss.

NHI Mgmt Group analysis

Agentic security only becomes credible when it is governed like infrastructure. The article is right to reject raw prompting as a control model, because ad hoc AI output is not repeatable enough for security operations. In identity terms, this is the same lesson enterprises learned with service accounts and secrets: uncontrolled runtime authority creates ambiguity and risk. Practitioners should treat agent workflows as managed systems with defined scope, validation, and review.

Benchmark inflation is now a governance problem, not just a tooling problem. If evaluators reward near-misses or pattern similarity, organisations will buy confidence they do not actually have. That creates the same kind of assurance gap seen in weak access reviews, where process exists but evidence quality does not. The field needs evidence-based comparability, not leaderboard theatre, and teams should demand reproducible results before adopting AI security tools.

Prompt discipline is a named control surface, not a usability detail. The article shows that wording, context, and sequence materially change outcomes, which means prompt design functions like a security policy layer. That makes it a close cousin to policy-as-code in identity and cloud governance. Teams that ignore prompt structure will get inconsistent findings and unstable operational trust.

AI security agents will not replace human auditors, but they will redefine where human judgment is spent. The article’s strongest point is that machines can scale pattern discovery while people remain responsible for adversarial reasoning, economic edge cases, and spec-versus-implementation mismatches. That division of labour mirrors mature IAM and PAM programmes, where automation handles volume and humans handle exception and governance. Practitioners should build around that split rather than pretend one layer can do both jobs.

Security tooling that can act must also be constrained like any other privileged system. Once an agent can inspect code, call tools, and iterate on hypotheses, it inherits the same governance concerns as other non-human identities. That means scope, approval boundaries, and verification logs matter as much as model quality. Teams should govern agent permissions with the same seriousness they apply to high-risk service accounts.

What this signals

Prompt discipline is becoming a governance boundary. As AI security tools move from experimentation into operational use, the most important control is not model choice but repeatability. That is why identity-style discipline around scope, logging, and permissioning matters for agentic tooling as much as it does for service accounts and other non-human identities.

The practical signal for programme owners is that AI security output should be managed like any other privileged workflow. The NHI Lifecycle Management Guide is a useful reference point because the same failure pattern appears whenever access is broad, unreviewed, or difficult to revoke. If the agent can act, the organisation must know who owns it, what it can touch, and how its authority is withdrawn.

The broader market signal is that security teams will increasingly ask for evidence of verification, not claims of intelligence. That aligns with frameworks such as the NIST Cybersecurity Framework 2.0 and the MITRE ATT&CK Enterprise Matrix, where controls are judged by observable outcomes, not by automation alone.

For practitioners

• Standardise vulnerability prompts Use a fixed set of targeted prompts for common issue classes, then compare outputs across runs to identify noise, drift, and hidden false negatives. This is the fastest way to turn a chatbot into a repeatable control.
• Separate discovery from verification Run one pass to flag candidate issues and a second pass to prove exploitability with traces, tests, or controlled reproduction. Findings that cannot survive verification should remain unconfirmed, not queued for remediation.
• Benchmark against reproducible evidence Score agents on whether they identify the correct code path, preconditions, and failure mode, not just whether they resemble a known finding. Keep benchmark datasets versioned so you can measure drift over time.
• Restrict agent tool access Limit what the agent can read, execute, and export, especially when code, secrets, or unreleased design details are involved. Treat agent permissions as a privileged workflow with explicit scoping and logging.
• Use human triage for edge cases Reserve human review for economic exploits, design flaws, and spec-versus-implementation mismatches that automated checks commonly miss. That preserves analyst time for the highest-value judgments.

Key takeaways

• The article argues that automated exploitation changes the economics of web3 risk by turning vulnerability discovery into a scalable attack process.
• AI security agents only improve assurance when they are benchmarked, verified, and governed as repeatable workflows rather than as chat interfaces.
• For practitioners, the main shift is to manage agent permissions, evaluation quality, and human triage with the same discipline used for privileged systems.

Key terms

• Security Agent: A security agent is an AI-driven workflow that can select checks, gather context, use tools, and validate results across multiple passes. It is not just a model prompt, because the useful unit of control is the repeatable process around the model's actions and outputs.
• Benchmark Inflation: Benchmark inflation happens when evaluation methods reward findings that are similar to ground truth but not actually correct in location, exploit path, or preconditions. It creates a misleading picture of capability and can make tools appear more reliable than they are in real operations.
• Prompt Discipline: Prompt discipline is the deliberate structuring of questions, context, and stopping criteria so that model output is repeatable and comparable. In security workflows, it functions like a policy layer, because small changes in wording can materially change whether a finding appears or disappears.

What's in the full article

Quantstamp's full blog post covers the operational detail this post intentionally leaves for the source:

• Benchmarking approaches for AI security agents in realistic smart-contract environments
• The specific workflow levers Quantstamp used to compare precision, recall, and verification quality
• Examples of multi-pass agent design across prompting, context handling, and tooling
• How Quantstamp thinks teams should integrate AI security checks into development cycles

👉 Quantstamp's full post covers the agent workflow details, benchmarking logic, and developer guidance in depth.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and identity lifecycle controls. It is designed for practitioners who need to govern high-risk non-human access across modern security programmes.