Gemini 4’s vulnerability discovery outpaces exploit reliability

At a glance

What this is: This research evaluates Gemini 4 in controlled web application labs and finds that it is strongest at vulnerability discovery, while exploit generation remains less reliable because of soft refusals.

Why it matters: For IAM and security teams, the key implication is that agentic security tooling may surface findings faster than it can safely or consistently turn them into actionable exploitation or validation steps.

Context

Autonomous security assessment is the use of a model or agent to explore an attack surface, test hypotheses, and report findings with limited human guidance. The governance gap is not whether the system can assist a security workflow, but how reliably it can move from reconnaissance to reproducible action. That distinction matters when teams try to operationalise agentic testing, triage, or validation.

Tenzai’s evaluation places Gemini 4 inside six controlled web application labs, which makes this a structured test of model behaviour rather than a live attack claim. The important question for practitioners is how to govern model variance, refusal behaviour, and tool-use patterns when the system is being treated as a security operator rather than a simple assistant.

Key questions

Q: How should teams evaluate autonomous security models when discovery is stronger than exploitation?

A: Evaluate them in two stages. First, measure whether they can identify plausible weaknesses. Then test whether they can reproduce or validate those findings consistently. A model that is good at reconnaissance but weak at follow-through still needs human review, because discovery quality alone does not prove operational reliability.

Q: What does a high soft-refusal rate mean for agentic security workflows?

A: It means the model may stop or divert from a task even when it has enough context to continue. That reduces predictability and makes results harder to compare across runs. In security workflows, soft refusals are a governance signal, not just a usability issue, because they affect whether outcomes can be trusted.

Q: Why does tool-use behaviour matter in autonomous security assessment?

A: Tool-use behaviour shows how the model handles uncertainty, verification, and task sequencing. A file-heavy workflow may improve traceability and reporting, while heavier use of shells or code execution may support faster end-to-end action. Teams should review the pattern because it changes what the model is actually capable of doing.

Q: Should security teams trust a model that finds issues but refuses to exploit them?

A: Only for discovery support, not for autonomous validation. A model that surfaces weaknesses but declines exploit reproduction can still add value, but it does not remove the need for analyst judgment. Use it to accelerate triage and evidence gathering, then keep human oversight between the finding and any operational decision.

Technical breakdown

Why vulnerability discovery and exploit reproduction diverge

Vulnerability discovery and exploit reproduction are different phases of offensive work. Discovery is reconnaissance, where the model maps inputs, behaviours, and likely weaknesses from observable signals. Exploit reproduction requires a second step: turning a promising lead into a stable sequence of actions that reliably triggers the flaw. A model can be excellent at identifying candidates yet still be unreliable at the reproduction stage if it hesitates, changes its workflow, or refuses to complete the exploit path. That distinction is central in autonomous testing because recall at discovery does not prove operational exploitability.

Practical implication: Treat high discovery scores as an input to analyst review, not proof that the agent can validate or weaponise the finding on its own.

What soft refusals do to agent reliability

A soft refusal is when the model declines to continue a task as part of its normal behaviour, rather than returning a hard safety block. In offensive security workflows, that creates partial completion: the agent may identify a target, inspect artefacts, and then stop before the action that would confirm or reproduce the issue. This makes outcomes harder to compare across runs because the model is not failing consistently, it is changing its willingness to proceed. For practitioners, the operational issue is not only safety but predictability, because repeatability is essential when a tool is being used for validation or controlled testing.

Practical implication: Measure refusal rate alongside task success so you can distinguish capability limits from workflow interruption.

How tool-use patterns shape autonomous security assessment

Tool choice is part of the model’s operating style, not just a by-product of the task. Tenzai observed Gemini 4 leaning more heavily on file reads and less on Bash, browser, and code execution than the comparison models. That suggests a workflow built around inspecting saved artefacts and verifying evidence before reporting. In an assessment setting, this can improve traceability, but it can also slow down interactive exploitation or reduce the chance of completing multi-step actions. Tool preference therefore becomes a governance signal as much as a performance metric, because it reveals how the system approaches uncertainty and action sequencing.

Practical implication: Track tool-call mix when assessing autonomous models, because changes in workflow often explain differences in reliability better than scorecards alone.

Breaches seen in the wild

• Gemini AI Breach — Google Calendar Prompt Injection: Gemini AI assistant prompt injection attack leaks sensitive Google Calendar data.
• Gemini CLI prompt injection flaw 2025: Tracebit showed a poisoned README could make Gemini CLI run hidden commands and exfiltrate developer secrets; Google fixed it in 0.1.14.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Agentic security testing is already a governance problem, not just a model benchmark. When a system can discover weaknesses but may not reliably finish exploit reproduction, the security question shifts from raw capability to operational trust. That matters for red-team support, validation workflows, and any environment where an AI agent is expected to take structured action without drifting or stopping early. The control issue is predictable execution, not only detection quality, so practitioners should govern agent output as an evidence stream rather than an autonomous verdict.

Soft refusals create a new reliability boundary for AI-enabled security tooling. A model that refuses opportunistically can produce uneven results even when its underlying reasoning is strong. That introduces a governance gap between discovery and action, because the security team cannot assume that a promising lead will be reproducible on demand. The practical implication is to separate vulnerability identification from exploit validation, and to require explicit human review before any agentic workflow is trusted for operational decisions.

File-heavy workflows can improve traceability while reducing attack-path momentum. Gemini 4’s preference for reading and inspection suggests a more evidence-oriented operating style than the comparison models. That may help with report quality, but it also means the system is optimised differently from a tool that is trying to complete a rapid end-to-end exploitation chain. For security programmes, the lesson is that tool orchestration policy matters as much as model selection, especially when the output is used in web application attack analysis or validation contexts.

The future benchmark for cyber-capable models will be consistency, not peak ability. A model that is occasionally exceptional but often hesitant will be harder to operationalise than a slightly weaker model that behaves predictably. This is the key programme-level lesson for defenders adopting agentic security tooling: reliability controls, runbooks, and human checkpoints become part of the security architecture, not optional process overhead. The practitioner conclusion is to measure variance as seriously as top-line performance.

Read next: Web Application Security Testing Guide

What this signals

Variance will become the decisive metric for AI-enabled offensive tooling. Top-line capability matters less if the model cannot reproduce the same outcome across similar tasks. For practitioners, the meaningful question is whether the system’s behaviour is stable enough to support triage, validation, and reporting without constant human correction.

Agentic security programmes need a separation between finding and proving. Discovery-heavy systems can widen coverage, but they should not be treated as authoritative until they have passed a reproducibility check. That is the programme design change this research points to: trust the lead, not the first answer.

Tool orchestration is becoming part of the control plane. When an agent’s preferred workflow changes the balance between file review, shell use, and code execution, the organisation is no longer just evaluating a model. It is evaluating how that model will behave inside a security process, and that requires tighter oversight of web application testing methods.

For practitioners

• Define separate success criteria for discovery and validation Score autonomous security tests in two phases: finding plausible weaknesses and reproducing them reliably. Do not treat a strong reconnaissance result as proof that the model can complete an end-to-end exploit sequence.
• Track soft-refusal rate as a control metric Record how often the model stops, detours, or declines to continue during assessments, then compare that rate across tasks and model versions. Variance is a governance issue when the output feeds security decisions.
• Review tool-call mix before operational use Look at whether the system relies mainly on file reads, browser actions, shell execution, or code execution, because that pattern affects traceability, momentum, and the kind of evidence the agent can produce.
• Keep human review between finding and exploitation Require a human checkpoint before any agent-generated lead is used for validation, testing, or reporting. That is especially important when the model shows high discovery recall but inconsistent reproduction behaviour.

Key takeaways

• Gemini 4’s strongest result is in vulnerability discovery, but its exploit reproduction is less reliable because of higher soft-refusal behaviour.
• The core evidence is a 9 percentage point drop in recall on exploit reproduction, compared with negligible drops for the other tested models.
• Security teams should judge autonomous assessment tools on repeatability and task separation, not just on the quality of the first finding.

Related NHI Mgmt Group resources: Web Application Security Testing Guide · Web Application Attacks Guide · Zero-Day and Exploited Vulnerabilities Guide

Key terms

• Soft Refusal: A soft refusal is when a model declines to continue a task as part of its normal behaviour rather than returning a hard safety block. In agentic workflows, this creates partial completion and makes reliability harder to judge because the system may stop despite having enough context to proceed.
• Autonomous Security Testing: Autonomous security testing is the use of software that plans and runs security checks with limited human direction. It combines scanning, exploitation simulation, and validation of findings across applications, cloud, endpoints, and networks. In practice, it helps teams continuously test controls, detect weaknesses, and verify whether defenses still work after changes.
• Reproducible Exploit: A reproducible exploit is a repeatable sequence of actions that reliably demonstrates a vulnerability under controlled conditions. It matters because a finding is far more operationally useful when another analyst or tool can confirm the same result without relying on one-off behaviour or hidden assumptions.
• Tool-Use Pattern: A tool-use pattern is the distribution of actions an agent prefers, such as file reads, browser calls, shell commands, or code execution. This pattern reveals how the system reasons and verifies evidence, and it can materially change traceability, speed, and reliability in security workflows.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity controls to the broader security workflows that increasingly depend on AI-assisted decision-making.