Training vulnerability discovery agents with long-horizon RL

At a glance

What this is: depthfirst describes dfs-large1 as a long-horizon vulnerability discovery agent that improves audit performance by pairing bounded repository scopes with executable verification and report discipline.

Why it matters: That matters because auditors and security teams need agentic tools that can prove reachability and exploitability, not just generate plausible findings from local code context.

Context

Long-horizon vulnerability discovery is the problem of proving that a suspected weakness is actually reachable, exploitable, and worth reporting across a large codebase. In practice, single-pass code review often misses upstream validation, shared libraries, and cross-repository dependencies that change whether a finding is real. For IAM and NHI-adjacent teams, the same control challenge appears when machine identities, secrets, or service-to-service trust paths are only visible once the broader system context is assembled.

Depth-first’s article frames this as a governance problem for security agents as much as a model problem: the agent must stay inside a bounded scope, retain enough surrounding context to reason accurately, and validate evidence before it reports. That is a useful lens for agentic AI security because the weakest point is often not model capability alone, but the control environment around tool use, context, and output acceptance.

Key questions

Q: What goes wrong when an audit agent reports vulnerabilities without executable verification?

A: The agent can produce locally plausible findings that fail under broader analysis because the weakness is not actually reachable or exploitable. That creates false positives, wastes triage time, and can hide the real issues that do survive execution. In practice, verification is the control that separates useful audit output from speculative pattern matching.

Q: Why do bounded repository scopes improve long-horizon vulnerability discovery?

A: They reduce search noise while keeping enough surrounding context to trace callers, dependencies, and trust boundaries. That matters because many vulnerabilities are not local to one file and cannot be judged correctly from a snippet. The scope boundary helps the agent focus without losing the evidence chain needed for reachability analysis.

Q: What are the signs that a vulnerability discovery agent is over-reporting?

A: A common sign is a rising count of plausible findings that do not survive runtime checks or broader dependency tracing. Another is long exploratory trajectories that keep expanding the candidate set without producing stronger evidence. If the tool is optimised for volume rather than validation, it will look productive while lowering audit integrity.

Q: How should teams govern context compression in agentic security workflows?

A: Treat it as part of the control boundary, not a convenience feature. Summaries should preserve the reasoning needed to justify a finding, and the original evidence should remain available for review. If compressed context cannot be audited, the team should not rely on it as the basis for a security conclusion.

Technical breakdown

Why bounded audit scopes matter for repository-scale analysis

Large repositories create a reachability problem. A model that sees only local snippets can misread whether an input is attacker-controlled, whether a sink is reachable, or whether upstream validation neutralises the risk. Bounded scopes solve this by constraining each agent’s initial investigation surface while still allowing the shell to inspect broader repository context when tracing dependencies, callers, and shared libraries. That pattern preserves focus without breaking the causal chain needed for accurate vulnerability analysis. In practice, the architecture is a compromise between search breadth and proof quality: enough context to reason, enough bounds to keep work tractable.

Practical implication: Use scoped investigation surfaces so agents can reason across trust boundaries without wandering the entire codebase.

Why executable verification changes vulnerability discovery quality

Executable verification turns a candidate finding into evidence. In systems code, reproducers, fuzz targets, and sanitiser output can confirm that a path is actually reachable and memory corruption is real. In web applications, validation is harder because the exploit path may depend on application state or external services, but the principle remains the same: findings should survive an execution test or a broader reachability check. This shifts agent behaviour away from pattern matching and toward hypothesis testing. It also reduces the risk of speculative reports that sound plausible but fail under a deeper audit.

Practical implication: Require agents to validate high-value findings with runtime evidence before treating them as actionable.

How reward shaping steers long-horizon agent behaviour

The article’s reinforcement learning recipe uses two behavioural constraints: an effort penalty and a finding budget. The effort penalty discourages wasteful token use, repeated browsing, and long undirected trajectories, while the finding budget reduces over-reporting when true vulnerabilities are sparse. Together they shape the policy toward targeted investigation and away from speculative enumeration. That matters because long-horizon security work is not just a reasoning problem. It is also a control problem where the training objective determines whether the agent optimises for useful evidence or for the appearance of productivity.

Practical implication: Align the reward or scoring function with validated findings, not with report volume or exploratory churn.

Threat narrative

Attacker objective: The practical objective is to produce high-confidence vulnerability findings that survive broader analysis and execution, rather than speculative reports that look credible in isolation.

1. Entry begins when the agent receives a bounded repository scope and a scoped threat model that defines the audit surface and the relevant trust boundaries.
2. Credential access is not the issue here; instead, the key step is evidence acquisition, where the agent traces callers, dependencies, and shared libraries to determine whether a suspected weakness is reachable.
3. Impact occurs when the agent converts the hypothesis into executable proof, separating real vulnerabilities from locally plausible but unreachable issues.

Breaches seen in the wild

• Sisense breach 2024: A credential in Sisense's GitLab reportedly opened S3 buckets of customer tokens, passwords and certificates; CISA urged a full reset.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Verified reachability is becoming the real control plane for agentic vulnerability discovery. The article shows that repository-local reasoning is not enough once evidence is distributed across files, callers, and related projects. For security teams, the governance question is whether an agent can prove a finding, not whether it can describe one. That shifts the centre of gravity from generation quality to evidence quality.

Long-horizon agent work exposes a control gap between hypothesis generation and report acceptance. A model can propose a plausible weakness long before it can demonstrate exploitability, and that gap is where false positives accumulate. The named failure mode here is speculative finding inflation, which appears when the scoring function rewards breadth or volume more than validated analysis. Practitioners should treat that as an audit integrity issue, not just a tuning issue.

Context compaction is an identity and governance problem for agents, not just a memory optimisation trick. When the same policy summarises its own trajectory and then continues from that summary, the summarisation step becomes part of the security boundary. That matters for agentic AI governance because compressed context can quietly erase the evidence trail that should support a finding. Teams should govern what state an agent may carry forward and what must remain auditable.

Research agents in security will increasingly be judged on proof discipline, not benchmark fluency. The article’s emphasis on executable validation and controlled scope is a signal that enterprise buyers should ask how a tool handles false positives, reachability, and multi-step evidence. In practice, that is the difference between a useful audit assistant and a noisy report generator.

AI agent identity matters here because tool use and trajectory ownership are part of the control story. Once an agent is allowed to inspect repositories, invoke shells, and preserve intermediate context, its identity is operationally meaningful even if it is not autonomous. Governance should therefore treat the agent’s permissions, scope, and output authority as first-class controls. That is the relevant boundary for agentic security programmes.

From our research library:

• Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
• Read next: NHI Lifecycle Management Guide

What this signals

Proof discipline will become the differentiator for agentic security tools. As these systems scale into large enterprise repositories, the valuable capability is no longer broad enumeration but the ability to carry a hypothesis through to executable evidence. Teams that evaluate agentic tools should therefore test validation quality, not just report volume or apparent recall.

Search, tracing, and runtime evidence need to be governed as one workflow. The article’s architecture shows that context boundaries and tool use are inseparable from the credibility of the final finding. That is a useful pattern for security programmes: if the evidence chain breaks anywhere in the workflow, the conclusion should not be treated as trustworthy.

For practitioners

• Define bounded investigation scopes for audit agents Limit each run to a targeted file set and explicit threat model, but preserve the ability to inspect shared libraries, callers, and dependencies when reachability needs broader context.
• Require executable proof for high-value findings Treat reproducers, sanitiser output, or other runtime evidence as the acceptance gate for claims that affect remediation priority or reporting confidence.
• Tune scoring to penalise speculative output Apply a report budget and an effort penalty so the agent is rewarded for validated findings and not for simply increasing the number of reported issues.
• Preserve audit trails through context compression If trajectory summaries are used, store the underlying evidence separately so summarisation does not erase the chain of reasoning needed for review.

Key takeaways

• The article shows that long-horizon vulnerability discovery improves when agents can test reachability across repository boundaries instead of relying on local code fragments.
• Its training recipe rewards validated findings over speculative output, which makes proof quality a more important performance signal than raw report count.
• For practitioners, the key control is to pair bounded context with executable verification so agentic audit workflows produce evidence that survives review.

Key terms

• Long-Horizon Vulnerability Discovery: Long-horizon vulnerability discovery is the process of tracing a suspected flaw across many steps, files, or repositories until the issue is either proven reachable or rejected. It matters because real security evidence is often distributed, and short-context analysis can miss the connections that determine exploitability.
• Executable Verification: Executable verification means testing a candidate vulnerability against runtime behaviour rather than relying only on static reasoning. It can include sanitizers, reproducer scripts, or harnesses that confirm the issue is reachable and exploitable, reducing false positives in automated security analysis.
• Reward Shaping: Reward shaping is the adjustment of training signals so an agent is encouraged to produce the behaviour a security programme wants. In this context, it pushes the model toward efficient, validated investigation rather than speculative reporting or excessive tool use.
• Context Compaction: Context compaction is the process of summarising or pruning conversation history so a long-running AI session can continue inside a fixed token window. It is useful for continuity, but it can also remove standing instructions, making it a security-sensitive operation in agentic systems.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is a fit for teams that need to connect governance, controls, and operational practice across identity-driven programmes.