LLMs are changing vulnerability discovery in large codebases

At a glance

What this is: Teleport’s analysis argues that LLMs can accelerate vulnerability discovery in large codebases, but simple prompts and expert review outperform heavier agentic harnesses.

Why it matters: For security and identity practitioners, the lesson is that AI-assisted code review changes how findings are surfaced and triaged, but it does not replace human judgement or control ownership.

👉 Read Teleport’s analysis of how LLMs are changing vulnerability discovery in large codebases

Context

Large codebases create a review problem, not just a scanning problem: once the search space gets big enough, the limiting factor becomes human bandwidth and the ability to isolate the right subset of code for inspection. In this article, the security question is how AI changes discovery workflows without replacing the need for judgement, verification, and remediation.

That matters for IAM, NHI, and broader security programmes because the same pattern appears whenever tooling increases detection volume faster than teams can validate it. In practice, the governance challenge is not whether an LLM can find more issues, but how organisations convert model output into disciplined triage, ownership, and fix prioritisation.

Teleport frames its approach as repeated, expert-led review of existing code rather than fully autonomous hunting. That is a familiar starting position for mature security teams, but the article suggests AI now makes the productivity gap between simple, well-directed workflows and complex orchestration much wider.

Key questions

Q: How should security teams use LLMs to find vulnerabilities in large codebases?

A: Use LLMs as a focused review accelerator, not as an autonomous scanner. Start with a narrow component, ask for one high-impact issue, and have an experienced human validate exploitability. The goal is to increase expert throughput on code that controls authentication, secrets, and privilege checks, while keeping false positives and triage noise under control.

Q: Why do simple prompts often outperform complex agentic harnesses in bug hunting?

A: Simple prompts preserve focus, reduce orchestration noise, and keep the reviewer close to the code and the exploit hypothesis. Once a harness adds too many agent stages, it often creates more candidate findings than a team can validate, which shifts the bottleneck from discovery to triage and weakens the overall workflow.

Q: What are the signs that AI-assisted vulnerability hunting is becoming noisy?

A: The clearest signs are a surge in low-confidence findings, duplicated issue reports, and a growing backlog of items that no one can verify quickly. When reviewers spend more time sorting output than understanding root causes, the workflow has outgrown the team’s triage capacity and needs tighter scope control.

Q: When should teams re-run AI-assisted review on legacy code?

A: Re-run it whenever model capability improves, when the codebase has aged into new risk, or when earlier reviews were too shallow to cover the full attack surface. Legacy modules often hide issues in authentication, secret handling, and access logic, so repeat passes can uncover problems that earlier tools missed.

Technical breakdown

Why simple prompts outperform complex agentic harnesses

The article describes a basic but important mechanism: LLMs are effective when a human constrains scope and asks for one concrete security task, such as reviewing a specific file or component. A complicated harness adds orchestration layers, but each layer can dilute focus, increase false positives, and make the output harder to verify. In vulnerability discovery, the quality of the prompt and the reviewer’s context matter more than the amount of automation wrapped around the model. That shifts the technical centre of gravity from pipeline design to human-guided task framing.

Practical implication: Use constrained prompts and narrow targets before adding orchestration layers.

Pressure washing large codebases as a repeatable review pattern

“Pressure washing” is essentially repeated, model-assisted inspection of existing code from multiple angles as models improve. That is different from point-in-time pull request review, because the search surface is broad and the goal is to surface latent issues that have accumulated over years. The method works best when teams subdivide the codebase into manageable components and revisit them over time. This is less about autonomous discovery and more about building a reusable inspection pattern that can be run again as models and threat understanding mature.

Practical implication: Break large estates into reviewable slices and treat model-assisted sweeps as recurring work, not one-off campaigns.

Human triage remains the control point for vulnerability discovery

The article repeatedly shows that LLMs can increase the number of candidate findings, but the bottleneck quickly becomes human triage. That means the security value comes from expert reviewers who understand exploitability, severity, and duplicate patterns well enough to separate signal from noise. In governance terms, the model expands search capacity, while the human still decides what is real, what is urgent, and what is actionable. For security teams, this is a workflow redesign problem, not a model-trust problem.

Practical implication: Plan for triage capacity before scaling AI-assisted hunting.

NHI Mgmt Group analysis

AI-assisted vulnerability discovery is now a bandwidth problem, not a novelty problem: the article shows that the advantage comes from increasing the number of credible searches humans can supervise, not from delegating security judgement to the model. That changes the governance question from whether to use AI at all to how much review capacity the organisation can absorb. Security teams should treat AI output as a force multiplier for expert analysis, not as an autonomous control.

Complex agentic harnesses can become a security liability by creating review debt: the more stages, judges, and summarisers inserted into the workflow, the more the process risks turning findings into noise. That matters because vulnerability discovery only has value when the team can verify and remediate the result quickly. The article’s central lesson is that operational simplicity often beats conceptual sophistication in security engineering.

Large codebase review now belongs in continuous security operations, not occasional clean-up projects: repeated model-assisted sweeps let teams revisit legacy code as capabilities improve and as new issue patterns emerge. This is especially relevant where code paths touch identity, credential handling, or access boundaries, because latent flaws in those areas can persist for years. Practitioner conclusion: build repeatable review cadence, not one-off AI experiments.

Expertise still determines whether AI finds exploitable bugs or just more findings: the article’s strongest results came when experienced humans pointed the model at the right area and interpreted the output with exploit context. That means the real differentiator is not model access, but the quality of the reviewer’s security intuition and the discipline of the workflow. Teams that remove expert oversight will likely increase output without improving risk reduction.

LLM vulnerability hunting is exposing a new governance gap: review capacity now lags discovery capacity: once AI raises the number of candidate issues, the limiting control is no longer detection but decision-making throughput. That is a programme design problem as much as a technical one, because remediation ownership, severity calibration, and release prioritisation all become more compressed. The practical conclusion is to scale analyst and engineering triage in parallel with AI-assisted discovery.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

AI-assisted vulnerability discovery is creating a new operating model for security engineering: teams now need enough review capacity to keep up with the number of findings LLMs can surface. The limiting factor moves from search to validation, so programme design has to include triage ownership and remediation routing from the start.

Simple, scoped prompts are becoming the practical control point: when reviewers constrain the model to one component and one task, output quality improves and the workflow stays auditable. That is a useful pattern for codebases where access logic, secrets handling, or identity flows are hard to inspect manually.

Pressure washing should become a repeatable assurance cycle, not a one-off experiment: recurring model-assisted reviews let teams re-check legacy code as the models improve and as the codebase changes. For practitioners, the strategic question is how to sustain expert review throughput without turning AI output into backlog debt.

For practitioners

• Constrain AI hunts to one component at a time Start with a specific file, module, or subsystem so reviewers can assess exploitability in context instead of drowning in broad candidate output.
• Use expert-led prompts before adding orchestration Begin with simple prompts that ask for a critical vulnerability in a defined scope, then add workflow layers only if they improve signal quality.
• Build triage capacity before expanding AI coverage Assign human reviewers who can validate severity, duplicate reports, and exploit paths, because discovery volume rises faster than decision throughput.
• Re-run legacy code review as models improve Treat pressure washing as a recurring programme for older code paths, especially where authentication, secrets, or access logic has accreted over time.

Key takeaways

• LLM-assisted bug hunting works best when humans tightly scope the task and retain final judgement over exploitability.
• The article’s experience shows that more automation can produce more noise if the review process is not designed around triage capacity.
• For security teams, the real control challenge is matching discovery volume with expert validation, not simply increasing the number of findings.

Key terms

• Agentic Harness: The orchestration layer that lets a model choose tools, execute actions, and repeat a task loop until a goal is met. In practice, it is what turns a model from a responder into an actor, so identity and privilege controls must govern the harness, not just the model.
• Pressure Washing: Pressure washing is repeated, model-assisted inspection of an existing codebase to surface latent security flaws. The term implies broad coverage across legacy code, with human reviewers narrowing scope and confirming which findings are actually exploitable.
• Triage capacity: The amount of human effort available to validate, prioritise, and route candidate findings into remediation. It is a critical control in AI-assisted security testing because discovery can scale faster than confirmation, creating backlog and risk-ranking problems if capacity is not planned.
• Secure Prompting: Secure prompting is the practice of giving AI coding tools explicit constraints that steer output toward safer patterns. It is a governance control, not a style preference, because prompts can influence how models handle authentication, input validation, logging, and other security-sensitive behaviours.

What's in the full article

Teleport's full blog post covers the operational detail this post intentionally leaves for the source:

• The team setup and quarter-long workflow used to pressure wash the codebase with frontier LLMs
• The comparison between multi-stage agentic harnesses and simple human-guided prompts
• Examples of how reviewers iterated on findings to find similar vulnerabilities in other code areas
• The security release impact, including the bug-fix outcomes from the quarter's review work

👉 Teleport’s full post covers the team workflow, prompt patterns, and bug-fix outcomes in more detail.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It helps security teams build the governance skills needed to manage machine identities, access, and AI-driven workflows.