LLMs are changing vulnerability discovery in large codebases

At a glance

What this is: Teleport describes how a team of 13 engineers used frontier LLMs over three months to pressure wash a large codebase and uncover dozens of vulnerabilities.

Why it matters: For IAM and security teams, the lesson is that LLM-assisted review can expand bug-finding capacity, but only if governance, triage, and remediation processes can absorb the output without creating noise.

By the numbers:

• Over the past quarter, Teleport had 13 software engineers dedicated to pressure washing its codebase with frontier models.
• Teleport says the team found almost twice as many high severity vulnerabilities in one quarter as it did in 2024 and 2025 combined.

👉 Read Teleport's post on using LLMs to find vulnerabilities in a large codebase

Context

Large language models are changing how security teams search for vulnerabilities, because the limiting factor is no longer only human attention. The practical problem is not whether models can surface more candidate bugs, but whether teams can triage, validate, and fix them without overwhelming engineering workflows.

In identity-heavy systems, that matters because code paths often govern authentication, authorisation, secrets handling, and service-to-service trust. When LLMs are used to inspect those paths, the governance question becomes how to operationalise model-assisted review without mistaking volume for quality. Teleport’s starting point is becoming increasingly typical for high-maturity engineering organisations that are trying to scale security review with AI.

Key questions

Q: How should security teams use LLMs to find vulnerabilities in large codebases?

A: Use LLMs as a focused review accelerator, not as an autonomous scanner. Start with a narrow component, ask for one high-impact issue, and have an experienced human validate exploitability. The goal is to increase expert throughput on code that controls authentication, secrets, and privilege checks, while keeping false positives and triage noise under control.

Q: When does AI-assisted code review become less effective than manual review?

A: It becomes less effective when orchestration layers add more noise than signal, or when reviewers cannot rapidly validate what the model returns. If the workflow produces lots of plausible candidates but no clear exploitability judgment, the bottleneck shifts to triage and the automation loses much of its value.

Q: What do security teams get wrong about agentic vulnerability hunting?

A: They often assume more agents and more stages will produce better results. In practice, simpler prompts and stronger human expertise usually outperform complex harnesses because the model still needs context, steering, and judgement. The best process is the one that helps a reviewer reach a defensible decision quickly.

Q: How should security teams prioritise vulnerabilities when AI speeds up attack discovery?

A: They should prioritise by exploitable context, not by severity alone. A weakness on an exposed, reachable, and privileged asset deserves more attention than a higher-scoring issue that cannot be reached. For cloud and NHI programmes, the practical test is whether fixing the issue will materially shrink attack paths and blast radius.

Technical breakdown

Why simple prompting outperformed complex agent harnesses

Teleport’s core finding is that orchestration complexity can subtract value from LLM-assisted vulnerability discovery. A multi-stage agentic harness introduces extra decision points, more noise, and more opportunities for the model to drift away from the actual code path being examined. In practice, the human reviewer still has to understand the system well enough to steer the model, recognise exploitability, and separate plausible findings from false positives. The result is not autonomous bug hunting, but expert-led augmentation. That distinction matters because the control surface is the review process itself, not the model output.

Practical implication: keep the workflow simple, and optimise for human review quality rather than agent count.

How LLMs change the economics of codebase hardening

The article shows that frontier models can widen the search space for vulnerability discovery across an established codebase. That changes the economics of security review because teams can revisit older code paths with fresh prompts, and can ask a model to look for variants of an already identified issue. The main constraint becomes triage capacity, not prompt generation. This is especially relevant in systems that depend on secure access logic, because authentication and privilege bugs tend to hide in patterns that repeated model queries can expose faster than manual review alone.

Practical implication: build a triage pipeline before scaling LLM-assisted review across authentication and access-control code.

Why AI-assisted review still depends on expert human judgment

Teleport’s experience reinforces a familiar security principle: models accelerate experts, they do not replace them. A strong reviewer can steer prompts toward critical components, validate exploitability, and notice when a finding is a duplicate or a dead end. That makes the operating model closer to security research than to fully automated scanning. In identity and access engineering, this matters because subtle flaws in trust boundaries, secret handling, and access checks often require contextual judgment that a model cannot reliably supply on its own.

Practical implication: assign experienced reviewers to AI-assisted findings and treat the model as an accelerator, not an authority.

Threat narrative

Attacker objective: The objective in the mirrored offensive use case is to identify exploitable software flaws faster than defenders can remediate them.

1. Entry begins with targeted LLM prompts applied to specific codebase components, rather than broad automated scanning.
2. Escalation occurs when the model surfaces plausible vulnerabilities faster than human teams can manually inspect them, increasing the review burden.
3. Impact is improved hardening and more bug fixes, but only after engineers separate real findings from model noise and triage the results.

NHI Mgmt Group analysis

LLM-assisted vulnerability discovery is now a governance problem, not just a tooling experiment. Once models can materially expand bug-finding capacity, the issue shifts from whether they can find flaws to how organisations control the review, validation, and remediation pipeline. That creates a governance burden across engineering, security, and risk functions, especially where code governs authentication, secrets, and access checks. The teams that win will be those that treat model-assisted review as a controlled operating process, not an ad hoc productivity hack.

Simple human-led workflows are the right named concept here: model output needs a narrow review path, not an elaborate agent stack. Teleport’s experience shows that additional orchestration layers can create more noise than signal, which is a familiar failure mode in security automation. In identity-adjacent systems, that noise can obscure real defects in privilege enforcement or secret handling. The practical conclusion is that the best control is a disciplined analyst loop, not a more complex harness.

AI does not remove the need for expert security judgement, it increases the premium on it. The article makes clear that experienced engineers and researchers extract more value from the same models because they can interpret findings, focus prompts, and test exploitability. For the broader security market, this validates a human-in-the-loop operating model for AI-enabled testing and review. Security leaders should expect AI to improve throughput, while quality still depends on reviewer skill and process maturity.

Code review at scale is becoming a parallel security channel, and that has identity implications. When LLMs inspect systems that manage authentication, service accounts, tokens, and access enforcement, they are indirectly probing the identity control plane embedded in the code. That means IAM and PAM teams need visibility into where AI-assisted testing touches authorisation logic, secret storage, and trust boundaries. The practitioner takeaway is to align AI-assisted review with identity governance, not leave it isolated inside engineering.

Security organisations should interpret higher bug-finding volume as a forcing function for prioritisation. If models can surface more issues than teams can fix, the bottleneck moves to risk ranking and remediation discipline. That changes how vulnerability management should be measured, because more findings without better prioritisation can slow secure delivery. The practical implication is to rank AI-discovered issues by exploitability, exposure, and identity impact before they reach production backlogs.

What this signals

LLM-assisted security review is becoming part of the modern vulnerability management operating model, but the workload it creates will only be useful if teams have strong confirmation and prioritisation discipline. The pressure point is no longer discovery alone. It is the governance layer that decides which findings deserve engineering time and which are just model-shaped noise.

AI review debt: the gap between how many issues models can surface and how many teams can responsibly fix will become a measurable governance risk. For identity-heavy systems, that gap is most dangerous where code governs trust, privilege, and token handling, because those defects can convert quickly into access exposure. Security leaders should prepare for AI-assisted review to increase both the quality of discovery and the demand for structured triage.

The broader signal for programmes is that AI should be integrated into security engineering with the same discipline used for other high-trust automation. That means defined review criteria, evidence capture, and clear ownership for remediation, rather than open-ended experimentation.

For practitioners

• Limit agentic complexity in vulnerability review workflows Use simple, component-specific prompts and avoid multi-stage orchestration unless it measurably improves exploitable finding quality. Reviewers should be able to trace why a model flagged a file or function, especially in access-control and secrets-handling paths.
• Create a human triage gate for all AI-generated findings Require experienced engineers or security researchers to validate exploitability, deduplicate findings, and prioritise by actual risk before remediation work is scheduled. This prevents model noise from consuming the same backlog that should be used for real security defects.
• Focus AI-assisted review on identity-critical code paths Prioritise code that governs authentication, privilege checks, token handling, service account use, and secret storage. Those paths are where subtle logic flaws can become high-impact access failures, and they benefit most from repeated model-driven inspection.
• Measure throughput against fix quality, not finding volume Track how many AI-discovered issues are confirmed, remediated, and prevented from reappearing. A high finding count is not success if it creates triage debt or distracts from the issues that materially affect access governance.

Key takeaways

• LLM-assisted review is expanding vulnerability discovery, but it also shifts the bottleneck to triage and exploitability judgement.
• Simple, human-guided workflows are outperforming complex agent harnesses in real codebase review.
• Identity-critical code paths deserve the first wave of AI-assisted hardening because access flaws carry the highest downstream risk.

Key terms

• LLM-assisted vulnerability discovery: The use of large language models to help locate software flaws in source code or related artefacts. The model does not replace human judgement, but it can expand the search space, surface patterns faster, and accelerate expert review when the workflow is tightly controlled.
• Agentic Harness: The orchestration layer that lets a model choose tools, execute actions, and repeat a task loop until a goal is met. In practice, it is what turns a model from a responder into an actor, so identity and privilege controls must govern the harness, not just the model.
• Triage capacity: The amount of human effort available to validate, prioritise, and route candidate findings into remediation. It is a critical control in AI-assisted security testing because discovery can scale faster than confirmation, creating backlog and risk-ranking problems if capacity is not planned.
• Identity-critical code path: An identity-critical code path is any software flow that governs authentication, authorisation, session state, token handling, secrets, or privilege checks. Failures in these paths can have disproportionate security impact because they directly shape who or what gets access.

What's in the full article

Teleport's full blog post covers the operational detail this post intentionally leaves for the source:

• The exact pressure-washing workflow the engineers used to target code components with LLM prompts.
• The examples of harness designs Teleport tried and why the more complex approaches created more noise.
• The bug-fix outcomes and release context for the vulnerabilities found during the quarter.
• The practical lessons the team drew from pairing expert humans with frontier models in security review.

👉 Teleport's full blog post covers the workflow choices, triage lessons, and bug-fix outcomes behind the pressure-washing effort.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and identity lifecycle controls. It gives security and identity practitioners a practical foundation for governing access risk across modern systems.