What breaks when organisations rely on LLMs alone to find OWASP Top 10 vulnerabilities?

The main failure is false confidence. An LLM can infer that input is safely handled upstream when it has not verified the path, so it may miss obvious issues like injection or unsafe output handling. Because it does not evaluate every line against fixed rules, results can vary between scans. That makes it unsuitable as the only control for known vulnerability classes.

Why This Matters for Security Teams

Relying on an LLM alone to find OWASP Top 10 issues turns vulnerability discovery into a probabilistic judgment exercise, which is a poor fit for controls that need repeatable coverage. The problem is not that LLMs are useless. It is that they can sound confident while missing path-specific flaws, unsafe data handling, or weak trust boundaries that a rules-based review would catch. That matters because OWASP Top 10 findings often hinge on concrete evidence, not plausible interpretation, and the security team still needs traceability for remediation decisions. Guidance from the NIST AI Risk Management Framework is clear that AI output needs governance, validation, and human accountability when used in security workflows.

The practical issue is coverage. An LLM may identify injection risk in one code path, then miss the same flaw in a slightly different handler because the model inferred a safe upstream sanitiser that was never verified. It can also overlook insecure deserialisation, broken access control, or output encoding issues when the evidence is spread across files, services, or generated code. In practice, many security teams encounter these failures only after a developer has already treated the LLM result as a pass and merged vulnerable code.

How It Works in Practice

An LLM can still be useful in a security review, but only as an assistive layer. The right workflow is to combine deterministic checks, source-of-truth standards, and analyst review. For OWASP Top 10 coverage, that means using the model to triage likely problem areas, summarise code context, and propose test hypotheses, then validating those hypotheses with static analysis, manual inspection, runtime testing, or dependency review. Where agentic or tool-using systems are involved, the scope widens to orchestration, tool abuse, and prompt-induced behaviour, which is why the OWASP Agentic AI Top 10 and NIST AI 600-1 Generative AI Profile are useful reference points for governance and validation.

• Use the LLM to prioritise where to look, not to certify that a class is absent.
• Require evidence links to code lines, request flows, or test artefacts for every finding.
• Cross-check model output against deterministic scanners and secure coding rules.
• Review access control, input validation, and output encoding separately rather than as one combined judgment.
• Retain human approval for remediation closure, especially when findings affect exposed interfaces or critical business flows.

For advanced threats, mapping model behaviour against the MITRE ATLAS adversarial AI threat matrix and the CSA MAESTRO agentic AI threat modeling framework helps teams recognise where prompt injection, data poisoning, or tool misuse may distort the review itself. These controls tend to break down when code is dynamically generated or stitched together from multiple services because the model cannot reliably reconstruct execution paths or trust boundaries from partial context.

Common Variations and Edge Cases

Tighter review workflows often increase analyst time and tool sprawl, requiring organisations to balance speed against assurance. That tradeoff becomes more visible when teams want LLM-assisted scanning in CI/CD, where fast feedback is valuable but false negatives are costly. Best practice is evolving, and there is no universal standard for letting an LLM make final vulnerability judgments on its own.

Some environments create especially sharp edge cases. Generated code may look internally consistent while hiding weak authentication, unsafe deserialisation, or insufficient output handling. RAG-enabled security assistants can also inherit stale policy text or incomplete repository context, which causes the model to miss the exact location of the flaw. In identity-heavy systems, this intersects with the OWASP Non-Human Identity Top 10 because secret handling, token scope, and service-to-service trust often determine whether an issue is exploitable or merely theoretical. The operational pattern is to treat the LLM as a qualified reviewer assistant, not as a control decision engine.

Where regulated products are involved, teams should document that AI-supported findings were validated against a repeatable method and mapped back to known vulnerability classes. That discipline matters most when the application mixes human user input, autonomous actions, and privileged API access, because a single missed boundary can turn a soft warning into a real incident.

Related resources from NHI Mgmt Group

• What breaks when organisations rely on MFA alone for digital interactions?
• What breaks when healthcare organisations rely on RBAC alone?
• What breaks when organisations rely on EDR alone for browser security?
• What breaks when organisations rely on endpoint controls alone for AI use?