Topic starter
02/08/2026 11:27 am
TL;DR: After benchmarking more than 2,000 findings, AI Assistant now matches internal security researchers on true positives in 96% of cases while still remaining conservative on false positives, according to Semgrep. That level of performance makes AI triage a governance issue, not just a workflow improvement, because teams must decide where machine judgment ends and human review begins.
Editorial analysis by NHI Mgmt Group, based on content published by Semgrep: “How we built an AppSec AI that security researchers agree with 96% of the time”.
Key questions
Q: When should security teams trust AI triage for AppSec findings?
A: Teams should trust AI triage only when the system has been benchmarked against a well-curated reviewer set, shows high agreement on real issues, and operates with clear override paths.
Q: Why can AI triage improve AppSec governance without replacing analysts?
A: AI triage improves governance when it decides which findings deserve immediate developer attention, while analysts retain authority over policy, exceptions, and backlog review.
Q: What are the signs that AI finding triage is too conservative or too noisy?
A: Too much conservatism shows up as borderline findings flooding review queues, developers ignoring alerts, and AppSec teams spending time on issues that rarely prove exploitable.
Practitioner guidance
- Define triage authority boundaries Specify which finding classes AI may suppress, which require human review, and which must always reach developers.
- Benchmark on true positives first Measure agreement against a reviewer set that includes enough real issues to test whether the model misses exploitable findings.
- Treat rule metadata as a control input Review rule confidence values, example findings, and prior triage decisions because those inputs materially affect verdict quality.
Bottom line: AI triage in AppSec is now a governance control because it decides which findings developers see first and which issues remain in backlog.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
02/08/2026 12:43 pm
AI triage is becoming a governance layer, not a convenience feature. Once a model decides which findings interrupt developers and which remain in backlog, it shapes risk prioritisation and remediation behaviour. That means the operating question is no longer whether the AI can score findings, but who is accountable for the threshold at which the AI is allowed to suppress noise. Practitioners should treat triage policy as a governed control, not a workflow preference.
A few things that frame the scale:
- Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap, according to the State of Secrets in AppSec.
A question worth separating out:
Q: How should AppSec teams govern automated verdicts from AI triage systems?
A: Teams should govern automated verdicts with explicit policy, reviewer audit rights, and a retained backlog of all suppressed findings. That structure lets AI accelerate the first pass without becoming the final authority. Governance should also include periodic benchmarking, drift checks, and a clear rule for when the model must defer to human review.
👉 Read our full editorial: AI triage accuracy changes AppSec governance more than noise reduction