AI reviewers and autofix engines now comment on, triage and rewrite code in the pull request itself, and their output is non-deterministic, sometimes wrong, and can be steered by the very code under review. Used well, they cut the backlog of static analysis findings and catch issues human reviewers skim past. Used as a substitute for review, they become a fast path for plausible but broken fixes, fabricated dependency changes and approvals nobody really gave. This guide covers the main patterns: LLM triage of SAST findings, LLM-generated fixes such as GitHub Copilot Autofix for CodeQL alerts, and LLM pull request reviewers. It sets out the guardrails, rollout, metrics and framework mapping each needs. It is for application security leads, platform engineers and engineering managers adopting these tools. Choosing a testing platform is covered in the Application Security Testing Buyer's Guide; reviewing code that AI wrote is covered in the AI-Generated Code Review Guide.
Key takeaways
- Treat AI output as an untrusted contribution. Vendor documentation is explicit that suggested fixes can be incomplete, introduce new vulnerabilities or contain unsafe or fabricated dependency changes, so every fix goes through the same tests and review as human code.
- Keep the deterministic scanner as the source of truth. Let the SAST engine find and re-verify; let the model explain, prioritise and draft. A finding is closed when a rescan says so, not when a model says so.
- AI review is not approval. Copilot code review does not count toward required approvals by default; if you enable AI approvals, decide deliberately which repositories and paths may rely on them.
- The reviewer reads attacker-controlled input. Code, comments and instruction files in a pull request are all model input, and some reviewers read their instructions from the pull request's own branch, which makes prompt injection a review-integrity risk.
- Measure acceptance and regressions, not volume. Fix acceptance rate, rescan-verified closure and post-merge reversions show value; comment counts do not.
What AI SAST and AI code review actually do
The label covers several distinct capabilities with different risk profiles. Separate them in policy, because a control that suits one does not suit another.
| Capability | How it works | Main benefit | Main risk |
|---|---|---|---|
| AI triage of SAST findings | An LLM reads a finding and surrounding code and labels it likely true or false positive, or ranks it | Cuts alert fatigue and backlog | Suppressing real vulnerabilities as noise (false dismissal) |
| AI explanation | The model explains the finding and data flow in plain language | Faster developer understanding | Confident but wrong explanations (OWASP LLM09:2025 Misinformation) |
| AI autofix | The model proposes a code change for a scanner finding, shown in the pull request | Shorter time to remediate | Fixes that do not fix, break behaviour or add dependencies |
| AI pull request reviewer | The model reviews the diff and comments, suggests changes and may approve | Extra pair of eyes on every change | False negatives treated as assurance; manipulation by the diff |
| AI-native detection | The model itself searches for vulnerabilities without a rules engine | Finds logic flaws rules miss | Non-repeatable results, hard to audit and gate on |
A worked example: Copilot Autofix for code scanning
GitHub's responsible-use documentation describes the flow. When CodeQL raises an alert, GitHub builds a prompt from the SARIF alert data, code snippets around the source, sink and other referenced locations, roughly the first ten lines of each involved file and the CodeQL query help text. A large language model returns a proposed change and explanation, shown inline on the pull request or alert. It covers a subset of queries in the default and security-extended suites for C#, C/C++, Go, Java/Kotlin, Swift, JavaScript/TypeScript, Python, Ruby and Rust. GitHub lists the limitations plainly:
- Non-deterministic: a fix may not be produced, or may differ between attempts.
- Possibly wrong: fixes may be incomplete, may not remove the vulnerability, may introduce new ones, and may contain syntax, location or semantic errors because the system does not understand the programmer's intent.
- Weaker on complexity: multi-file data flows and subtle logic flaws are harder; very large files may be truncated, in which case no fix is attempted.
- Dependency changes: suggested dependency changes may be unsafe or fabricated, so they must be verified before merge. This overlaps with package hallucination; see the Slopsquatting and Package Hallucination Guide.
GitHub states that data handled by Copilot Autofix is not used for LLM training. Whatever the tool, confirm the equivalent statement, the model provider and data residency in the contract before enabling it on private code.
Guardrails for AI-generated fixes
- Same pipeline, no shortcuts: an accepted AI fix is a commit like any other. It must pass unit and integration tests, the full SAST suite and dependency scanning before merge.
- Rescan to close: the original scanner must re-run on the fixed code and report the alert as fixed. Never close a finding on the model's assertion.
- Block dependency edits by default: treat any fix that adds or changes a dependency, lockfile or build file as requiring a separate, human-approved change.
- Semantic review: reviewers check that the fix preserves intended behaviour, for example that an input validation fix has not rejected legitimate inputs or moved the check after the sink.
- Scope by risk: allow autofix suggestions everywhere, but require a security champion's review before merging AI fixes in authentication, authorisation, cryptography and payment code paths.
- Attribution: label AI-suggested commits (a commit trailer or pull request label) so you can measure them and find them later if a pattern of bad fixes emerges.
- Kill switch: know how to turn the feature off. For Copilot Autofix, enterprises set it to not allowed under Policies, Advanced Security; organisations and repositories switch it off in their Advanced Security settings. Disabling closes open suggestion comments, and agentic autofix has separate controls.
Guardrails for AI triage
Triage is where AI can quietly do the most damage, because a dismissed finding generates no further signal.
- Recommend, do not dismiss: let the model label findings, but require a human to dismiss, or auto-dismiss only categories with a measured false-dismissal rate below your threshold.
- Keep the evidence: store the model's reasoning with the finding so auditors and later reviewers can see why it was deprioritised.
- Sample the dismissed set: each sprint, a security engineer re-reviews a random sample of AI-labelled false positives. If real issues appear, tighten the threshold for that rule.
- Never let triage change severity of critical classes: injection, authentication bypass and deserialisation findings keep their scanner severity regardless of model opinion.
- Know what the model saw: triage and fix prompts are built from snippets around the finding, not the whole program. GitHub's Autofix, for example, sends code around the source and sink plus the first lines of each involved file. A model judging a taint flow without the sanitiser two files away can confidently call a real issue a false positive, so weight its labels lower for multi-file data flows and framework-heavy code where context matters most.
- Feedback loops are configuration: organisation-specific instructions or "memories" that tell a triage model to ignore a pattern are effectively suppression rules. Review them like suppression rules, with an owner and expiry.
Guardrails for AI pull request reviewers
Review integrity and approvals
GitHub documents that Copilot's reviews do not count toward required approvals by default, and that an approval feature in public preview can let Copilot submit an approving review that satisfies a required-approval rule, dismissed by a new push. Before turning anything like that on, decide in branch protection or rulesets which repositories may accept an AI approval, and keep human approval mandatory for code owners of sensitive paths. GitHub's responsible-use page says the reviewer may miss problems, especially in large or complex changes, and should be supplemented with careful human review. See the Secure Code Review Guide for the human process it supplements.
Prompt injection through the diff
An AI reviewer's input is the change under review, which the author controls. Comments, strings, documentation and test data can carry instructions aimed at the model, for example telling it the code has been security-approved (OWASP LLM01:2025 Prompt Injection). Two documented behaviours raise the stakes:
- Instructions from the head branch: GitHub states that Copilot code review reads custom instructions and skills (
.github/copilot-instructions.md,.github/instructions/**/*.instructions.md,AGENTS.md) from the pull request's head branch. A pull request can therefore change the reviewer's instructions in the same change it asks to be reviewed. Protect these files with CODEOWNERS so changes to them need security approval, and flag any pull request that modifies them. The Coding Agent Configuration File Attacks guide covers this class. - Tools during review: GitHub's documentation lists MCP tools as enabled by default for Copilot code review. Review what those tools can reach, and restrict them for repositories that accept outside contributions.
Treat model output as untrusted too (OWASP LLM05:2025 Improper Output Handling): a suggested change that is applied with one click must still go through tests, and a reviewer bot's token should not have write access beyond posting reviews.
Where identity fits
AI reviewers and autofix engines act through GitHub Apps, bot accounts or tokens. Give each the narrowest permissions (read code, write pull request comments), separate the identity that comments from any identity that can merge or push, and log its actions like any other non-human identity. Restrict who can trigger reviews; GitHub provides a setting to only allow Copilot code review to be triggered by authorised users. See the GitHub and GitHub Actions Security Guide.
Choosing and scaling: decision guidance
| Situation | Recommended pattern | Why |
|---|---|---|
| Small team, few security staff | Autofix suggestions on scanner findings plus AI reviewer comments; humans approve everything | Highest leverage for scarce reviewers with low risk of silent failure |
| Large backlog of legacy findings | AI triage to rank, humans dismiss; autofix campaigns per rule with batch review | Burns down backlog without auto-dismissing real issues |
| Regulated or safety-critical code | AI explanation and suggestions only; no AI approvals, no auto-dismissal; documented model and data terms | Auditors need repeatable, attributable decisions |
| Open source or outside contributors | AI review on maintainer request only; instruction files under CODEOWNERS; tools restricted | Untrusted authors control the model's input |
| AI-native detection tools | Run as an additional advisory scanner, not as a merge gate | Non-deterministic results are hard to gate on fairly |
Phased rollout
- Baseline: record current mean time to remediate SAST findings, backlog size and false positive rate per rule, so you can measure change.
- Pilot ring: enable autofix suggestions and AI review comments on a few volunteer repositories with good test coverage. No approvals, no auto-dismissal.
- Measure for four to six weeks: track acceptance, rescan-verified fixes, edits made to accepted fixes, and any reverts or incidents.
- Harden configuration: protect instruction files with CODEOWNERS, restrict review triggers and tools, label AI commits, block dependency edits in AI fixes.
- Expand by tier: roll out to further repositories, keeping sensitive paths under mandatory human security review.
- Consider triage automation last: only after measuring false-dismissal rates per rule.
- Rollback: keep the organisation-level switch documented; if a bad fix pattern appears, disable for the affected rule or repository, then search labelled AI commits for the same pattern.
Detection and measurement
- Fix acceptance rate: share of AI fix suggestions merged, and share merged without edits. Very high unedited acceptance on complex rules can signal rubber-stamping.
- Rescan-verified closure: percentage of accepted AI fixes where the scanner confirms the alert closed and no new alert opened in the same code.
- Regression rate: AI-labelled commits reverted, or linked to bugs or new findings within 30 days.
- False-dismissal rate: real vulnerabilities found in samples of AI-triaged false positives.
- Time to remediate: median days from finding to verified fix, compared with the baseline.
- Integrity signals: pull requests that modify AI instruction files, AI approvals on protected paths (should be zero) and reviews triggered by unexpected users.
Framework mapping
| Control | NIST SP 800-53 Rev 5 | NIST CSF 2.0 / SSDF | OWASP LLM Top 10 2025 |
|---|---|---|---|
| SAST plus AI review as developer testing | SA-11 Developer Testing and Evaluation | PR.PS-06; SSDF PW.7 (review and analyse code) | LLM09 Misinformation (over-reliance) |
| Tests and rescans before merging AI fixes | SA-11, CM-3 Configuration Change Control | SSDF PW.8 (test executable code) | LLM05 Improper Output Handling |
| Approved tools, models and data terms | SA-15 Development Process, Standards, and Tools | PR.PS-06; SP 800-218A for AI development | LLM02 Sensitive Information Disclosure |
| Instruction files under CODEOWNERS, review integrity | SA-10 Developer Configuration Management | PR.PS-06 | LLM01 Prompt Injection |
| Measuring acceptance and regressions | SA-11 | ID.IM-02 | LLM09 |
Common failure patterns
- Closing findings on the model's word: the fix looked right and the alert was closed manually, but the scanner would still have flagged it. Require rescan closure.
- Merging fabricated or risky dependency changes: an autofix adds or bumps a package that does not exist or is malicious. Block dependency edits in AI fixes.
- Counting AI review as the second reviewer: teams quietly drop human review because the bot commented. Keep human approval rules explicit in rulesets.
- Unprotected instruction files: a pull request edits the reviewer's instructions to relax checks, and the review runs with the edited instructions. Put them under CODEOWNERS.
- Auto-dismissing by model label: real issues are buried as false positives with no audit trail. Recommend, sample and measure before automating.
- Enabling on code with poor tests: behaviour-breaking fixes pass because nothing tests the behaviour. Pilot on well-tested repositories first.
Practitioner checklist
- Each AI capability in use (triage, explanation, autofix, review, AI-native detection) is listed with its owner and allowed scope.
- AI fixes pass the full test, SAST and dependency pipeline, and alerts close only on a verified rescan.
- AI fixes that change dependencies or build files are blocked or need separate human approval.
- Human approval remains required on all repositories, and on sensitive paths without exception.
- Reviewer instruction files are protected by CODEOWNERS, and pull requests that change them are flagged.
- Review triggers are limited to authorised users, and reviewer tools and token permissions are minimal.
- Model provider, data handling and training terms are documented and approved for private code.
- Acceptance, rescan closure, regression and false-dismissal metrics are reported monthly.
- The organisation-level kill switch and rollback steps are documented and tested.
Standards and references
- GitHub Docs: Responsible use of Copilot Autofix for code scanning
- GitHub Docs: Disabling Copilot Autofix for code scanning
- GitHub Docs: Responsible use of GitHub Copilot code review
- GitHub Docs: About GitHub Copilot code review
- NIST: SP 800-218 Secure Software Development Framework
- NIST: SP 800-218A SSDF Community Profile for Generative AI
- OWASP: Top 10 for LLM Applications 2025
- NIST: SP 800-53 Rev. 5 Security and Privacy Controls
Related NHI Mgmt Group resources: Application Security Testing Buyer's Guide · AI-Generated Code Review Guide · CodeQL Guide · AI Coding Agents Security Guide · GitHub Copilot Security Guide