Notifications
Clear all
Topic starter
19/08/2026 3:36 pm
TL;DR: AI triage on SAST and SCA findings can swing between true positive and false positive on the same evidence, exposing why prompt-driven classification is too unstable for enterprise AppSec governance, according to Pixee. The real control surface is the surrounding harness, where policy, reporting, memory, caching, and deterministic integrations make decisions auditable and repeatable.
NHIMG editorial — based on content published by Pixee: A Prompt Is Not a Security Function
Questions worth separating out
Q: What breaks when AI triage is driven only by prompts?
A: Prompt-only triage breaks consistency.
Q: Why do AI-assisted security decisions need a harness instead of a prompt?
A: A harness is needed because security decisions depend on repeatability, provenance, and control ownership.
Q: How do organisations know if AI triage is actually working?
A: Measure whether the AI improves high-fidelity detection, shortens time to verified response, and preserves reviewer trust in its decisions.
Practitioner guidance
- Separate model output from policy decisioning Define whether the model is recommending, classifying, or adjudicating, and stop it from collapsing those roles into one response.
- Require replayable evidence for every adjudication Store the input, the model version, the prompt, the prior precedent, and the final human decision so the same case can be reproduced later.
- Add state and caching before scaling AI triage Use caching to prevent the same unchanged finding from being re-argued and state to preserve prior outcomes across runs.
What's in the full article
Pixee's full blog post covers the operational detail this post intentionally leaves for the source:
- The full decision-path breakdown for how a harness separates recommendation, classification, and adjudication in AppSec workflows.
- Examples of deterministic integrations, memory, and caching patterns that reduce run-to-run classification drift.
- The reporting and traceability approach used to explain why a model reached a specific finding classification.
- The operational trade-offs involved in tuning model behaviour across repositories, business units, and security policies.
👉 Read Pixee's analysis of why prompt-driven AppSec triage needs a harness →
AppSec triage harnesses: what changes when prompts are not enough?
Explore further
19/08/2026 4:17 pm
Prompt-only AI triage creates governance debt, not just classification noise. The core failure is not that the model sometimes gets the answer wrong. The deeper problem is that the same evidence can be interpreted differently across runs, which breaks consistency in risk acceptance, remediation prioritisation, and audit evidence. In practice, that means enterprises are embedding policy variance into the control plane. The practitioner conclusion is simple: if the decision must be defensible, the process must be governed outside the prompt.
A question worth separating out:
Q: Who is accountable when an AI system makes a harmful decision?
A: Accountability should follow the identity chain that authorized, configured, or triggered the action, including the human owner, the platform team, and any delegated agent or tool account. If the organisation cannot name that chain, the governance model is too weak for regulated AI use.
👉 Read our full editorial: A prompt is not a security function for AppSec triage
