A prompt is not a security function for AppSec triage

At a glance

What this is: Pixee argues that prompt-only AppSec triage is too unstable for enterprise use because identical findings can be classified differently across runs.

Why it matters: This matters because AppSec and IAM teams need repeatable, auditable decisions, not model output that changes classification, precedent, or risk treatment from one run to the next.

Context

AppSec triage is a governance problem as much as a classification problem. When the same finding can be called a true positive, false positive, or accepted risk depending on the prompt or run, the organisation does not have a reliable control surface for deciding what to fix, defer, or accept. That instability becomes more serious when the finding sits inside a broader security programme with rules, precedents, and ownership.

The identity angle is indirect but real: the article treats the AI system as a decision-making component whose behaviour must be governed, not simply prompted. In practice, the issue is similar to any non-human decision process that needs permissions, memory, evidence, and auditability before it can be trusted at scale.

Key questions

Q: What breaks when AppSec triage is driven by prompts alone?

A: Prompt-only triage breaks repeatability. The same evidence can be labelled differently across runs because the model is not operating inside a governed decision system with fixed precedence, state, and audit logic. That creates inconsistent risk treatment, noisy reporting, and weak accountability for the final security decision.

Q: Why do identical SAST findings need a governed harness?

A: Because the finding is not the problem, the decision process is. A governed harness preserves policy, remembers prior rulings, and keeps classification tied to evidence rather than to whichever response the model produces on a given run. That is what makes triage auditable and operationally useful.

Q: How do organisations know if AI triage is actually working?

A: Measure whether the AI improves high-fidelity detection, shortens time to verified response, and preserves reviewer trust in its decisions. A system that merely closes more alerts is not enough. The right signal is whether the SOC can validate its conclusions quickly and use them in real investigations without rework.

Q: What should teams do when a critical finding is not actually exploitable?

A: They should re-score it using environment context, document the compensating controls, and keep it in a tracked risk queue rather than treating it as an emergency. The goal is not to ignore the issue but to place it in the right priority band so it does not displace findings with real exploitation paths.

Technical breakdown

Why prompt-only triage drifts across runs

A prompt is only one instruction layer in a larger decision system. In AppSec triage, the model is being asked to weigh exploitability, code context, policy intent, and organisational tolerance at the same time. Large language models are weak at maintaining stable instruction priority across repeated runs, especially when the evidence admits more than one defensible interpretation. That is why the same SAST or SCA finding can land on different labels even when the underlying code has not changed. The problem is not that the facts move. The problem is that the decision logic is under-specified and the model is being asked to improvise governance.

Practical implication: Treat prompt-only triage as advisory input, not the decision mechanism.

Why deterministic harnesses matter for AppSec governance

A harness turns model output into a governed workflow. Deterministic integrations, caching, memory, and reporting create a chain of custody for the decision, so the team can see what was classified, why it was classified that way, and whether the same issue was already adjudicated. This is what moves the system from plausible text to repeatable operations. In the article's framing, the harness is also where policy lives, including severity ordering, repository-specific guidance, and the handling of latent risk. Without that structure, the model may sound aligned while still producing inconsistent outcomes.

Practical implication: Build the triage workflow so classification is reproducible, reviewable, and tied to policy evidence.

How the insecure-randomness example exposes latent risk

The RavenDB example shows a common triage tension: a finding can be technically valid even when the current attack path is weak. A non-cryptographic random value is being used to populate a credential field, but the application mode does not currently rely on that password. Some teams will still call that a true positive because latent security debt should not be left in place. Others will call it a false positive because exploitability is not obvious today. The control gap is not the finding itself. The gap is the absence of an agreed rule for how to handle latent credential risk before it becomes operational.

Practical implication: Define explicit adjudication rules for findings that are safe today but dangerous after configuration change.

NHI Mgmt Group analysis

Prompt-only triage is not a security function because it cannot guarantee governance consistency. A model can describe a finding, but it cannot by itself encode the organisation's precedence rules, exception logic, and risk tolerance. That makes prompt-first triage fragile in any programme that needs repeatable decisions across teams and repositories. The practitioner conclusion is simple: the decision system must exist outside the prompt.

AppSec triage now has a non-human governance problem. Once an AI system is making or shaping security decisions, it must be governed like any other non-human decision process with clear ownership, traceability, and measurable behaviour. The article's harness model is really about decision rights, not clever prompting. Practitioners should treat model output as one input inside a broader control system.

State, memory, and reporting are the difference between classification and control. A triage system that cannot remember prior rulings, fetch prior analyses, or expose its decision trace cannot support enterprise governance. That becomes a compliance and operations problem as much as a technical one. The named concept here is compounding derangement: small instruction errors that accumulate across repeated model interactions until the programme becomes unpredictable. The practitioner conclusion is to constrain the system before it drifts.

Latent credential risk needs a governance rule, not a better prompt. The RavenDB example shows that some findings are intentionally unresolved until a later configuration change makes them actionable. That is a policy question about when to retain, suppress, or escalate a finding, not a question of model eloquence. Teams should define that rule centrally and apply it consistently across all repositories and business units.

What this signals

AppSec triage is moving from model output to governed workflow. The article's real lesson is that security teams cannot operationalise AI by prompting harder. They need determinism, retained precedent, and an audit path that survives review, because enterprise triage decisions have to stand up to challenge.

Compounding derangement: once a model starts carrying ambiguous instructions, each new triage run can drift a little farther from policy intent. That is why memory, caching, and reporting matter as much as classification accuracy. The programme should be designed to constrain drift before it affects decision quality.

For practitioners

• Define triage adjudication rules Document how your programme classifies latent risk, exploitability, accepted risk, and false positives so the same finding is treated consistently across runs and repositories.
• Separate prompting from decision logic Use prompts to assist analysis, but keep the final classification, precedence rules, and exception handling in governed workflow logic.
• Add audit trails for model-influenced decisions Record the evidence, policy inputs, prior rulings, and final outcome for each adjudicated finding so reviewers can reconstruct why the system chose that label.
• Use caching and state to preserve precedent Retain prior analyses and reuse them when no relevant code change has occurred, so repeated scans do not re-litigate the same issue.
• Limit model authority over high-severity findings Force deterministic routing for the most sensitive categories so the model cannot overrule your severity policy or escalation thresholds.

Key takeaways

• Prompt-only AppSec triage is unstable because the same evidence can receive different labels across repeated runs, which makes the result hard to trust in an enterprise setting.
• The real control is the harness around the model, where policy, memory, caching, and reporting turn classification into an auditable process.
• Teams need explicit rules for latent risk and exception handling so findings are governed consistently instead of being re-decided by each prompt interaction.

Key terms

• AppSec triage harness: The governed workflow around an application security model that turns raw analysis into a repeatable decision. It combines policy, state, reporting, and deterministic integrations so findings can be classified consistently and audited later.
• Latent risk: A condition where a finding is not immediately exploitable but may become security-relevant after configuration, context, or dependency changes. In AppSec governance, latent risk often sits between a technical issue and a formal exception decision.
• Decision trace: The record of how an access decision was made, including inputs, policy logic, and the final allow or deny outcome. For AI-assisted identity systems, decision traces are necessary for auditability, troubleshooting, and proving that automated access was bounded and explainable.
• Compounding derangement: A pattern where small instruction or state errors in an AI-driven workflow accumulate across repeated interactions until the system's outputs drift away from policy intent. It is a governance and reliability problem, not just a model-quality issue.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course, the industry's only accredited NHI security programme. It is designed for practitioners who need governed control over non-human identities and AI-driven decision systems.