A prompt is not a security function for AppSec triage

At a glance

What this is: This is an analysis of why prompt-only AI triage is too unstable for AppSec workflows and how a purpose-built harness changes the control model.

Why it matters: It matters to IAM and security practitioners because the same governance failure pattern appears whenever identity, entitlement, or finding decisions depend on inconsistent AI output instead of auditable process and control.

👉 Read Pixee's analysis of why prompt-driven AppSec triage needs a harness

Context

Prompt-only AI classification creates governance drift when the same evidence can produce different outcomes across runs. In AppSec, that is not just a model-quality issue, it is a control issue, because triage decisions influence remediation priority, risk acceptance, and audit traceability. The underlying lesson extends to identity governance as well: when access, entitlement, or secret decisions are made by unstable logic, the programme loses repeatability before it loses technical accuracy.

A purpose-built harness turns the model into one component inside a broader control system. For identity and security teams, that means policy, precedent, reporting, and state management have to sit around the model rather than inside a prompt. The article’s starting point is typical of modern AI operational risk, where teams discover that effective governance depends more on process design than on one-off model instructions.

Key questions

Q: What breaks when AI triage is driven only by prompts?

A: Prompt-only triage breaks consistency. The same evidence can be classified differently across runs because the model is not enforcing a fixed policy, preserving precedent, or replaying prior decisions. That creates governance drift, weak auditability, and inconsistent remediation priority. A secure workflow needs state, reporting, and accountable human ownership around the model.

Q: Why do AI-assisted security decisions need a harness instead of a prompt?

A: A harness is needed because security decisions depend on repeatability, provenance, and control ownership. A prompt can produce a plausible answer, but it cannot reliably preserve prior outcomes, record the evidence trail, or enforce organisational policy across changing contexts. The harness is what makes the AI output governable.

Q: How do organisations know if AI triage is actually working?

A: Measure whether the AI improves high-fidelity detection, shortens time to verified response, and preserves reviewer trust in its decisions. A system that merely closes more alerts is not enough. The right signal is whether the SOC can validate its conclusions quickly and use them in real investigations without rework.

Q: Who is accountable when an AI system makes a harmful decision?

A: Accountability should follow the identity chain that authorized, configured, or triggered the action, including the human owner, the platform team, and any delegated agent or tool account. If the organisation cannot name that chain, the governance model is too weak for regulated AI use.

Technical breakdown

Why prompt-only triage produces inconsistent security classifications

A prompt-driven model can describe the same code finding differently because it is not executing a fixed policy engine. It is generating a response from the evidence and the instruction set available at that moment, which makes its output sensitive to wording, context order, and ambiguity in the risk model. In the example given, the code contains a non-cryptographic random value stored as a credential-like field, but teams may disagree on whether latent exposure is actionable today or only if the authentication mode changes later. That disagreement is normal in AppSec, but a prompt alone cannot encode the organisation’s adjudication standard reliably.

Practical implication: classify the risk rule in a governed workflow, not in a free-form prompt.

What a harness changes in the security decision path

A harness adds deterministic integrations, memory, caching, reporting, and control logic around the model. Deterministic integrations reduce compounding errors, memory preserves prior adjudications, caching avoids re-litigating unchanged findings, and reporting exposes why the system reached a conclusion. That changes the model from a text generator into a governed decision component. The article’s point is not that AI disappears from the workflow, but that AI must be constrained by policies and state if teams want repeatable outcomes and auditable accountability.

Practical implication: design for replayable decisions, not just accurate-sounding outputs.

Why this is also an identity governance problem

The identity angle is straightforward: triage logic, entitlement decisions, and secret handling all fail when the organisation cannot explain why one case was accepted and another was escalated. If a model decides on behalf of a team, it is effectively participating in governance, which means its inputs, prior decisions, and policy boundaries must be controlled like any other privileged workflow. That is the same reason identity programmes need lifecycle evidence, access history, and decision provenance. A prompt can suggest an answer, but it cannot satisfy auditability on its own.

Practical implication: treat AI-assisted security adjudication as a governed identity and control process.

NHI Mgmt Group analysis

Prompt-only AI triage creates governance debt, not just classification noise. The core failure is not that the model sometimes gets the answer wrong. The deeper problem is that the same evidence can be interpreted differently across runs, which breaks consistency in risk acceptance, remediation prioritisation, and audit evidence. In practice, that means enterprises are embedding policy variance into the control plane. The practitioner conclusion is simple: if the decision must be defensible, the process must be governed outside the prompt.

Harness design is becoming the real control boundary for AI-assisted security work. Deterministic integrations, stateful memory, caching, and reporting are not conveniences, they are the mechanisms that make model output reviewable and repeatable. This is the same structural shift identity teams face when they move from ad hoc approvals to lifecycle governance. The practitioner conclusion is to build for traceable decision paths, not for clever prompt iteration.

AI-assisted classification now needs explicit accountability for prior precedent. If a model can reclassify a finding today that was accepted yesterday, the programme has no stable governance anchor. That is especially relevant where the output influences code risk, privileged access decisions, or secrets handling, because the decision itself becomes part of the security evidence chain. The practitioner conclusion is to require decision provenance as a control objective, not an afterthought.

People, process, and technology still define the security function even when the model does the wording. The article correctly frames the model as one component inside a larger system. That larger system is what determines whether the output is trusted, repeatable, and auditable. For identity and AppSec leaders, the implication is that AI governance must be operational, with ownership, policy, and review loops that survive model drift. The practitioner conclusion is to govern the system, not the prompt.

What this signals

Prompt-based AI triage is already exposing a broader governance gap: organisations are trying to automate judgment before they have formalised judgment itself. That matters for identity programmes because access, entitlement, and secret decisions also depend on stable precedent, evidence retention, and reviewability. The control lesson is to treat the harness as part of the governance model, not as implementation detail.

Decision provenance becomes the named control gap: if the programme cannot reconstruct why a finding, access request, or exception was accepted, the AI layer is creating audit risk rather than reducing it. Teams should align AI-assisted workflows with the same traceability expectations they would apply to privileged access approvals or lifecycle exceptions.

For practitioners

• Separate model output from policy decisioning Define whether the model is recommending, classifying, or adjudicating, and stop it from collapsing those roles into one response. Use a written decision policy that states when latent risk becomes actionable and who can override the model.
• Require replayable evidence for every adjudication Store the input, the model version, the prompt, the prior precedent, and the final human decision so the same case can be reproduced later. That is the minimum evidence trail for security decisions that may affect remediation, risk acceptance, or audit review.
• Add state and caching before scaling AI triage Use caching to prevent the same unchanged finding from being re-argued and state to preserve prior outcomes across runs. This reduces unnecessary churn and makes classification behaviour inspectable over time.
• Map AI adjudication to existing control owners Assign ownership for model-guided decisions to the same programme leaders who own the underlying control, such as AppSec, IAM, or GRC. Do not let a prompt become an informal policy layer outside accountability.

Key takeaways

• Prompt-only AI triage is too unstable to serve as a security control because it can change classification without any real change in evidence.
• The control boundary shifts from the model to the harness, where policy, state, reporting, and determinism make decisions auditable.
• Identity and security leaders should treat AI-assisted adjudication as a governed workflow with ownership, precedent, and evidence retention.

Key terms

• Decision Provenance: Decision provenance is the ability to explain what signals, data, and reasoning context led to a system’s choice. For autonomous or agentic systems, it is critical because review teams need to know not only what happened, but why the decision was made and where human authority still applies.
• Harness: The harness is the layer of instructions, policies, and approval logic wrapped around an AI agent. It is where organisations try to constrain behaviour, but it only works if the rules are explicit, current, and enforced outside the model itself.
• Classification Drift: Classification drift is the gradual mismatch between a system's labels and the real sensitivity of the content as files change over time. It happens when documents are edited, copied, or repurposed faster than the model or rules are updated, creating gaps between visibility and actual protection.

What's in the full article

Pixee's full blog post covers the operational detail this post intentionally leaves for the source:

• The full decision-path breakdown for how a harness separates recommendation, classification, and adjudication in AppSec workflows.
• Examples of deterministic integrations, memory, and caching patterns that reduce run-to-run classification drift.
• The reporting and traceability approach used to explain why a model reached a specific finding classification.
• The operational trade-offs involved in tuning model behaviour across repositories, business units, and security policies.

👉 Pixee's full post covers the classification example, harness design, and governance mechanics in more operational detail.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and identity lifecycle control. It suits practitioners who need to connect policy, evidence, and operational accountability across security programmes.