How should security teams test whether an LLM’s code judgments are prompt-framing sensitive?

How to structure a prompt-framing sensitivity test

Test the model with the same code sample and a controlled set of prompts that differ only in framing. Use a neutral prompt, then variants that describe the same code as malicious, vulnerable, legal, or safe, and keep everything else constant. The goal is to isolate whether the model’s judgment changes because of wording rather than because of the code itself.

What matters is not whether the model can be persuaded to produce a different explanation, but whether the decision outcome itself stays stable. A useful test design records the final judgment, the confidence or rationale if available, and any internal routing or classifier signals that indicate the model treated the prompt as a different kind of request.

For teams building evaluation harnesses, a paired design is stronger than one-off spot checks. Run the same case through multiple framings, compare the outputs side by side, and repeat across a small set of code examples with different risk profiles so you can see whether the instability is systematic or limited to a specific pattern.

What counts as prompt-framing sensitivity in code review

A model is prompt-framing sensitive when the judgment changes materially even though the underlying code and the substantive evaluation task have not changed. That may show up as a secure snippet being labelled unsafe when the prompt implies hostility, or a risky snippet being treated as acceptable when the prompt sounds benign. Either pattern means the model is responding to framing cues instead of applying a stable code-security standard.

This is especially important when the model is used as an automated reviewer, triage assistant, or decision aid. If the same code receives different outcomes depending on how the question is worded, the model is not reliable enough to act as the sole arbiter for security-sensitive decisions. Teams should treat that as a control weakness, not just an oddity in model behavior.

Good evaluation also looks for partial instability. Sometimes the final label stays the same, but the explanation shifts enough to change how a human reviewer would interpret the risk. That still matters, because it can distort escalation decisions, false positive handling, or the trust a reviewer places in the model’s rationale.

How teams should use the results

Use the test to decide where the model can assist and where it needs guardrails or human review. If the judgment is stable across framings, the model may be useful as a support signal. If the result changes with framing, separate the model’s descriptive help from any authoritative decision path, and require a second control for the final call.

For highest value, compare both output text and any available internal signals, such as routing, classification, or refusal behavior. A visible answer that looks consistent can still hide unstable internal handling, and that instability can matter when prompts arrive from different tools, users, or workflows. Tracking both layers helps teams determine whether the issue is superficial wording drift or a deeper reliability problem.

Risk and Threat Considerations

Prompt-framing sensitivity creates a judgment integrity risk because it lets wording steer the security conclusion. In code review workflows, that can produce inconsistent triage, missed issues, or unnecessary escalations, especially when the model is embedded in a pipeline that assumes its output is repeatable.

Failure mechanism: The model anchors on framing cues such as malicious or benign wording, then shifts its interpretation of the same code sample instead of applying a stable evaluation rubric. That can be amplified when downstream systems trust the model as if it were a deterministic policy engine.

Impact: Teams may approve unsafe code, block safe code, or build false confidence in automated review. Over time, that can degrade both security posture and reviewer trust, because the model becomes difficult to validate or audit consistently.

Practitioner Guidance

What to verify: Keep the code identical, vary only the framing, and verify that the output difference is not caused by hidden changes in prompt length, instructions, or context. If you cannot hold those variables constant, the test is not measuring framing sensitivity cleanly.

Decision rule: If the model’s verdict changes materially across framings, do not use it as a standalone code-security decision source. Treat it as a supporting signal that still needs a stable rule-based or human-reviewed control.

What good looks like: Stable judgments across neutral, adversarial, and benign framings, with explanations that stay aligned to the code evidence rather than the prompt tone. A consistent internal route is a stronger sign than a consistent-sounding narrative alone.

Practitioner takeaway: The real test is whether the model judges the code, not the prompt mood; once framing changes the verdict, the system is no longer dependable enough for autonomous security decisions.

Related resources from NHI Mgmt Group

• How should security teams test whether an LLM can be manipulated into revealing sensitive information?
• How should security teams test whether LLM safety controls still work after harmful generation starts?
• How should security teams test whether prompt injection telemetry can name the compromised agent?
• How should security teams scan LLM application code for prompt injection risks in pull requests?