Look for prompts that contain secrets, internal service names, access policy text, or moderation instructions that the application still depends on at runtime. If removing those values would break the security model, the app is relying on prompt secrecy. Good controls survive prompt exposure because they are enforced elsewhere.
How to test whether your LLM app depends on prompt secrecy
Prompt secrecy is a fragile control because it only works while the prompt remains hidden. A stronger test is to ask whether the application still behaves safely if the prompt is exposed to a user, red teamer, or attacker. If the answer becomes unsafe, the prompt is carrying security logic that should live in the application, gateway, policy layer, or downstream control instead.
The quickest way to spot the pattern is to inspect prompts for security-relevant content that is being treated as confidential state. That includes secrets, internal service names, access policy text, moderation rules, and hidden routing instructions. If the prompt is doing more than steering language, it is probably acting as a security boundary.
What makes a prompt “security-dependent” rather than merely informative?
A prompt becomes security-dependent when removing or revealing it would change the system’s trust, authorization, or safety behaviour. In practice, the prompt is being used as a control plane if it contains values the model must not see, or if the app assumes the model will obey instructions only because they are hidden.
This is especially visible in apps that embed policy text in system prompts and then rely on that text to block data exfiltration, restrict tools, or enforce refusal behaviour. Those controls are not durable if the model can be induced to reveal the prompt or if another component reuses the prompt content outside its intended context. The safer design is to make the policy enforceable outside the model, then treat the prompt as advisory rather than authoritative.
Good prompts may still contain context, role definition, and task instructions. The red flag is dependency: if the app breaks, leaks, or over-permits when the prompt is exposed, then prompt secrecy is part of the security model. That is a design smell, not a feature.
How to verify the control is real, not just hidden
Run a simple exposure test: assume the full prompt becomes visible. Then check whether the system still blocks sensitive data, resists policy bypass, and keeps tool use constrained. If the answer depends on the attacker never seeing the prompt, the control is weak.
Look for these signs in code and runtime behaviour: policy text duplicated only inside the prompt, moderation rules that are not independently enforced, privileged instructions that disappear if the prompt is shortened, and any secret value placed in a system message because “the user will never see it.” The moment a prompt contains a secret needed for function, secrecy has become part of availability and safety, not just confidentiality.
For teams using retrieval or agent workflows, the same test applies to connectors, tool instructions, and hidden routing logic. If an attacker can infer the prompt and then influence the app to disclose data, call tools differently, or bypass filters, the prompt is not protecting the system, it is merely obscuring it. For related control patterns in AI apps, see Enterprise AI Copilot Security Guide and Permission-Aware RAG Guide.
Risk and Threat Considerations
Prompt secrecy creates brittle security because the prompt can be reconstructed, leaked through logs, or extracted by prompt injection and other model interaction tricks. Once hidden policy text or embedded secrets become visible, attackers can adapt their inputs to defeat the intended guardrails or reuse exposed values elsewhere.
Failure mechanism: The application uses hidden prompt content as a security dependency, so prompt disclosure removes the very instruction set that was supposed to constrain model behaviour.
Impact: Sensitive instructions, internal identifiers, or embedded secrets can be exposed, and the app may then lose its ability to enforce refusal, routing, or tool restrictions in a reliable way.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST SP 800-53 Rev 5 and OWASP ASVS set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | IA-5 — Authenticator Management | Prompt secrecy fails when secrets are embedded in prompts and need lifecycle control. |
| AC-6 — Least Privilege | Hidden prompt instructions often try to enforce excess access instead of least privilege. | |
| Recommendation — Keep secrets out of prompts and manage them through dedicated secret controls. Enforce least privilege in the application and tooling, not inside the prompt. | ||
| OWASP ASVS | V15 — Secure Coding and Architecture | The question is about whether security depends on hidden prompt logic rather than sound architecture. |
| Recommendation — Design security controls so they remain effective if prompts are exposed. | ||
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Prompt secrecy often hides privilege and tool constraints in agentic apps. |
| Recommendation — Move identity and privilege decisions out of prompts and into enforceable policy. | ||
| OWASP Non-Human Identity Top 10 | NHI-02 — Secret Leakage | Prompts that contain secrets create secret leakage risk when exposed or logged. |
| Recommendation — Remove secrets from prompts and rotate any exposed values immediately. | ||
Practitioner Guidance
What to verify: Confirm that every security-relevant control is enforced outside the prompt. If the prompt were logged, echoed, or revealed during testing, the application should still deny unsafe actions and protect sensitive data.
Common mistake: Treating system prompts as a vault. A prompt may guide behaviour, but it should not be the only place where policy, authorization, or secret material exists.
Decision rule: If the prompt contains anything that would be harmful to reveal, remove it or replace it with a reference to an external control. If exposure changes the security outcome, the app is relying on prompt secrecy too much.
Practitioner takeaway: The right test is not whether the prompt is hidden, but whether the application remains secure after the prompt is assumed public.
Related resources from NHI Mgmt Group
- How can security teams tell whether an LLM workflow is high risk for indirect prompt injection?
- How can security teams tell whether a project prompt is being abused?
- How can security teams tell whether a mobile app is collecting too much identity-linked data?
- How can teams tell whether prompt security is actually working?