Structured-schema extraction is an exfiltration pattern where an attacker supplies a formal output template, such as JSON or YAML, to induce a model to fill hidden fields or secrets. It is effective because the request resembles a normal data-processing task, making disclosure feel like formatting rather than theft.
How structured-schema extraction works
Structured-schema extraction is a prompt-abuse pattern that uses the shape of a normal transformation request to elicit hidden data. Instead of asking for a secret directly, the attacker asks the model to “complete” a schema, expand fields, or populate missing values, which can make disclosure feel like routine formatting.
The key distinction is intent plus structure. A harmless transformation task asks for reformatting or extraction from content the user is allowed to see. A malicious one supplies a schema that is designed to coax the model into inventing or revealing content that should not be surfaced, including system instructions, embedded credentials, or private context that the model has access to.
Why it is dangerous
The danger comes from the model’s tendency to satisfy well-formed output requests. If the system allows the model to see hidden context, the attacker can frame the request as data cleaning, record completion, or structured export and induce the model to treat sensitive material as part of the requested payload.
This matters because the attack can bypass user suspicion and some naive content filters. The disclosure may appear as a normal JSON or YAML response, which can hide the fact that the model is exposing information that was never meant to leave the trusted context.
In agentic or tool-enabled systems, the same pattern can become more serious when the model has access to downstream systems, because the structured output can be used as an instruction carrier for later actions. That is why prompt-injection and output-handling controls should be assessed together with tool permissions and data egress paths, as reflected in the OWASP Agentic AI Top 10 and the CSA MAESTRO agentic AI threat modeling framework.
How organisations reduce exposure
Defence starts with understanding what the model can actually see. If hidden prompts, connectors, memory stores, or retrieved documents are in scope, organisations should classify them as sensitive inputs and assume they may be discoverable through shaped output requests. Guardrails are strongest when prompt separation, retrieval scoping, and secret redaction are enforced before the model ever reaches the generation step.
Output controls also matter. Schema validation, allowlisting of response fields, and strict refusal handling reduce the chance that a model can populate unexpected secrets into a supposedly structured response. For broader technical hardening, the OWASP Web Security Testing Guide is a useful companion for testing the application paths that deliver and consume model output.
Detection should focus on prompt patterns that ask for hidden state, reconstruction, or “complete the template” behaviour around sensitive domains. The MITRE ATT&CK Enterprise Matrix remains useful for mapping post-compromise abuse paths when disclosure is followed by credential access or lateral movement.
Practical response when it is suspected
When structured-schema extraction is suspected, treat it as a data-exposure event, not just a bad prompt. Review the transcript, the retrieved context, the tool calls, and the exact fields returned by the model to determine whether the model revealed hidden instructions, secrets, or other protected material.
If the exposed content includes tokens, API keys, session material, or other authentication material, rotate it and invalidate any sessions or trust relationships that may have been derived from it. For identity-bearing material that can persist beyond a single request, the OWASP Non-Human Identities Top 10 is a useful reference point for secret leakage, overprivilege, and long-lived secret risk.
Risk and Threat Considerations
Structured-schema extraction is dangerous because it exploits a trusted interaction style to make disclosure look like ordinary formatting. The same technique can expose hidden prompts, private retrieval content, or secrets without triggering the user’s intuition that they are asking for sensitive data.
Failure mechanism: The attacker uses a formal schema or template to steer the model into completing fields from hidden context, then harvests whatever the model considers consistent with the requested structure.
Impact: The result can be secret leakage, prompt disclosure, compromised sessions, or a foothold for follow-on abuse if the output is later consumed by another system or agent.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP API Security Top 10, MITRE ATT&CK and OWASP Non-Human Identity Top 10 address the attack and risk surface, while CIS Controls v8 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP API Security Top 10 | API8 — Security Misconfiguration | Structured output abuse often exploits weak API and response handling boundaries. |
| Recommendation — Harden response handling so models cannot return unexpected or sensitive fields. | ||
| CIS Controls v8 | CIS-14 — Security Awareness and Skills Training | Users and builders need pattern awareness for prompt-abuse and disclosure tricks. |
| Recommendation — Train teams to recognise schema-shaped prompts that try to elicit hidden data. | ||
| MITRE ATT&CK | T1003 — OS Credential Dumping | Exposed secrets can be used for credential access and follow-on compromise. |
| Recommendation — Hunt for secret exposure paths that could enable credential theft and lateral movement. | ||
| OWASP Non-Human Identity Top 10 | NHI-02 — Secret Leakage | This pattern directly targets hidden secrets and authentication material. |
| Recommendation — Remove secrets from model-reachable context and rotate any exposed credentials immediately. | ||
Practitioner Guidance
What to watch for: Treat schema-completion requests as higher risk when they ask the model to fill missing fields, infer absent values, or return machine-readable output that could carry hidden context into downstream systems. The safest pattern is to keep secrets and system instructions outside the model’s reachable context whenever possible.
Governance implication: Teams should decide who owns output validation, secret handling, and prompt boundary design, because this pattern sits at the intersection of application security, AI safety, and identity protection. If the model can reach sensitive material, the organisation should assume that structured output may become an exfiltration channel.
Related resources from NHI Mgmt Group
- How should teams choose between JSON mode, function calling, and prompt-only extraction for structured data generation?
- What are the signs that a structured data extraction setup is not working well enough?
- What is the difference between JSON mode and function calling for structured extraction?
- Why does schema-constrained generation improve reliability for structured LLM outputs?