A chat template is the formatting layer that turns user messages, role markers, and system instructions into the input a model processes. In practice, it is part of the trust boundary around the model because it can determine which instructions are treated as authoritative before inference begins.
Expanded Definition
A chat template is the instruction and message formatting layer that turns a conversation into the exact prompt structure a model receives. In NHI and agentic AI systems, that layer matters because it helps determine which content is treated as system-level guidance, which content is user input, and how tool or role boundaries are represented before inference begins.
Definitions vary across vendors, but the core idea is consistent: the template is not the model itself, and it is not just cosmetic formatting. It is part of the control plane around prompt construction, which means errors can change model behavior, weaken instruction hierarchy, or expose hidden context. For governance teams, chat templates sit close to the trust boundary described in the NIST Cybersecurity Framework 2.0 because they influence how inputs are assembled and validated before the model acts on them.
The most common misapplication is treating the template as harmless presentation logic, which occurs when teams let application code, prompts, and role markers drift without review.
Examples and Use Cases
Implementing chat templates rigorously often introduces compatibility and maintenance overhead, requiring organisations to weigh prompt consistency against the cost of version control and regression testing.
- A support agent uses a template that separates system policy from end-user text so the model does not confuse policy instructions with customer requests.
- A developer builds a template for an AI coding assistant that wraps tool calls, conversation history, and safety rules in a predictable order.
- A security team reviews a template after noticing that role tags were being inserted by the application layer instead of the orchestration layer, creating a prompt injection path.
- An enterprise evaluates template changes alongside secret-handling workflows because chat formatting can expose tokens, API keys, or hidden instructions if logs are poorly controlled, a risk pattern highlighted in the Ultimate Guide to NHIs.
- A model provider publishes a reference chat format, while internal teams adapt it to their own agent framework and test whether the adapted structure still preserves instruction priority.
This topic also intersects with message formatting conventions in NIST Cybersecurity Framework 2.0 because predictable control behavior depends on repeatable input handling.
Why It Matters in NHI Security
Chat templates become security-relevant because they can either preserve or collapse the separation between trusted instructions and untrusted content. When a template is misdesigned, an AI agent may execute the wrong instruction, disclose context intended to stay hidden, or ignore guardrails that were assumed to be authoritative. That is especially important when the agent has tool access, access to secrets, or authority to act on behalf of a service account.
NHIMG research shows that 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools, which means template flaws can compound broader exposure paths rather than exist in isolation. The Ultimate Guide to NHIs also notes that 79% of organisations have experienced secrets leaks, with 77% of those incidents resulting in tangible damage, underscoring how small control gaps can produce real operational loss.
Organisations typically encounter the impact only after a prompt injection event, a policy bypass, or an unintended tool action, at which point chat template governance becomes operationally unavoidable to address.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | Agentic AI guidance covers prompt handling and instruction hierarchy risks around chat templates. | |
| NIST CSF 2.0 | PR.DS | Chat templates affect how data is protected and transformed before the model processes it. |
| OWASP Non-Human Identity Top 10 | NHI-06 | Template errors can expose or misroute secrets, tokens, and hidden instructions in AI workflows. |
Review chat templates alongside secret access paths and remove sensitive data from prompt assembly.
