AI Red Team Test Plan Template: Scope, Harms, Attack Cases and Reporting

Most AI red team exercises fail before they start because nobody wrote down which system was in scope, which harms mattered, what counted as a finding, or what the testers were allowed to touch. This template is the test plan for one AI red team engagement at [Organisation]: the target and its layers, the harms list, attack case catalogue, success criteria, data handling, finding records and reporting. It covers LLM applications, retrieval systems and AI agents with tools. It is for AI security leads, red team managers and system owners who commission or run the test. Authorisation, deconfliction and stop conditions belong in the Red Team Rules of Engagement Template, which this plan references rather than repeats. The method behind it is explained in the AI Red Teaming Guide.

How to use this template

• Complete one plan per engagement and per system version. Replace text in square brackets, such as [System name] and [N days], with your own values.
• Sign the rules of engagement first. This plan assumes authorisation, contacts and stop conditions are already agreed there.
• Sections 4 and 5 are the core: the harms list sets what matters, the attack case catalogue sets what will be tried. Keep both in a tracker so testers can add cases as they go.
• Use benign canary payloads (a harmless marker string or action) to prove that an injection or exfiltration path works. Do not use real customer data or real harmful content where a canary proves the point.
• Thresholds and severity levels in brackets are NHI Mgmt Group suggestions; set your own in line with your risk appetite.

1. Engagement summary

2. Objectives

1. Identify ways the system can be made to produce harmful, unauthorised or incorrect outputs or actions within the harms list in section 4.
2. Test whether mitigations (guardrails, filters, policy checks, approval gates) work as designed, by testing with and without them where the environment allows.
3. Produce reproducible findings with severity, so the system owner can fix and retest.
4. [Add engagement-specific objectives, for example "confirm the agent cannot send email to external domains without approval".]

Red teaming identifies risk; it does not measure prevalence. Microsoft's guidance on planning LLM red teaming makes the same point: individual examples should not be read as a metric of how widespread a harm is. Systematic measurement follows separately.

3. Scope and test layers

These four areas (model, implementation, infrastructure and runtime behaviour) follow the OWASP GenAI Red Teaming Guide. Record exactly which endpoint each test used: if early rounds use staging or an API, plan a later round on the production interface.

3.1 Tool and permission inventory

3.2 Out of scope

[List systems, data, tools and techniques excluded, for example production payment tools, real customer records, denial of service against shared model endpoints, social engineering of staff.]

4. Harms list

Start with an open-ended round, then turn what testers find into a harms list that guides later rounds. Rate each harm before testing so severity is not decided after the fact.

5. Attack case catalogue

Each case names the harm it tests, the entry point, the technique family, the canary or success signal, and the mitigation expected to stop it. Describe cases at the level below; keep payload detail in the restricted test repository.

For agents, add identity cases: can the agent be induced to use a credential outside its task, act for a different user, or keep access after the task ends? See Red Teaming AI Agents for Identity Abuse.

6. Team, rounds and method

• Team: [names or roles]. Mix security testers with domain experts and people who did not build the system, as Microsoft's planning guidance recommends, so both adversarial and ordinary-user harms surface.
• Round 1, open-ended: [N days] of exploration; output is an updated harms list.
• Round 2, guided: run the attack case catalogue with and without mitigations; assign testers by harm.
• Round 3, regression: rotate assignments; retest fixed findings; run automated suites [tool name] for breadth.
• Automation: automated attack generation adds coverage but needs human review of results. Record the tool, version and configuration.
• Standby: the red team lead stays available throughout for access issues and questions.

7. Success criteria and severity

Record a success rate over [N] attempts for each case, because model outputs vary between runs. A case that succeeds once in twenty is still a finding.

8. Data handling

1. Use synthetic or test data and test accounts. Real personal data only with written approval from [Data protection officer].
2. Store prompts, outputs and screenshots in [restricted repository], with access limited to the team and the system owner.
3. Label reports containing harmful content with a content warning.
4. Delete test artefacts [N days] after closure unless retained as evidence.

9. Finding record

10. Reporting

1. Interim updates: at the end of each round, a short note to [system owner] with top issues, a link to raw findings, and the plan for the next round.
2. Final report: scope as tested, harms list, cases run, findings by severity, mitigations that held, detection coverage, residual risk, and retest plan.
3. Critical findings: reported to [system owner] and [CISO] within [24 hours], not held for the final report.
4. Sign-off: [system owner] accepts or schedules each finding; [risk owner] signs residual risk.

Implementation guidance

Making the plan work

• Write the harms list with the system owner: testers know attacks; owners know which outcomes matter. Both are needed.
• Test the system, not just the model: foundation model evaluations do not cover your prompts, retrieval, tools or credentials.
• Include the SOC: runtime cases show whether attacks are visible. A finding with no alert is two findings.
• Retest after every material change: a new model version, system prompt or tool is a new system for red team purposes. The Continuous Automated Red Teaming (CART) Guide covers keeping regression cases running.
• Small teams: cut rounds, not harms. One guided round against the full harms list beats three open-ended rounds.

Framework mapping

Standards and references

• OWASP: GenAI Red Teaming Guide
• OWASP: Top 10 for LLM Applications 2025
• MITRE ATLAS
• Microsoft Learn: Planning red teaming for large language models and their applications
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: OWASP GenAI Red Teaming Manual Guide, Prompt Injection Guide, AI Red Teaming and Testing Tools Buyer's Guide, Red Team Services RFP Template.