Most generative AI red team exercises still end as a list of jailbreak screenshots, which tells an organisation little about whether its controls actually hold. The OWASP GenAI Security Project published the GenAI Red Teaming Manual Version 1.0 on 20 October 2026 to give testers a structured, reproducible method that covers models, applications, multimodal interfaces and AI agents, and that ends in measurable findings and remediation. This guide explains how the manual is organised, what each of its eight phases asks for, how it treats agents and identity, how to turn it into a programme with evidence and metrics, and how it relates to OWASP's earlier GenAI Red Teaming Guide and to other frameworks. It is for AI red team leads, AppSec and offensive security teams, and the security owners who commission and act on their work. General red teaming concepts and tools are covered in the AI Red Teaming Guide; this page is about applying the OWASP manual specifically. It reflects the manual as read on 27 October 2026; as a version 1.0 document it is likely to be revised.
Key takeaways
- Eight phases, end to end. The manual runs from planning and scoping through reconnaissance, surface mapping, exploitation, persistence and escalation, post-exploitation, evaluation and reporting, to post-engagement remediation.
- Boundaries must live outside the model. The manual states that "Enforceable security boundaries must exist in services that the model cannot bypass or rewrite", and treats prompt instructions and classifiers as defence in depth, not enforced controls.
- Gate on control failures, not refusal rates. Threat hypotheses are turned into test oracles and CI gates tied to deterministic control failures, and over-refusal is itself a mandatory test oracle.
- Agents get explicit treatment. Surface mapping covers tool and agent authority, delegation chains, confused deputy paths, per-action authorisation and kill switches.
- Taxonomy editions matter. Findings are tagged with OWASP LLM and Agentic identifiers including the edition year, and the manual reminds teams that a taxonomy is not itself a compliance standard.
What the manual is and how it differs from the 2025 guide
OWASP's GenAI Security Project has two red teaming documents that are easy to confuse. Read them together but use them for different jobs.
| Aspect | GenAI Red Teaming Guide | GenAI Red Teaming Manual Version 1.0 |
|---|---|---|
| Published | 22 January 2025 | 20 October 2026 |
| Purpose | Holistic orientation for security professionals, engineers, risk managers and leaders | Practical methodology for running engagements |
| Organising idea | Four areas: model evaluation, implementation testing, infrastructure assessment and runtime behaviour analysis | Eight engagement phases with deliverables per phase |
| Best used for | Explaining red teaming to stakeholders and deciding what to cover | Planning, executing and reporting a specific exercise |
| Relationship | The guide's resource page lists the manual as an additional resource; OWASP does not state that one supersedes the other | |
The manual sits alongside other GenAI Security Project outputs that help with the commercial side: the Vendor Evaluation Criteria for AI Red Teaming Providers and Tooling v1.0 (February 2026), which aims to separate meaningful adversarial testing from "jailbreak-only" offerings, and the Solutions Landscape Red Teaming Taxonomy (June 2026), a common classification of red, blue and purple team capabilities. If you are buying testing services or tools, see the AI Red Teaming and Testing Tools Buyer's Guide.
Cross-cutting guidance
- Access posture: choose black-box, grey-box or white-box testing according to the adversary being emulated. The manual cautions that model identification should be treated as a hypothesis unless independently confirmed, and that a model's self-reported identity is unverified.
- Lifecycle mapping: red teaming applies across scoping, model selection, customisation, development integration, deployment and continuous improvement, with roles and expected outcomes for each stage.
- Tester welfare: an adversarial testing advisory recognises that testers may handle harmful content and calls for psychological support and clear policies.
The eight phases
| Phase | Focus | Typical outputs |
|---|---|---|
| 1. Planning and Scoping | Stakeholders, team, scope, rules of engagement, environments, access, evidence handling, escalation | Rules of engagement, test accounts, escalation triggers, agreed deliverables |
| 2. Reconnaissance and Fingerprinting | Public artefacts, model and component identification, supply chain | AI bill of materials, endpoint inventory, guardrail notes, artefact hashes |
| 3. Surface Mapping | Orchestration, trust boundaries and data flows, threat modelling | Surface-map matrix, threat register, attack trees or misuse cases, test oracles, CI gate definitions |
| 4. Exploitation and Impact | Testing across a catalogue of about 40 risk domains, direct AI vulnerabilities and agentic attack vectors | Reproducible findings with evidence |
| 5. Persistence and Escalation | Whether access or influence survives and grows | Escalation paths |
| 6. Post-Exploitation | Blast radius, severity and risk scoring, containment | Scored findings |
| 7. Evaluation and Reporting | Evaluation methods, reproducibility, coverage, metrics, red and blue integration | Audience-specific reports |
| 8. Post-Engagement and Remediation | Fix verification and follow-up | Remediation verification |
Phase 1: Planning and scoping
- Stakeholders: system owners, ML engineering, AppSec, incident response, legal and the SOC, with a team that mixes red teamers, ML engineers, data scientists and ethics or policy expertise.
- Target types: the manual distinguishes chatbots, agentic platforms, enterprise RAG and API architectures, self-hosted open-weight models and edge devices, because each changes what can be tested and how.
- Environments: sandbox, staging or production, each with constraints. In production, keep attacks non-destructive, use tagged accounts, throttle request rates and define abort conditions.
- Access and logistics: provision test accounts across tenants and roles (needed to test cross-tenant and privilege issues), arrange allowances such as WAF allowlisting, and tag test traffic so the SOC can tell it apart from real attacks.
- Evidence and escalation: store evidence in encrypted, access-controlled repositories with retention and purge rules, and agree critical escalation triggers in advance, such as remote code execution, live personal data exposure or uncontrolled agent loops.
- Training data hygiene: do not feed attack data into training pipelines automatically.
- Deliverables: an executive summary, technical walkthroughs, proof-of-concept replay scripts and a remediation verification timeline.
The Red Team Rules of Engagement Template can be adapted to capture these points.
Phase 2: Reconnaissance and fingerprinting
- Public artefacts: papers, documentation, model cards, API documentation, terms of service and forums.
- Refusal-pattern probes: useful heuristics for identifying models and guardrails, never proof.
- AI bill of materials: build one, and verify provenance, digests and version pinning of models and components.
- Serialised models: inspect statically in a quarantined environment and never execute them; the manual recommends tools such as pickletools with resource limits rather than unpickling untrusted artefacts.
- Scanners are not enough: do not rely solely on software composition analysis, since the manual notes vendors often close findings without CVEs.
Phase 3: Surface mapping
This phase is where the manual is most useful to defenders, because it forces an explicit map of what the model can reach.
- Orchestration: probe non-aggressively to infer model type and interfaces, and use canaries to test for exposed system prompts.
- Trust boundaries and data flow: map ingress and egress, mediators, storage, secrets, modalities, tools and agents, RAG stages, identity, CI/CD, observability, human-in-the-loop controls and environment boundaries.
- Threat modelling: convert the map into testable hypotheses tagged with OWASP LLM and Agentic identifiers, prioritised by local business and safety risk. AIVSS or another scoring system may be used alongside the tags.
Phase 4: Exploitation and impact
The manual's catalogue of roughly 40 testing domains includes sensitive data leakage, malicious code generation, cross-tenant leakage, unauthorised actions, unbounded consumption (denial of wallet), over-refusal and frontier cyber-offence capability. Its section on direct AI vulnerabilities gives concrete test designs:
| Area | What the manual asks testers to do | Defensive lesson |
|---|---|---|
| Data, memory and state poisoning | Check whether an injected instruction persists in memory, then test recall in a clean session; test which other users can retrieve poisoned content | Segment memory and RAG stores per user and task |
| Feedback loop exploitation | Verify whether feedback actually feeds an update process before claiming impact | Review what user feedback can change |
| Vector and embedding weaknesses | Test whether ingestion accepts malicious documents and whether the deployed retriever surfaces them; record embedding model, access level and reconstruction metric for inversion tests | A document's presence in a knowledge base is not evidence it is trustworthy |
| Model extraction and inversion | Report membership inference with false positive and false negative rates | Rate limits and monitoring on inference APIs |
| Credential extraction | Test direct requests, credentials as fields in generated output, and system prompt extraction; the manual cites a study in which 32 of 52 applications disclosed a planted credential, mostly through system prompt extraction | Keep credentials out of model-visible context and supply them to the tool executor out of band |
The credential finding is the clearest identity lesson in the manual: output filters that look for literal secrets are a supplementary control, because encoded or reformatted secrets evade them. See the OWASP LLM Top 10 LLM07 System Prompt Leakage Guide.
Phases 5 to 8
The later phases cover privilege escalation and persistence, blast radius, severity and risk scoring and containment, evaluation methods, reproducibility controls, coverage tracking, metrics and red and blue team integration, and remediation follow-up. Reporting is audience-specific: engineers need reproducible prompts and payloads, the SOC needs timestamps and headers to build detections, and governance teams need risk summaries. Read these sections in the manual itself before scoring or reporting an engagement; the structure above is the one to follow.
Testing agents and identity with the manual
Section 3.2.7 of the manual addresses tool and agent authority, delegation chains, confused deputy paths, per-action authorisation and kill switches. In practice this means each engagement against an agent should answer a fixed set of identity questions:
- Whose authority does each tool call use? Map every tool to the credential it runs with and test whether a low-privilege user or injected content can trigger it. See the Confused Deputy in AI Agents Guide.
- Does authorisation happen per action? Test whether permissions are checked on each call or only at session start, and whether claimed scopes in a token are trusted without server-side checks.
- Do delegation chains attenuate? In multi-agent systems, test whether a sub-agent receives broader access than its task needs.
- Is cross-tenant isolation real? Use the multi-tenant test accounts from Phase 1 to attempt retrieval and actions across tenants.
- Can the agent be stopped? Verify that the kill switch works while an agent is mid-task and that it revokes the agent's credentials, not just its UI.
- Where can data leave? Map egress paths and test exfiltration through network calls, rendered URLs and communication tools. See the Agent Egress Control and Network Isolation Guide.
For an identity-focused attack plan built on these questions, see Red Teaming AI Agents for Identity Abuse.
Running a programme on the manual
- Pick the targets. Rank AI systems by data sensitivity, autonomy and exposure, and start with agents that hold write access or customer data.
- Agree rules of engagement. Use Phase 1 to fix environments, abort conditions, escalation triggers, evidence handling and test account provisioning, signed off by system owners, legal and the SOC.
- Build the surface map with the owners. Phase 3 is most accurate when engineers walk the red team through tools, identities and data flows; it also produces the threat register that remediation will track.
- Define oracles before testing. For each hypothesis decide what a deterministic control failure looks like (an unauthorised tool call executed, cross-tenant data returned, a credential disclosed), and include over-refusal checks so fixes do not break legitimate use.
- Test, record, replay. Keep prompts, payloads, model versions, configuration and timestamps so every finding can be replayed by engineers and turned into a detection by the SOC.
- Score and report by audience. Tag findings with edition-qualified OWASP identifiers and the chosen severity method; deliver executive, engineering and SOC views.
- Turn findings into CI gates. Convert reproducible findings into regression tests that run on every model, prompt or tool change. See the Continuous Automated Red Teaming (CART) Guide.
- Verify remediation. Re-test against the remediation verification timeline agreed in Phase 1 and close findings only on evidence.
Scaling by organisation size
| Situation | How to apply the manual |
|---|---|
| Small team, one or two AI applications | Use Phases 1 to 4 and 7 as a checklist for an annual or pre-launch engagement, often with an external provider evaluated against OWASP's vendor criteria |
| Product company shipping agents | Run Phase 3 surface mapping in design reviews, keep a standing threat register, and convert findings to CI gates |
| Large enterprise with many AI systems | Central AI red team using the manual as the house method, tiered depth by system risk, purple team sessions with the SOC on Phase 7 outputs |
Evidence, detection and metrics
- Evidence pack per engagement: rules of engagement, AI bill of materials, surface-map matrix, threat register, test oracles, replay scripts, findings with edition-qualified tags, and remediation verification records. These serve as assurance evidence for AI governance frameworks.
- Detection value: because test traffic is tagged, the SOC can measure which attacks it detected. Treat undetected successful attacks as detection engineering backlog.
| Metric | What it shows |
|---|---|
| Deterministic control failures per engagement, by OWASP tag | Where real boundaries break, independent of refusal rates |
| Share of AI systems with a current surface map and threat register | Coverage of the method |
| Findings converted to CI regression tests | Whether fixes stay fixed |
| Median time from finding to verified remediation | Programme effectiveness |
| Over-refusal rate on legitimate tasks after fixes | Whether remediation harmed usability |
| Share of tagged test attacks detected by the SOC | Detection coverage |
Mapping to other frameworks
| Framework | Where the manual fits |
|---|---|
| OWASP Top 10 for LLM Applications and Top 10 for Agentic Applications | Tagging taxonomy for hypotheses and findings, with edition recorded |
| MITRE ATLAS | One of the attack taxonomies the manual cites; relevant techniques include AML.T0051 LLM Prompt Injection, AML.T0054 LLM Jailbreak, AML.T0085 Data from AI Services (including AML.T0085.003 System Instructions), AML.T0020 Training Data Poisoning, AML.T0080 AI Agent Context Poisoning, AML.T0053 AI Agent Tool Invocation |
| NIST AI RMF and its Generative AI Profile | Governance input listed by the manual; red team results feed measurement and management of AI risk. See the NIST AI RMF Guide |
| ISO/IEC 42001 | Governance input; engagement evidence supports AI management system assurance |
| EU AI Act and GDPR | Governance inputs listed by the manual for regulated systems and personal data handling |
| NIST SP 800-53 Rev. 5 | CA-8 Penetration Testing, CA-8(2) Red Team Exercises, SA-11 Developer Testing and Evaluation, RA-5, SR-4 Provenance (for the AI bill of materials) |
| NIST CSF 2.0 | ID.RA-01 vulnerabilities identified and recorded; ID.IM-02 improvements from security tests and exercises |
Common failure patterns
- Measuring refusal rates instead of control failures: a lower jailbreak rate says nothing about whether an agent can still call a privileged tool for an unauthorised user.
- Skipping surface mapping: going straight to prompts misses the tools, identities and data flows where the serious findings are.
- Single-tenant, single-role test accounts: without accounts across tenants and roles, cross-tenant leakage and privilege issues cannot be tested.
- Untagged test traffic: the SOC either chases the red team or, worse, learns to ignore similar alerts.
- Undated taxonomy tags: findings tagged with an identifier but no edition become ambiguous as the OWASP lists are revised.
- Findings without replay: screenshots instead of replay scripts mean engineers cannot reproduce, fix and regression-test the issue.
Lessons from incidents
- Research disclosures are red team findings in public: ForcedLeak chained indirect prompt injection through a public form, the agent's own CRM access and an expired allowlisted domain. A Phase 3 surface map covering ingress (public forms), agent authority and egress would have exposed the chain before the researchers did.
- Zero-click paths need system-level tests: EchoLeak worked because an external email was retrieved as context and data left through a URL. Model-only testing would not have found it; end-to-end tests with realistic data flows would.
Practitioner checklist
- Read the manual alongside OWASP's 2025 GenAI Red Teaming Guide and use each for its purpose: orientation versus engagement method.
- Agree rules of engagement covering environments, abort conditions, escalation triggers, evidence storage and retention before testing starts.
- Provision tagged test accounts across tenants and roles and tell the SOC how to recognise test traffic.
- Build an AI bill of materials and inspect model artefacts statically in quarantine.
- Produce a surface-map matrix and threat register covering tools, identities, data flows, egress and human approvals.
- Define deterministic test oracles and over-refusal checks before running attacks.
- Test agent authority, per-action authorisation, delegation chains, cross-tenant isolation and kill switches explicitly.
- Keep credentials out of model-visible context and test for credential extraction through system prompt leakage.
- Tag findings with edition-qualified OWASP identifiers and deliver engineer, SOC and governance views with replay scripts.
- Convert findings into CI regression gates and verify remediation against an agreed timeline.
Standards and references
- OWASP GenAI Security Project: GenAI Red Teaming Manual Version 1.0
- OWASP GenAI Security Project: GenAI Red Teaming Guide
- OWASP GenAI Security Project: Vendor Evaluation Criteria for AI Red Teaming Providers and Tooling v1.0
- OWASP GenAI Security Project: Solutions Landscape Red Teaming Taxonomy
- OWASP: Top 10 for LLM Applications
- MITRE ATLAS: LLM Prompt Injection
- NIST: SP 800-53 Rev. 5
- NIST: Cybersecurity Framework 2.0
Related NHI Mgmt Group resources: AI Red Teaming Guide · Red Teaming AI Agents for Identity Abuse · Continuous Automated Red Teaming (CART) Guide · OWASP Top 10 for Agentic Applications Guide · AI Agent Threat Modelling Guide