Yes. Combine both because they expose different failure modes. AI red teaming is useful for fast, large-scale adversarial probing of model behavior, prompt injection, jailbreaks, data leakage, and unsafe tool use. Traditional manual testing catches workflow gaps, privilege misuse, edge cases, and control failures across IAM, NHI, cloud, and application layers. Together they give broader coverage, better prioritisation, and a more realistic view of operational risk.
Why Combine AI Red Teaming and Manual Testing?
Use ai red teaming to pressure-test model behavior, prompt handling, jailbreak resistance, and unsafe tool invocation at speed and scale. Use traditional manual testing to follow the operational path the model sits inside, including workflow design, privilege boundaries, approval steps, and interactions with cloud, application, and identity controls. The value is not redundancy, it is complementary coverage.
AI red teaming is strongest when you want broad adversarial coverage against the model or agent itself. Manual testing is strongest when you need a human to reason through business logic, chained permissions, fallback paths, and the ways a real operator or attacker could turn a weak model decision into an unsafe action. The two methods answer different questions about the same system.
They also differ in what they tend to miss. Automated or model-focused testing can reveal how the system reacts under pressure, but it may not fully surface control failures that only appear when a workflow crosses team boundaries, calls internal services, or relies on a service account with broader reach than intended. Manual testing can expose those joins, but it is slower and less exhaustive against large prompt spaces and variant adversarial inputs.
What Each Method Catches That the Other Can Miss
AI red teaming is especially useful for detecting prompt injection, instruction hierarchy failures, hidden data disclosure, unsafe content generation, and tool-use abuse. It is the better fit when you need repeatable adversarial probing across many scenarios and want measurable evidence of where the model or agent can be manipulated.
Manual testing is better for uncovering permission mistakes, approval bypasses, privilege escalation paths, unsafe defaults, brittle exception handling, and edge cases in integrated systems. It can also reveal whether the control environment around the model is actually enforceable, which matters when a model is connected to operational systems rather than isolated in a lab.
That distinction is why a combined approach produces better prioritisation. Findings from AI red teaming can tell you where the model itself is brittle, while manual testing can show whether that brittleness matters in production because the surrounding workflow gives it real consequences.
How to Use Both Without Creating False Confidence
The most useful approach is to treat AI red teaming and manual testing as one programme, not two unrelated exercises. Start with the model, agent, and tool surfaces that are most likely to create unsafe outputs or unauthorized actions, then validate the end-to-end workflow to see whether a bad model response can actually become a bad business outcome.
For AI agents, that means testing both the conversational layer and the operational layer. A jailbreak that merely produces an unsafe suggestion is serious, but a jailbreak that triggers an approved action, leaks data, or abuses delegated access is materially more important. Manual review is what often shows whether the surrounding guardrails are real or only present in the design document.
This is also where AI Red Teaming Guide is useful, because it frames scoping, metrics, and reporting in a way that pairs well with conventional security testing. For adversarial testing focused on agent behavior and misuse of access, Red Teaming AI Agents for Identity Abuse helps connect model weaknesses to privilege, delegation, and exfiltration paths.
Risk and Threat Considerations
Relying on only one test style creates blind spots. AI red teaming can identify how a model fails under adversarial input, but it may not show whether that failure can reach production systems through overbroad permissions, weak approvals, or reused credentials. Manual testing can reveal workflow and control failures, but it may under-sample the prompt patterns and adversarial variations that make AI systems unsafe at scale.
Failure mechanism: An attacker or internal user first manipulates the model or agent response, then uses weak integration controls, excessive privilege, or permissive tool access to turn the output into real impact. The weakness is often not a single bug, but the combination of model behavior and downstream authority.
Impact: The result can be unauthorized data access, unsafe actions in connected systems, hidden policy bypasses, or a false sense of assurance because each testing method looked acceptable on its own.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST SP 800-53 Rev 5, OWASP ASVS and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI02 — Tool Misuse | AI red teaming here probes unsafe tool use and action abuse. |
| ASI03 — Identity & Privilege Abuse | The question explicitly includes privilege misuse across workflows and agents. | |
| Recommendation — Test agent tool calls for unsafe or unauthorized actions. Constrain agent permissions and validate privilege boundaries. | ||
| OWASP Non-Human Identity Top 10 | NHI-05 — Overprivileged NHI | Manual testing should expose excessive machine or service access around AI systems. |
| Recommendation — Review and reduce non-human access that exceeds task needs. | ||
| NIST SP 800-53 Rev 5 | AU-6 — Audit Review, Analysis, and Reporting | Red teaming and manual testing both need evidence that findings are observable and reviewable. |
| IA-5 — Authenticator Management | Workflow abuse often depends on exposed or mismanaged credentials and tokens. | |
| Recommendation — Correlate test results to logs and control evidence. Rotate and manage authenticators that enable tool or service access. | ||
| OWASP ASVS | V8 — Authorization | Manual testing must validate permission checks and business-logic access paths. |
| Recommendation — Verify authorization checks on every sensitive action path. | ||
| NIST AI RMF | GV.2 — ? | AI testing programs need governance, accountability, and risk-based evaluation. |
| Recommendation — Establish accountable oversight for AI testing decisions. | ||
Practitioner Guidance
What to prioritise: Test the highest-consequence paths first, especially where a model can trigger actions, retrieve sensitive data, or influence privileged workflows. If an agent can call tools or services, the test must include what happens when the model is wrong, not just when it is compliant.
What to verify: Confirm that red-team findings map to actual control failures, not just interesting prompts. A useful result is one that changes a decision about access, approval, logging, containment, or exception handling.
Practitioner takeaway: AI red teaming finds how the system can be bent, manual testing shows where that bend becomes harm, and the strongest assurance comes from testing both the model and the operational path it can influence.
Related resources from NHI Mgmt Group
- Why do AI systems need red teaming beyond traditional penetration testing?
- What breaks when AI red teaming is treated like traditional penetration testing?
- How should organisations compare automated AI red teaming with human-led testing?
- What is the difference between traditional penetration testing and AI red teaming?