Should organisations keep humans in the pentesting loop even when agents outperform them on some tasks?

Why Humans Still Matter When Agents Outperform on Execution

Agentic systems can outperform people on repeatable execution, branch exploration, and evidence gathering, but that does not remove the need for human judgment. Pentesting is not just task completion, it is a security decision process. The human role is to define what is worth testing, interpret ambiguous evidence, and decide when a finding is material enough to change commercial or operational risk.

That distinction matters because faster execution can widen coverage without improving decision quality. A machine can enumerate, probe, and correlate at scale, but it cannot reliably decide which findings should alter scope, escalation, disclosure timing, or acceptance of residual risk. In AI Agent Authorisation Guide, the key pattern is task-scoped and per-action authority, which maps directly to testing because an agent should only be allowed to execute within the objectives a human has set.

Human-in-the-loop testing is therefore best understood as a division of labour, not a compromise. Agents can do the high-volume work of path discovery, proof gathering, and repeated validation, while humans retain accountability for test intent, acceptable collateral impact, and the interpretation of borderline cases. That is especially important when a pentest touches production-like systems, regulated data, or business-critical workflows.

What Changes in the Testing Model

The main change is from manual execution to human-directed orchestration. The human no longer needs to perform every probe or evidence collection step, but still needs to own the attack story, the scope boundary, and the final conclusion. This is closer to supervised adversarial testing than to autonomous offensive action.

That model also changes what “good” looks like. Success is not simply the number of checks completed. It is whether the testing process produces defensible evidence, stable reproduction steps, and conclusions that a security leader can act on. The best use of agents is to increase breadth and consistency, then hand the human the curated result set that still requires expert triage.

Where the test involves autonomous tooling, the control problem becomes one of authorization and containment. NHIMG’s Privileged Access Management Guide is useful here because it frames just-in-time access, zero standing privilege, and session controls as the practical boundary around high-impact actions. The same logic applies when a testing agent is given powerful capabilities.

Where Human Oversight Adds the Most Value

Human oversight is most valuable in three places. First, objective setting: deciding what a test is trying to prove, and what outcomes would justify remediation. Second, interpretation: separating real exposure from noisy or context-free findings. Third, escalation: deciding when an issue is severe enough to pause a test, notify owners, or widen the response.

This is where agentic security guidance becomes relevant. The Agentic AI Security Guide highlights the need to control tool use, orchestration, and identity, which mirrors the core pentest question: how much power should a machine have before a human reviews the next step? For a testing program, the answer should be “less power than production attackers would want, more power than a manual tester can efficiently wield.”

Human review is also the point where business context enters the decision. A finding that is technically exploitable may be low priority if the asset is isolated and low value, while a weaker issue may deserve immediate attention if it affects a crown-jewel process or a control relied upon for audit evidence. Machines can surface the candidate issues; humans decide which ones matter.

Risk and Threat Considerations

When humans are removed too far from the loop, the main risk is not that testing stops working, it is that testing stops being trustworthy. Autonomous execution can create false confidence, over-scoping, or uncontrolled activity if the system is allowed to pursue findings without clear bounds, especially when it can chain actions across tools or environments.

Failure mechanism: The testing agent expands its own search or action space, produces plausible but unvetted evidence, or takes a path that exceeds the intended scope, making the final result hard to trust or safely use.

Impact: The organisation may mis-rank risk, overlook business context, or expose systems and data to unnecessary probing, and the resulting report may be operationally misleading even if individual technical steps were successful.

Practitioner Guidance

What to prioritise: Keep humans responsible for objectives, boundaries, and final triage, and let agents handle the repetitive branches, checks, and evidence collection. If a task can change scope, touch sensitive systems, or trigger remediation decisions, it needs human approval before execution.

What to verify: Confirm that every automated action is attributable, that the test plan defines stop conditions, and that the evidence set is sufficient for a human reviewer to reproduce the finding without trusting the agent’s judgment alone. NHIMG’s AI Agent Observability, Audit and Incident Response Guide is a useful companion for the logging and attribution side of that verification.

Practitioner takeaway: The goal is not to keep humans doing every action, it is to keep humans owning the decisions that make a pentest operationally safe, commercially meaningful, and defensible.

Related resources from NHI Mgmt Group

• When should organisations keep coding tasks human instead of delegating them to agents?
• How can organisations prevent AI agents from becoming overprivileged?
• How can organisations govern AI agents that use service accounts and tokens?
• Should organisations keep humans in the loop for AI-driven remediation?