AI is changing offensive security testing, but context still matters

At a glance

What this is: This is an analysis of how AI is changing offensive security testing, with the key finding that AI improves speed, scale, and cost but still struggles with business context, human behaviour, and chained attacks.

Why it matters: It matters because security teams need to decide where AI can extend offensive coverage and where human-led testing remains necessary, especially when AI systems themselves become part of the attack surface.

By the numbers:

• Traditional, human-led penetration testing takes months and costs in the range of $10K to $30K per test.
• With AI-based vulnerability discovery, a pen test can be conducted and documented in days for less than $10K.
• When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes and as quickly as 9 minutes in some cases.

👉 Read Xbow's analysis of how AI is changing offensive security testing

Context

AI is now affecting offensive security in two directions at once. It changes how testing is conducted by accelerating reconnaissance, attack simulation, and reporting, and it changes what must be tested by bringing AI systems, prompts, and model-connected workflows into scope. For identity and access teams, that means the attack surface increasingly includes the credentials, tokens, and service access behind AI-enabled workflows.

The core governance gap is not whether AI can help test more broadly, but whether organisations can still distinguish automation from judgement. Offensive security still depends on interpreting chained failures, business logic, and human response patterns, which is why AI-assisted testing should be treated as coverage expansion, not a replacement for expert-led validation.

Key questions

Q: How should security teams govern AI agents used for offensive testing?

A: Treat offensive AI agents as distinct workloads with explicit ownership, scoped tools, and logged approvals. Give them only the environments, credentials, and actions needed for authorised testing. Separate research targets from production systems, and review retries, data access, and output handling as part of standard governance, not as an afterthought.

Q: Why do AI tools still miss real attack chains in pentesting?

A: AI often identifies individual weaknesses, but many real compromises depend on how several weaknesses combine across systems, roles, and processes. Those chains require environmental judgement and business context. Without that context, AI can understate risk by treating each issue as isolated instead of part of a path to compromise.

Q: Why do AI systems create identity and access risk beyond traditional AppSec?

A: Because AI systems often act through delegated access. When a model can use tools, retrieve data, or trigger actions, it becomes a runtime decision-maker with privileges that can be misused through prompt injection, poisoned context, or overbroad permissions. That is an identity problem as much as a code problem.

Q: What should teams do when offensive testing includes AI workloads?

A: Test the model interface, the tools it can call, the identities it uses, and the data it can reach. That approach reveals whether a prompt injection or workflow abuse can escalate into broader access. The goal is to validate the full runtime path, not just the model’s output quality.

Technical breakdown

How AI changes offensive security coverage

AI speeds up the mechanics of offensive testing by automating reconnaissance, large-scale attack simulation, phishing generation, and documentation. In practice, that means a testing programme can move from point-in-time assessments toward repeatable or continuous coverage. This is useful in large cloud and application estates where manual testing cannot keep up with change. The architectural shift is that AI becomes a force multiplier for test execution, not the deciding authority on exploitability or business impact.

Practical implication: use AI to increase test cadence and breadth, but keep human review on exploitability, prioritisation, and remediation decisions.

Why AI struggles with chained attacks and human context

AI tools can identify common weaknesses, but they still struggle to chain multiple findings into a coherent attack path when the path depends on business logic, environmental nuance, or human behaviour. Social engineering tests are especially sensitive to context because the meaning of a response depends on role, timing, and organisational process. This is where offensive testing remains more than pattern matching. The value of human testers is not just creativity, but the ability to judge which weak signals actually combine into a real-world compromise path.

Practical implication: reserve human-led testing for business-logic exploits, multi-step attack chains, and scenarios where behaviour matters as much as vulnerability.

Why AI systems now belong inside offensive security programmes

AI is no longer just a tool used during offensive testing. It is also a target that needs to be tested, including prompt injection, data poisoning, tool misuse, and other model-facing attack paths. That matters because AI systems often sit inside broader identity and access flows, calling tools, reaching data, and acting on behalf of users or operators. In that sense, AI security and IAM now intersect at runtime, where access scope, delegation, and secret handling determine how much damage a compromised workflow can do.

Practical implication: add AI workflows, delegated tools, and their underlying credentials to offensive security scope, not just the model interface.

Threat narrative

Attacker objective: The attacker seeks to turn a small weakness into broader access across applications, credentials, or AI-connected workflows before defenders can recognise the full attack path.

1. Entry begins when an attacker targets exposed credentials, weak AI workflow controls, or attacker-facing application flaws to gain a foothold in the environment.
2. Escalation follows when the attacker uses that foothold to probe linked systems, abuse overbroad access, or chain weaknesses that automated testing may miss.
3. Impact occurs when the attacker reaches sensitive data, privileged workflows, or AI-enabled business processes and turns the initial foothold into operational compromise.

NHI Mgmt Group analysis

AI is best understood as an offensive security amplifier, not an autonomous tester. It can accelerate reconnaissance, simulation, and evidence collection, but it cannot reliably replace human judgement where business logic and chained compromise matter. That distinction matters because many security programmes still optimise for tool output instead of decision quality. Practitioners should treat AI as a throughput gain inside a human-governed testing process, not as a substitute for expertise.

Offensive testing is moving closer to identity and access governance. Once AI systems are part of the attack surface, the real question becomes who or what can call tools, access data, and act on behalf of a workflow. That makes secrets management, workload identity, and delegated access visible concerns in offensive security. The result is a stronger need to connect red-team findings to IAM and NHI controls rather than leaving them in separate operational silos.

AI exposes a coverage gap, not just a capability gap. Manual pen tests are expensive and episodic, while AI can extend testing to more assets and more frequent cycles. The governance issue is that speed can create false confidence if teams mistake faster output for deeper assurance. Practitioners should decide which tests need scale, which need nuance, and which need both.

Business context is the named concept that still separates testing from exploitation. AI can enumerate weaknesses, but real attacks often depend on sequencing, timing, and operational meaning. That is why the hard problems in offensive security remain the ones closest to the business process. Security leaders should measure whether their programme can actually detect chained abuse, not just single-control failures.

AI-assisted offensive security will increasingly drive convergence between application, cloud, and identity teams. The more testing touches prompts, secrets, and delegated access, the less useful it becomes to treat identity as a downstream remediation domain. This makes cross-functional ownership the practical outcome, because the same failure can present as an app issue, a cloud issue, or an identity issue depending on where the attacker enters.

What this signals

Offensive security programmes that adopt AI will need to prove more than throughput gains. The practical test is whether faster testing actually improves detection of chained compromise, identity abuse, and workflow escalation across the parts of the environment that matter most.

Business context gap: This is the boundary where AI-assisted testing becomes a governance issue. Teams should expect more findings involving delegated access, connected tools, and service identities, which means offensive security results will increasingly need to feed identity and access remediation workflows, not just vulnerability tracking.

For practitioners

• Define the scope of AI-assisted testing Map which parts of the estate AI can test efficiently, and which scenarios still require human-led analysis because they depend on business logic, chained exploits, or behavioural judgement.
• Add AI workflows to offensive security scope Include prompts, tool-calling agents, connected SaaS accounts, and the underlying credentials or tokens in red-team and pentest coverage so AI exposure is tested as part of the attack path.
• Separate speed metrics from assurance metrics Track how quickly tests are executed, but also measure whether the programme is finding multi-step attack paths, not only isolated issues.
• Tie findings back to IAM and NHI controls Route offensive-security findings involving service accounts, tokens, or delegated access into identity remediation queues so remediation closes the actual access path rather than just the surface symptom.

Key takeaways

• AI makes offensive security broader and faster, but not automatically smarter.
• The hardest testing problems still involve chained attacks, human behaviour, and business context.
• Once AI workflows are in scope, identity and access controls become part of offensive security governance.

Key terms

• Offensive Security: Offensive security is the practice of simulating attacker behaviour to uncover weaknesses before real adversaries do. It includes penetration testing, red teaming, and social engineering exercises, and it is most effective when findings are tied to remediation and governance rather than treated as isolated technical results.
• Red Teaming: Red teaming is structured adversarial testing used to find how an AI system fails under realistic misuse or attack conditions. In AI security, it is a discovery method, not a proof of safety, because probabilistic behaviour and changing models prevent any lasting guarantee.
• Agentic AI attack surface: The set of AI workloads, tools, prompts, and connected services that can be influenced or abused at runtime. It includes not only the model itself but also the identities and integrations that let the system act. For governance, the surface is defined by behaviour as much as by deployment.
• Business Context: Business context is the interpretive layer that explains what a dataset means, who owns it, how trustworthy it is and where it came from. In governance programmes, it turns raw metadata into something practitioners can use for accountability, access decisions and audit evidence.

What's in the full article

Xbow's full analysis covers the operational detail this post intentionally leaves for the source:

• How the vendor frames AI-assisted pentesting workflows for different offensive security tasks.
• Examples of where AI testing is positioned to reduce manual effort across reconnaissance, red teaming, and reporting.
• The vendor's explanation of current limitations in business-context reasoning and human-behaviour interpretation.
• The source's own framing of how AI changes the economics of offensive security testing.

👉 Xbow's full post covers the benefits, limitations, and testing trends in more operational detail.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through the NHI Foundation Level course, the industry's only accredited NHI security programme. It is designed for practitioners who need to connect runtime access decisions to broader identity governance.