Continuous offensive security testing needs proof, not just cadence

At a glance

What this is: This is an opinion-led analysis of continuous offensive security testing that argues COST only works when it proves exploitation, models attack chains, and governs autonomous execution.

Why it matters: It matters to IAM and security practitioners because offensive testing increasingly intersects with credentials, access boundaries, auditability, and controlled autonomy across hybrid estates.

By the numbers:

• The typical gap between tests is around 365 days, while new CVEs get weaponized in roughly three days.
• Most programs test roughly 20% of the attack surface, leaving peripheral assets and shadow apps under-tested.
• DAST scanners routinely run 40-70% false positive rates, which pushes teams back into triage instead of remediation.

👉 Read FireCompass's analysis of continuous offensive security testing and governance

Context

Continuous offensive security testing is the attempt to run attack validation on an ongoing basis instead of relying on periodic penetration tests. The problem is that many programmes still measure activity by cadence rather than by whether the testing proves exploitability, coverage, and safe execution across the environments that actually change.

For IAM and NHI teams, the governance question is straightforward: can offensive testing reach the same trust boundaries that attackers target, including credentials, service accounts, and app-to-app access paths? In practice, the control gap is usually not the test itself, but whether the test can be trusted, repeated, and constrained enough to be operationally useful. That is the common starting position in mature programmes, but it is still atypical outside advanced security teams.

Key questions

Q: What breaks when offensive security is limited to annual testing cycles?

A: The evidence window is too short and too stale. Environments change, APIs move, identities are added, and AI-enabled workflows expand before the next assessment. That leaves teams with findings that may be true but no longer operationally useful, and it pushes remediation into a separate process that often loses context.

Q: Why do proof-of-exploit requirements matter in offensive testing?

A: Proof-of-exploit matters because findings without reproducible evidence create noise, not risk reduction. Security teams need the exact request, response, and steps that show a control failed. That lets developers verify the issue, prioritise fixes, and trust the output enough to operationalise it.

Q: How do security teams know if autonomous testing is working?

A: Look for fewer disputed findings, faster triage, and a higher percentage of issues that map to real attack paths. If the output still requires extensive manual cleanup or generates findings with no ownership and no exploit narrative, the system is adding speed without improving decision quality.

Q: How should teams evaluate continuous offensive security testing against scanners?

A: Teams should evaluate whether the tool models attack chains, not just isolated vulnerabilities. A scanner can enumerate issues, but COST should show how a disclosure becomes credential access, how access becomes lateral movement, and where the chain can be halted. That is the operational difference that matters.

Technical breakdown

Why cadence alone does not make testing continuous

Continuous offensive security testing is not defined by how often a scan runs. It is defined by whether the testing loop keeps pace with change in assets, code, credentials, and exposed services. If a programme replays a quarterly pen test against largely the same scope, it is only increasing frequency, not improving signal. Real continuity requires delta-aware testing, on-demand execution, and validation that follows deployment velocity instead of calendar rhythm. That is especially important when attack surface changes faster than test cycles, because stale scope creates blind spots even when reports arrive more often.

Practical implication: align offensive testing triggers to release and exposure change events, not calendar intervals.

Why proof-of-exploit matters more than finding volume

A finding without proof is only a hypothesis. In offensive testing, proof means the request, the response, and reproducible steps that show a control actually failed under test conditions. Without that evidence, teams spend time triaging noise, and the output loses credibility with developers, risk teams, and auditors. This is why validation discipline matters more than raw scanner breadth. If the system cannot re-execute candidates against the live target and discard what does not exploit, it is producing a report, not security evidence.

Practical implication: require reproducible exploit evidence before a finding is allowed into remediation workflows.

How autonomous execution changes the control model

LLMs can help plan attack paths, but they do not safely execute them. Autonomous offensive testing needs a separate runtime that manages state, credentials, target scope, action gating, and stop conditions. That runtime becomes the control plane for safety, because the model itself is non-deterministic while execution against production must be deterministic. This is where governance moves from policy language to enforced guardrails, including allowlists, rate limits, kill switches, and audit logs. Without that separation, autonomy creates uncontrolled interaction with live assets rather than controlled testing.

Practical implication: separate reasoning from execution and enforce deterministic guardrails before any live action is taken.

Threat narrative

Attacker objective: The attacker objective is to turn a small exposure into a validated path to deeper application or network access without being detected during isolated testing.

1. Entry begins with exposed or under-scoped assets that point-in-time testing often misses, especially peripheral services, shadow applications, and API endpoints buried in application code.
2. Escalation happens when a low-severity issue or disclosure is chained into a working exploit path, then reused across a broader environment instead of being viewed as an isolated finding.
3. Impact occurs when the programme misses the end-to-end attack path, leaving credential abuse, lateral movement, or application compromise untested and therefore ungoverned.

NHI Mgmt Group analysis

Cadence is not continuity: The market still overuses the word continuous to describe faster versions of periodic testing. That is a governance error, not just a terminology issue, because the real requirement is coverage that tracks change across assets, access paths, and exposure windows. In NIST-CSF terms, the control question is whether the organisation can identify and protect what changed before the next exploit window opens. Practitioners should treat cadence as a delivery detail, not the definition of the category.

Proof of exploitation is the dividing line between signal and noise: Offensive testing only becomes decision-grade when it demonstrates a live exploit path that another practitioner can reproduce. That is where this topic intersects with identity governance, because credential abuse and app-to-app trust are often the easiest way to turn a maybe into a breach. The named concept here is exploitability evidence debt: the backlog created when tools produce findings that cannot be independently verified. Teams should not let that debt accumulate.

Autonomy without a control plane is just uncontrolled access: The hard problem is not whether an AI can suggest a path, but whether the execution layer can prevent it from touching the wrong target, using the wrong credential, or modifying the wrong asset. That is why this category now overlaps with NHI governance, because machine-driven actions need identity-scoped guardrails just like human operators do. OWASP-NHI and NIST SP 800-53 both matter here, especially where action scope and auditability must be enforced. Practitioners should design for governed autonomy, not impressive demos.

Attack-chain modelling is where COST becomes materially different from scanners: The article’s strongest claim is that attackers do not stop at single findings, and neither should defenders. That shifts the category toward adversarial path validation, where the value lies in chaining weak signals into an operationally relevant route. MITRE ATT&CK is the right lens for that work because it ties findings to tactics such as credential access and lateral movement. Teams should judge tools by whether they can model the chain, not just enumerate flaws.

What this signals

Exploitability evidence debt: COST programmes will increasingly be judged by whether they reduce the backlog of unverified findings and stale assumptions, not by how many assets they scan. As AI-driven testing spreads, control owners will need to prove that output is reproducible, scoped, and tied to real change rather than static reports. Practitioners should expect greater demand for audit-ready evidence and tighter links between testing, identity controls, and remediation workflows.

The governance pressure will also move upstream into access design. If offensive testing can safely exercise service accounts, tokens, and app-to-app trust paths, teams will start treating those identity paths as first-class test objects rather than invisible plumbing. That shifts programme attention toward better credential scoping, cleaner audit trails, and stronger links between offensive security and identity governance.

For practitioners

• Define continuous as change-driven coverage Tie offensive testing to release events, new CVEs, exposed assets, and identity changes so the programme tests what actually moved.
• Require reproducible proof for every finding Reject any output that does not include a working request, response evidence, and step-by-step reproduction against the live target.
• Separate AI planning from execution runtime Place scope checks, rate limits, kill switches, and audit logging between the model and any live target so autonomy remains constrained.
• Model attack chains, not isolated issues Validate whether a disclosure, credential, and lateral movement path can be chained end to end, because isolated findings miss the breach route.

Key takeaways

• Continuous offensive security testing only works when it follows change, proves exploitation, and models real attack chains.
• The category’s credibility depends on replacing noisy findings with reproducible evidence and controlled execution.
• As offensive testing becomes more autonomous, identity governance and execution guardrails become part of the control plane.

Key terms

• Continuous offensive testing: A defensive approach that uses attacker-like testing on an ongoing basis rather than on a fixed schedule. It focuses on chained findings, live exposure, and validation of real exploit paths, not just the presence of isolated vulnerabilities.
• Exploitability proof: Exploitability proof is evidence that a vulnerability can or cannot be turned into a working attack in a specific environment. It goes beyond severity scores by testing real paths, privileges, configurations, and dependencies that determine whether an attacker can achieve impact.
• Attack Chain: An attack chain is a sequence of prompts, observations, and tool calls that moves an AI agent from a benign starting point to a harmful result. In agent security, the chain matters more than any single prompt because real risk often emerges only when actions accumulate across steps.
• Execution Runtime: The control layer that carries out model-generated actions in a bounded, deterministic way. It manages scope, credentials, state, and safety checks so that AI-assisted planning does not directly interact with production systems without governance.

What's in the full article

FireCompass's full blog covers the operational detail this post intentionally leaves for the source:

• Step-by-step explanation of the COST execution model, including how validation runs move from planning to safe live testing.
• Detailed discussion of governance controls such as scope enforcement, kill switches, and credential scoping for autonomous testing.
• Performance evidence from a large enterprise deployment, including coverage expansion, false positive reduction, and cost comparisons.
• Examples of how the platform chains findings into attack paths across applications, services, and network boundaries.

👉 The full FireCompass post covers the COST execution model, proof requirements, and governance controls in more detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity control discipline to broader security operations.