Offensive Security Programme Maturity Model: Five Levels

Many organisations buy an annual penetration test, file the report and repeat the same findings the following year, which tests nothing except the budget cycle. This model describes five levels of offensive security programme maturity, from Ad hoc to Optimised, across six dimensions: penetration testing, red teaming, purple teaming, continuous validation, governance and safety, and remediation and learning. Each dimension has observable indicators, self-assessment questions and the steps to move up a level. It is written for heads of offensive security, CISOs, security operations leads and the executives who fund testing. It is about running authorised testing as a programme; the testing disciplines themselves are explained in the Penetration Testing Guide, the Red Teaming Guide and the Purple Teaming Guide, and this page does not repeat them.

Key takeaways

Maturity is measured by what changes after a test. A programme is mature when findings are fixed, detections improve and the same weakness does not appear twice, not when it runs more tests.
Testing should confirm what you already know. The UK NCSC advises that you should largely know what testers will find before they find it; surprises point to gaps in internal vulnerability management.
Each activity answers a different question. Penetration tests find weaknesses in scope, red teams test detection and response against a realistic adversary, purple teams improve detections together, and continuous validation checks controls between exercises.
Level 3 means threat-informed and tracked. At Defined, scope comes from a risk register and threat intelligence, every finding has an owner and deadline, and retests are routine.
Safety and authorisation scale with maturity. Rules of engagement, legal authorisation, deconfliction and handling of credentials given to testers must be stronger as testing becomes more realistic.

The five levels

Level	Name	Description
1	Ad hoc	Testing happens because a customer, auditor or regulator asks for it. Scope is whatever the budget allows, usually an external perimeter or one application. Reports go to one person and findings are tracked informally, if at all. There is no retest and no link to detection.
2	Developing	An annual test plan covers the main external and internal environments and key applications. A testing policy and rules of engagement template exist. Providers are chosen against basic criteria such as recognised accreditation. Findings enter a tracker with severities, and critical items are retested.
3	Defined	Scope is driven by the risk register, asset criticality, major changes and threat intelligence. Testing types are chosen for the question being asked: application, infrastructure, cloud, identity, social engineering. Every finding has an owner and a remediation deadline by severity, and retests are routine. Results feed the SOC, which reviews whether test activity was detected.
4	Managed	Objective-based red team exercises emulate threat actors relevant to the organisation, measuring detection and response as well as prevention. Purple team sessions turn findings into tested detections. Breach and attack simulation validates controls between exercises. Metrics such as time to remediate, repeat findings and detection coverage are reported to leadership.
5	Optimised	Testing is continuous and threat-led: new intelligence on adversary behaviour produces new scenarios within weeks. Validation is automated across the estate and integrated with change, so a new system or control is tested as it goes live. Lessons change architecture and investment, and the programme measures its own effectiveness, for example through regulated threat-led testing where it applies.

The six dimensions

Dimension	Level 2 indicator	Level 3 indicator	Level 4 to 5 indicator
Penetration testing	Annual plan; accredited provider	Risk-driven scope; testing on major change; routine retests	Coverage tracked against the asset inventory; specialist testing (cloud, identity, OT, AI) where risk requires
Red teaming	None, or a one-off exercise	Scenario-based exercise planned with leadership sponsorship	Threat-intelligence-led exercises on critical services, measuring detection and response; regulated schemes where required
Purple teaming	None	Occasional joint sessions after a test	Regular sessions mapped to ATT&CK techniques, each producing tested detections
Continuous validation	Vulnerability scanning only	Periodic control validation for priority techniques	BAS or automated red teaming across environments, results feeding the SOC backlog
Governance and safety	Testing policy; rules of engagement template	Written authorisation per test; deconfliction with the SOC; controlled handling of test credentials and data	Control group for covert exercises; independent oversight; legal review of scenarios
Remediation and learning	Findings tracker	Owners and deadlines by severity; retest evidence	Root-cause analysis; repeat-finding tracking; lessons drive architecture and training

Why these dimensions

NIST SP 800-53 Rev. 5 separates penetration testing (CA-8), independent testing (CA-8(1)) and red team exercises (CA-8(2)) from vulnerability monitoring and scanning (RA-5), and that separation is the core of this model: each activity answers a different question. Purple teaming and continuous validation exist to close the gap between an exercise and day-to-day detection. Governance and remediation decide whether the programme is safe and whether it changes anything. NIST CSF 2.0 outcome ID.IM-02 expects improvements to be identified from security tests and exercises.

Choosing the right activity

Question you need answered	Activity	Typical maturity level where it pays off
Does this application or environment have exploitable weaknesses?	Penetration test	Level 2 and above
Would we detect and stop a realistic adversary reaching a critical asset?	Red team exercise	Level 3 and above, once basics are fixed
Do our detections work for specific techniques, and how do we improve them?	Purple team session	Level 3 to 4
Are our controls still working after this week's changes?	Breach and attack simulation, automated validation	Level 4 and above
Do we meet a regulator's threat-led testing requirement?	Threat-led penetration testing (TLPT) under schemes such as CBEST or TIBER-EU	Level 4 and above, where required

Running a red team at Level 1 usually produces a report saying the team reached everything, which the organisation already knew. Fix the basics first. See the Breach and Attack Simulation (BAS) Guide and the Threat-Led Penetration Testing Guide.

Regulated threat-led testing

For financial services, threat-led schemes set an external benchmark for the upper levels.

CBEST (UK): the Bank of England's intelligence-led testing framework, used with the PRA and FCA, tests important business services. It runs through initiation, threat intelligence, penetration testing and closure phases, typically over 9 to 12 months, with accredited threat intelligence and penetration testing providers who must also be CREST members.
TIBER-EU: the European framework for threat intelligence-based ethical red teaming, updated in 2024 to align with the DORA regulatory technical standards on threat-led penetration testing. Its process includes scoping, threat intelligence, red team testing, a blue team report, a remediation plan and an attestation. The ECB also publishes TIBER-EU purple teaming guidance.
DORA: requires certain EU financial entities to carry out threat-led penetration testing, and TIBER-EU can help meet that requirement. Confirm scope and frequency with your competent authority.

Even outside finance, these schemes are useful models for a Level 4 or 5 red team: separate threat intelligence from testing, keep a small control group, test critical services rather than random systems, and end with a remediation plan that is supervised to closure.

Self-assessment questions

Answer yes only with evidence (a test plan, a tracker export, a detection record). Two or more "no" answers in a dimension place you below the level named.

Penetration testing (Level 3)

Is the annual scope derived from the risk register, asset criticality and planned changes rather than last year's scope?
Do you share your current vulnerability assessment with testers so they can check its accuracy, as the NCSC recommends?
Were more than a few findings in the last test a surprise? If yes, your internal vulnerability management is behind.

Red teaming (Level 4)

Is each exercise built around a named objective (for example reaching payment systems) and threat actors relevant to you?
Do you measure time to detect and time to respond, not only whether the objective was reached?
Is there an executive sponsor and a small control group that knows the exercise is running?

Purple teaming (Level 3 to 4)

Do joint sessions follow each major test, mapped to ATT&CK techniques?
Does every session end with detections written, tested and deployed, with evidence?

Continuous validation (Level 4)

Are priority techniques, such as abuse of valid accounts (T1078) and remote services (T1021), validated automatically at least monthly?
Do validation failures create tickets in the SOC or engineering backlog with owners?

Governance and safety (Level 3)

Does every test have written authorisation from someone with authority over the systems in scope, including third-party and cloud providers where their terms require it?
Are accounts and credentials given to testers created for the test, time-limited, monitored and removed afterwards?
Is there a deconfliction process so the SOC can confirm whether activity is a test?

Remediation and learning (Level 3 to 4)

Does every finding have an owner and a deadline by severity, and is retest evidence recorded before closure?
Do you track repeat findings across tests and fix their root cause?
Are results summarised for leadership with trends, not only individual reports?

Dimension to framework mapping

Dimension	NIST SP 800-53 Rev. 5	NIST CSF 2.0	CIS Controls v8.1
Penetration testing	CA-8 Penetration Testing; CA-8(1) Independent Penetration Testing Agent or Team	ID.RA-01	Control 18 Penetration Testing
Red teaming	CA-8(2) Red Team Exercises	ID.RA-03; ID.IM-02	Control 18
Purple teaming and continuous validation	SI-4 System Monitoring; CA-8	DE.CM-01; ID.IM-02	Controls 13 and 18
Remediation and learning	RA-5 Vulnerability Monitoring and Scanning	ID.RA-01; ID.IM-02	Control 7 Continuous Vulnerability Management

Scoring and evidence

Score each dimension separately and report the profile, not an average: a strong penetration testing practice does not offset the absence of any remediation tracking. A level claimed without evidence should be scored one lower. Useful evidence includes the annual test plan with the risk rationale for each item, the findings tracker export with dates opened, closed and retested, red team reports with detection timelines, purple team session records listing detections created, BAS dashboards with trend data, and the register of test accounts with their removal dates. Repeat the assessment every 12 months, and after a major change such as a cloud migration, an acquisition or a new regulatory testing requirement.

Where identity fits

Most real intrusions use valid accounts, and offensive testing should reflect that: test for exposed secrets in code and pipelines, over-privileged service accounts, weak OAuth consent and dormant third-party access, not only unpatched software. The programme also creates identities of its own. Test accounts, VPN credentials, API keys for BAS agents and accounts in cloud tenants are privileged non-human identities that outlive the test unless someone removes them. Register them, time-limit them and rotate them afterwards. Forgotten test assets have been a real route in: Microsoft reported that Midnight Blizzard password-sprayed a legacy non-production test tenant account without MFA, then abused a legacy test OAuth application with elevated access. See the Microsoft Midnight Blizzard breach and the Red Teaming AI Agents for Identity Abuse guide.

Moving up a level

Level 1 to 2: make it a plan. Write a testing policy and rules of engagement template. Agree an annual plan covering external, internal and key applications. Choose providers against recognised accreditation. Put every finding into a tracker. See the Penetration Testing Policy Template.
Level 2 to 3: make it risk-driven and tracked. Derive scope from risk and change. Add testing types the estate needs (cloud, identity, social engineering). Set remediation deadlines by severity and require retests. Brief the SOC after each test on what it should have seen.
Level 3 to 4: test detection and response. Run an objective-based red team with an executive sponsor and control group, using the Red Team Rules of Engagement Template. Follow each exercise with purple team sessions using the Purple Team Exercise Plan Template. Introduce automated validation for priority techniques. Report metrics to leadership.
Level 4 to 5: make it continuous. Turn new threat intelligence into scenarios within weeks. Integrate validation with change so new systems are tested at go-live. Use root-cause trends to change architecture and training. Where regulation applies, meet threat-led testing requirements as part of normal operation.

Small organisations should aim for a solid Level 3: a risk-driven annual test, retests and tracked remediation. Red teams and continuous validation pay off when there is a SOC able to act on results. Larger and regulated organisations should plan for Level 4, and Level 5 where they face capable, persistent adversaries.

Metrics that show maturity

Metric	What it shows	Level 3 to 4 target
Median time to remediate by severity	Whether findings get fixed	Within policy deadlines
Repeat findings across consecutive tests	Whether root causes are fixed	Falling every year
Critical assets tested in the last 12 months	Coverage	100% of crown jewels
Red team actions detected and time to respond	Detection and response effectiveness	Rising detection rate; falling response time
Detections created from purple team sessions	Learning converted into capability	Every session produces tested detections
Test accounts and credentials removed after tests	Programme safety	100% within the agreed window

Common failure patterns

Testing for the certificate. Scope is set to pass an audit, so the riskiest systems are never tested.
Reports that change nothing. Findings are not owned or retested, so the same issues appear year after year.
Red team too early. An expensive exercise confirms that basics are missing, which a penetration test would have shown for less.
The SOC left out. Nobody checks whether test activity was detected, so the most valuable lesson is lost.
Test credentials left behind. Accounts, keys and implants created for testing remain after the engagement.
Unclear authorisation. Testing of cloud, SaaS or third-party systems without the required permission creates legal and contractual risk.

Practitioner checklist

Publish a testing policy, rules of engagement and an authorisation process.
Derive the annual test scope from the risk register, asset criticality and planned changes.
Choose the activity for the question: pen test, red team, purple team or automated validation.
Give every finding an owner, a deadline by severity and retest evidence before closure.
Debrief the SOC after every test and turn gaps into tested detections.
Register, time-limit and remove every account and credential created for testing.
Track repeat findings and fix root causes.
Report programme metrics to leadership at least quarterly.

Offensive Security Programme Maturity Model: Five Levels From Annual Pen Test to Continuous Validation