Continuous threat exposure management (CTEM) is a standing programme and a penetration test is a bounded engagement, so asking which one to choose is the wrong question: the real decisions are where penetration tests sit inside a CTEM programme, what they should be scoped to prove, and how often they still need to run. Organisations adopting CTEM often expect it to replace their annual test, while auditors and regulators keep asking for test reports. Both positions miss how the two fit together. This guide compares them as operating models, explains the validation stage of CTEM and the testing methods that feed it, sets out where regulation still requires a penetration test, and gives scoping, cadence and budget guidance. It is written for security leaders, offensive security managers and exposure management owners. For the CTEM cycle itself see the Exposure Management and CTEM Guide; for how CTEM changes vulnerability management see CTEM vs Vulnerability Management.
Key takeaways
- Programme versus engagement. CTEM is a repeating cycle of scoping, discovery, prioritisation, validation and mobilisation; a penetration test is a time-boxed, rules-of-engagement-bound exercise that produces a report.
- Penetration testing lives in the validation stage. Gartner describes validation as confirming that an exposure can be exploited, analysing attack paths and checking whether the response is fast enough; human-led testing is one of several ways to do that.
- CTEM does not remove compliance testing. FedRAMP, DORA and CIS Controls v8.1 all name penetration testing or threat-led testing with stated cadences; a CTEM dashboard does not satisfy those requirements on its own.
- CTEM makes tests better aimed. The NCSC says penetration testing should be a way of gaining assurance, "not as a primary method for identifying vulnerabilities"; CTEM supplies the vulnerability picture that a good test then checks.
- Split the budget by question. Automated validation answers "do our controls still work?" continuously; human testers answer "what did we not think of?" periodically; red teams answer "can we detect and respond?" rarely and deeply.
The short answer
Keep penetration testing and run it inside CTEM rather than beside it. Use the CTEM cycle to decide what each test is for: scope tests at the exposures and attack paths your programme ranks highest, at new or changed systems, and at areas where automated validation cannot reach, such as business logic, chained identity abuse and multi-tenant isolation. Keep the regulated tests at the cadence the rule requires, and feed their findings back into the CTEM backlog so that they are tracked, owned and retested like any other exposure. Small organisations without a CTEM programme should still test at least annually and use the report to start a basic exposure backlog; large organisations should expect more, smaller, targeted tests and fewer broad annual ones.
Side by side
| Dimension | CTEM | Penetration testing |
|---|---|---|
| What it is | An operating programme with a repeating five-stage cycle | An engagement: scoped, time-boxed testing under rules of engagement |
| Question answered | Which exposures matter most to the business right now, and are they being reduced? | Can a skilled tester, within this scope and time, compromise this system, and how? |
| Time model | Continuous, with cycles tied to business scopes | Point in time; NCSC notes it validates only against known issues on the day of testing |
| Output | A ranked, owned and tracked exposure backlog with trend metrics | A report: findings, risk ratings, evidence and remediation advice |
| Who runs it | Internal exposure management function across security, IT and business owners | Internal offensive team or external testers; independent assessors where a rule requires it |
| Main strength | Coverage, prioritisation and follow-through | Human creativity, chaining and proof of real impact |
| Main weakness | Can be confident about what it models and blind to what it does not | Narrow scope, long gaps between tests and findings that age before they are fixed |
| Control anchors | NIST SP 800-53 RA-3, RA-5, CA-7; CSF 2.0 ID.RA | NIST SP 800-53 CA-8; CIS Control 18; NIST SP 800-115 |
CTEM as an operating model
Threat model. CTEM assumes the attack surface changes faster than any annual exercise can track: new SaaS tenants, cloud resources, identities and code ship weekly, and attackers choose whichever exposure is easiest, not the one in this year's test scope. Gartner frames the programme around five stages and says the goal "is not to fix every single security issue" but to focus on what is most likely to be exploited against high-value assets.
Architecture in words. Business-led scopes (for example, the external attack surface, the payment platform or the identity tier) define what each cycle covers. Discovery pulls in assets, vulnerabilities, misconfigurations and identity exposures. Prioritisation ranks them by exploitability, reachability and business impact. Validation proves which ranked items are real and whether controls and responders would stop them. Mobilisation turns validated items into owned changes with agreed triggers for remediation. Each cycle produces evidence that feeds the next scope.
Failure modes. CTEM programmes fail by confusing discovery volume with progress, by scoring exposures without proving any of them, and by modelling only the asset and edge types their tools collect. A programme that never puts a human tester against its top-ranked paths has no independent check that its ranking reflects how an attacker would actually behave.
Penetration testing as an engagement
Threat model. A penetration test assumes a capable adversary with a defined starting point (external, internal, authenticated user, tenant, mobile client) and asks how far they get within the scope and time allowed. NIST SP 800-115, the technical guide to security testing and assessment (final, September 2008, no revision notice on the NIST page as of November 2026), describes testing as a way to find weaknesses and check compliance, and covers planning, execution and remediation of findings.
Architecture in words. A scoping exercise agrees technical boundaries, test types, timeframes and reporting. Rules of engagement set permitted actions, contacts and escalation for critical findings. Testers combine tooling with manual work, chain weaknesses and document evidence. A report rates each finding and proposes fixes; the organisation decides remediation and, ideally, commissions a retest. The NCSC adds that scoping should include risk owners, technical staff and a test team representative, and that the organisation should share its current vulnerability assessment so testers can check it is accurate and complete.
Failure modes. Tests fail when scope is set by budget instead of risk, when the same perimeter is tested every year while new systems go untested, when findings are accepted at the tester's rating without internal review, and when nobody retests. The NCSC also notes that gaps of a year or more between tests are common, and that test quality depends heavily on tester skill.
Validation methods inside CTEM
Validation is where the two models meet. CTEM does not prescribe one method; each answers a different question and should be chosen per exposure. The Breach and Attack Simulation vs Penetration Testing guide compares automated simulation and human testing in depth; the summary below is about placement inside CTEM.
| Method | Question it answers in CTEM | Typical cadence | Best used for |
|---|---|---|---|
| Automated exploitability checks and reachability analysis | Is this ranked exposure actually reachable and exploitable here? | Continuous | Triage of large backlogs; confirming fixes |
| Breach and attack simulation (BAS) | Do preventive and detective controls still stop or alert on known techniques? | Daily to weekly | Control drift, regression after changes, detection coverage |
| Targeted penetration test | Can a skilled human chain the top-ranked exposures and paths into real impact? | Per cycle, per major change, and at least annually | New systems, business logic, identity chains, tenant isolation |
| Red team or threat-led test | Can a realistic adversary reach objectives without detection, and do responders act? | Every one to three years, or as a regulator sets | Detection and response, crisis decisions, people and process |
| Purple team exercise | Which detections fail, and can they be fixed while the test runs? | Quarterly or after red team findings | Turning findings into tuned detections quickly |
CISA's red team assessment advisory AA24-326A shows why human validation stays in the mix. The team got in through a known, deprioritised flaw on an internet-facing server that still hosted a web shell from an earlier disclosure programme, then used exposed SSH keys, cleartext credentials and unconstrained delegation to reach domain administrator rights. The organisation's tooling raised some alerts that were ignored. The lessons CISA drew included over-reliance on endpoint detection, weak segmentation and leadership accepting risk on a known flaw: exactly the prioritisation and mobilisation decisions a CTEM programme is meant to get right, and exactly what a scanner-only programme would have scored and left. See the Red Teaming Guide and the Purple Teaming Guide.
Where regulation still requires penetration testing
CTEM is a management model, not a compliance artefact. Where a standard or law names penetration testing, the test still has to happen, with the independence and evidence that rule expects. Checked against the official sources in November 2026:
| Source | What it requires | What CTEM can add |
|---|---|---|
| FedRAMP Penetration Test Guidance | Initial test by a third-party assessment organisation (3PAO) no more than six months before the security assessment report; repeat at least every 12 months during continuous monitoring; six mandatory attack vectors including tenant-to-tenant and tenant to CSP management system; rules of engagement approved before testing | Evidence of which exposures changed since the last test, to shape scope within the mandatory vectors |
| DORA, Regulation (EU) 2022/2554 | Article 24(6): appropriate tests at least yearly on all ICT systems and applications supporting critical or important functions; Article 25(1) lists tests including vulnerability assessments and scans, penetration testing and end-to-end testing; Article 26(1): advanced threat-led penetration testing (TLPT) at least every three years for identified entities, with external testers every three tests where internal testers are used | Continuous testing evidence under Article 25, threat intelligence to scope TLPT, tracked remediation |
| Commission Delegated Regulation (EU) 2025/1190 | Regulatory technical standards on TLPT under DORA: criteria for which entities must test, internal tester rules, scope, methodology, results and remediation; in force from 8 July 2025 | A remediation pipeline that can absorb TLPT findings |
| CIS Controls v8.1, Control 18 | 18.1 programme; 18.2 external tests no less than annually; 18.3 remediate findings; 18.4 validate security measures after each test; 18.5 internal tests no less than annually (18.1 to 18.3 from IG2, 18.4 and 18.5 IG3) | Remediation tracking and validation of fixes (18.3 and 18.4) |
| NIST SP 800-53 Rev. 5 | CA-8 Penetration Testing at an organisation-defined frequency on defined systems | Risk-based selection of the systems and frequency |
Other regimes, including card payment standards and sector rules, have their own testing clauses; the Penetration Testing Requirements Map collects them, and the DORA Resilience Testing Guide covers the DORA testing chapter. A practical rule: CTEM can change what a regulated test covers within the rule, never whether it happens or who performs it.
Scoping and cadence with CTEM in place
CTEM changes the way tests are scoped more than how often they run. Use these triggers:
- Regulated baseline: schedule the tests a rule requires first, with the independence it requires, and treat them as fixed points in the CTEM calendar.
- Top-ranked paths: each CTEM cycle, pick the highest-ranked attack paths that automated validation could not fully prove and commission a short, targeted human test against them.
- Change-driven tests: new internet-facing applications, major architecture changes, new identity providers, mergers and new multi-tenant features get a test before or shortly after go-live.
- Blind spot tests: once a year, scope a test at what the CTEM programme does not model well (business logic, SaaS integrations, pipeline identities, physical and social routes where in scope) to check the programme's own coverage.
- Retest on closure: every high or critical finding is retested, by the tester or by automated validation where it can reproduce the issue, before the item is closed.
When scoping, give testers the CTEM view: the ranked backlog, known attack paths and accepted risks. NCSC guidance supports this. The test then answers two questions: what did the tester find, and how accurate was our own exposure picture? A test that finds mostly items already in the backlog confirms the programme works; a test that finds new classes of issue is a signal to widen discovery. See How to Read a Penetration Test Report for reviewing findings.
Where identity fits
Identity is where CTEM and testing disagree most often. CTEM tools increasingly model identity exposures, yet many exploit chains in test reports are identity chains: cached privileged sessions, over-permissioned service accounts, credentials in scripts and shares, and delegation settings. The CISA red team found 61 private SSH keys and a file with cleartext domain credentials on one share. Scope tests to include non-human identities explicitly: service accounts, API keys, pipeline tokens and cloud workload identities, and the systems that store their secrets. Feed every credential found in a test into the CTEM backlog as an identity exposure with rotation, ownership and least privilege actions, not just a "password found" finding. See the Service Account Security Guide and Attack Path Analysis vs Attack Surface Management.
The budget split
There is no standard ratio, and any fixed percentage would be invented. The defensible approach is to fund by the question each activity answers and reallocate as the programme matures:
| Programme maturity | Where the money goes | Why |
|---|---|---|
| No CTEM yet | Mostly human penetration testing plus basic scanning | Tests are the main source of exposure insight; use reports to build the first backlog |
| Early CTEM | Discovery and prioritisation tooling, regulated tests, one targeted test per cycle | Build the backlog and remediation workflow; keep human validation on top items |
| Established CTEM | Continuous automated validation, more frequent short targeted tests, periodic red team or TLPT | Automation covers known techniques; humans focus on novel chains and response |
| Regulated financial entity | Fixed TLPT and annual testing budget first, CTEM tooling and remediation capacity second | Legal obligations set the floor |
One line item is often missing: remediation capacity. Spending on more validation without engineering time to fix findings only produces a longer backlog. See How Much Does a Penetration Test Cost for the testing side.
Worked scenario
A payments company subject to DORA has run one broad annual external and internal test for years and has recently started a CTEM programme scoped first to its external attack surface and identity tier.
- Fixed points: the annual testing required under Article 24 and the three-yearly TLPT under Article 26 are placed in the calendar first.
- CTEM ranking: the first cycle ranks three items highest: an exposed partner API, a pipeline service principal with subscription-wide rights, and a legacy VPN appliance with a known exploited flaw.
- Method choice: the VPN flaw is validated automatically and patched with session termination; BAS confirms detections for the pipeline identity misuse techniques are firing; a five-day targeted human test is commissioned against the partner API and the pipeline path.
- Findings: the test confirms the pipeline path and finds an authorisation flaw in the partner API that no tool had flagged. Both enter the CTEM backlog with owners and due dates.
- Retest and learn: fixes are retested; the API flaw class prompts adding API authorisation checks to discovery. The broad annual test is narrowed to regulated scope, freeing budget for two more targeted tests a year.
The scenario is illustrative. The pattern is typical: CTEM did not reduce testing, it made each test answer a sharper question.
Decision criteria
| If your situation is | Do this |
|---|---|
| Regulated (FedRAMP, DORA, similar) | Keep required tests at required cadence and independence; use CTEM to shape scope within the rule |
| Fast-changing cloud and SaaS estate | More frequent short tests triggered by change and CTEM ranking, plus continuous automated validation |
| Small team, no CTEM | Annual external and internal test; turn the report into an owned backlog; add automated scanning |
| Mature detection and response | Add red team or threat-led testing and purple teaming to validate response, not just prevention |
| Backlog growing faster than fixes | Pause new validation spend; fund remediation and retesting first |
Detection and measurement
- Track test findings in the CTEM backlog, tagged with source (test, BAS, scanner) so you can see which method finds which class of issue.
- Use tests to measure the programme: record whether each test finding was already known to CTEM.
| Metric | Healthy direction |
|---|---|
| Share of penetration test findings already in the CTEM backlog | Rising: the programme sees what testers see |
| Median time from test report to closure of high and critical findings | Falling, with retest evidence for each |
| Share of top-ranked CTEM items validated by any method each cycle | Close to all |
| Repeat findings across consecutive tests | Falling towards zero |
| Share of new internet-facing systems tested before or soon after go-live | Rising |
Framework mapping
| Framework | Reference | Where it applies |
|---|---|---|
| NIST CSF 2.0 | ID.RA-01, ID.RA-05, ID.IM-02 | Identify and validate vulnerabilities; prioritise by likelihood and impact; improvements from security tests and exercises |
| NIST SP 800-53 Rev. 5 | CA-8, CA-2, CA-7, RA-3, RA-5 | Penetration testing, control assessments, continuous monitoring, risk assessment, vulnerability monitoring |
| NIST SP 800-115 | Technical testing and assessment | Planning, conducting and following up tests; FedRAMP rules of engagement reference its Appendix B |
| CIS Controls v8.1 | Control 18 (18.1 to 18.5), Control 7 | Penetration testing programme; continuous vulnerability management that CTEM extends |
| DORA | Articles 24, 25, 26, 27; Delegated Regulation (EU) 2025/1190 | Annual testing, test types, TLPT and tester requirements |
Common failure patterns
- Dropping the annual test because "we do CTEM now". Regulators and customers still ask for test reports, and the programme loses its only independent check.
- Testing the same scope every year. The perimeter that was tested last year is rarely where the new exposure is; use CTEM ranking and change triggers.
- Keeping test findings outside the backlog. PDF reports age in a folder; findings must become owned, tracked exposures with retest evidence (CIS Safeguard 18.3).
- Accepting tester ratings unchanged. The NCSC is clear that risk and fix decisions remain the organisation's; rate findings against business context and the CTEM ranking.
- Accepting risk on known exploitable flaws without impact analysis. CISA's red team entered through exactly such a flaw.
- Funding validation, not remediation. More testing without fixing capacity produces longer reports, not lower risk.
Practitioner checklist
- List every testing obligation (FedRAMP, DORA, contracts, CIS 18 commitments) with cadence and independence requirements.
- Put regulated tests in the CTEM calendar as fixed points.
- Each CTEM cycle, choose a validation method per top-ranked exposure: automated, BAS, targeted test or red team.
- Give testers the CTEM backlog, known paths and accepted risks at scoping.
- Scope non-human identities and their secret stores explicitly in tests.
- Import every finding into the CTEM backlog with owner, due date and source tag.
- Retest every high and critical finding before closure.
- Measure how many test findings CTEM already knew, and widen discovery where it did not.
- Budget remediation capacity alongside validation spend.
Standards and references
- NIST: SP 800-115, Technical Guide to Information Security Testing and Assessment
- NCSC: Penetration testing
- FedRAMP: Penetration Test Guidance
- EUR-Lex: Regulation (EU) 2022/2554 (DORA)
- EUR-Lex: Commission Delegated Regulation (EU) 2025/1190 on threat-led penetration testing
- CIS: Controls v8.1 Control 18, Penetration Testing
- CISA: Enhancing Cyber Resilience, Insights from CISA Red Team Assessment (AA24-326A)
- Gartner: How to Manage Cybersecurity Threats, Not Episodes
Related NHI Mgmt Group resources: Exposure Management and CTEM Guide, CTEM vs Vulnerability Management, Breach and Attack Simulation vs Penetration Testing, Penetration Testing Guide, Threat-Led Penetration Testing Guide.