CIS Control 18 Penetration Testing Guide: Safeguards, Implementation Groups and Evidence

Most organisations that commission an annual penetration test still struggle with CIS Control 18, because the Control measures the programme around the test, not the test itself. CIS Control 18 (Penetration Testing) in CIS Controls v8.1 asks for a documented programme, external and internal tests at least annually, tracked remediation of findings and a deliberate check that defences were improved afterwards. This guide covers the five Safeguards, which Implementation Group (IG) each belongs to, how CIS assesses them, the identity questions the Control raises and the evidence an assessor expects to see. It is written for security managers, GRC teams and assessors. It deliberately does not cover how to test: for methodologies, test types and provider selection, see the Penetration Testing Guide.

Key takeaways

• There is no IG1 Safeguard in Control 18. Safeguards 18.1 to 18.3 start at IG2, and 18.4 and 18.5 apply only at IG3, so small organisations meet the intent through Controls 7 and 16 first.
• The programme document is the anchor. The CIS assessment specification fails 18.1 if the programme document does not exist or has not been updated for more than twelve months, and scores it on five required components.
• Annual is the floor, not the target. External (18.2) and internal (18.5) tests fail assessment when the most recent report is more than twelve months old; the programme should add triggers for significant change.
• Remediation is measured across two reports. Safeguard 18.3 is assessed by comparing the latest external report with the one before it, so findings that reappear are visible to any assessor.
• Test identities are part of the attack surface. Accounts, keys and roles issued to testers are real non-human and human identities that need an owner, an expiry and verified removal, and identity systems belong in scope.

What it is and who it applies to

CIS states the purpose of Control 18 as testing the effectiveness and resiliency of enterprise assets by identifying and exploiting weaknesses in controls across people, processes and technology, and simulating the objectives and actions of an attacker. The CIS Control text separates this from Control 7: vulnerability testing checks for the presence of known weaknesses and stops there, while penetration testing exploits them to see how far an attacker could get and which business processes or data would be affected. It also distinguishes red team exercises, which simulate the tactics of a specific adversary or class of adversary, from a scoped penetration test.

CIS lists three common purposes for a test: a demonstration to convince decision makers, verification that defences operate correctly, and validation that the right defences were built in the first place. Control 18 is the governance wrapper that makes those outcomes repeatable rather than a single annual report.

• IG1: no Control 18 Safeguards. CIS treats IG1 as essential cyber hygiene, and an organisation without mature asset inventories, patching and configuration control (Controls 1 to 7) gains little from a penetration test that would only rediscover known gaps. See the CIS Controls Guide for how the Implementation Groups are built.
• IG2: 18.1 (programme), 18.2 (external tests) and 18.3 (remediation). This is the realistic target for organisations with dedicated security staff or a managed provider, and the level many customer questionnaires assume.
• IG3: adds 18.4 (validate security measures after each test) and 18.5 (internal tests). IG3 organisations are expected to defend against targeted attackers, so they test from inside the network and feed what testers did back into detection engineering.

CIS Controls v8.1 was released in June 2024 as an iterative update to v8. It added a Governance security function, revised asset classes and expanded some Safeguard descriptions, and CIS states that no Implementation Group assignments changed. The Control 18 Safeguard numbers and titles are the same in v8 and v8.1. The asset type and security function values in the table below come from the CIS Controls Assessment Specification, which is written against v8; if you report those attributes, take them from the v8.1 spreadsheet.

Structure and key requirements

What the programme document must contain

Safeguard 18.1 names the characteristics of the programme. Treat them as the table of contents of the document:

• Scope: the classes of target, which CIS lists as network, web application, application programming interface (API), hosted services and physical premise controls. Name the systems by tier or business service, not by IP range alone, and say explicitly whether identity providers, cloud control planes and SaaS tenants are included.
• Frequency: the minimum (annual for each of external and internal) plus event triggers such as a major release, a cloud migration, an acquisition or a change of identity platform.
• Limitations: acceptable hours, excluded attack types (for example denial of service), production restrictions and third-party systems that must not be touched.
• Point of contact: named internal contacts reachable during testing, with an escalation route and authority to stop a test.
• Remediation: how findings are routed internally, who owns them and the time allowed by severity.
• Retrospective requirements: the review after each engagement, including what defences detected and what the programme will change.

The assessment specification scores completeness on five components (scope, frequency, point of contact, remediation and retrospective requirements) and computes completeness as the number present divided by five. Limitations appear in the Safeguard description, so include them too; the rules of engagement for each engagement then inherit them. The Penetration Test Scoping Checklist and the Red Team Rules of Engagement Template can be used as the engagement-level documents beneath the programme.

What "qualified party" and "clear or opaque box" mean for governance

Safeguard 18.2 says external testing requires specialised skills and experience and must be conducted through a qualified party. CIS does not name a certification scheme, so the programme should define what qualified means for the organisation: for example, an accredited company and named individually certified testers, or an internal team that is organisationally independent of the systems tested. NIST SP 800-53 expresses the same independence requirement as CA-8(1) Independent Penetration Testing Agent or Team. Both 18.2 and 18.5 accept clear box (tester has internal knowledge) or opaque box (tester has none); the programme should record which was used and why, because it changes what an assessor can conclude from a clean result.

Identity and NHI requirements

Control 18 does not mention identities explicitly, but three of its requirements turn on them.

• Test credentials are issued identities: clear box and assumed-breach tests run on accounts, API keys, cloud roles and VPN profiles created for the engagement. The programme should require named, tagged accounts per tester, delivery through a secrets vault rather than email, an expiry matching the test window and verified removal recorded in the closure report. Leftover test keys are a classic orphaned non-human identity. See the Service Account Security Guide.
• Identity systems belong in scope: scope that excludes the identity provider, privileged access tooling, CI/CD runners and cloud workload roles leaves out the paths attackers use most. Name them in the 18.1 scope statement so their exclusion has to be a recorded decision.
• Identity findings need identity owners: hard-coded secrets, over-privileged service accounts and long-lived tokens are frequent findings, and their remediation sits with platform and application owners rather than infrastructure teams. Route them through the 18.3 process to the owner recorded in the service account inventory (Control 5).
• Agentic and automated testing: where a provider uses AI agents or automated tooling, the programme should state which data may reach which models, who approves autonomous actions and how scope is enforced. Misconfigured AI-driven security tests have already reached real systems outside their intended scope, as the Anthropic Claude Evaluation Incidents 2026 show.

Implementation steps

1. Confirm readiness. Check that Controls 1, 2, 4 and 7 are working at IG1: an asset inventory exists, configuration baselines are applied and authenticated scanning runs. A test commissioned before this point mostly reproduces the scanner output.
2. Write and approve the programme (18.1). Cover the six characteristics above, assign an owner, have it approved by the risk owner and set a review date no more than twelve months out. Record the version and approval date, because the assessment uses the date of last update.
3. Define the remediation policy (feeds 18.3). Set time limits by severity, the route for accepting risk (who may sign, for how long) and a rule that findings close only after retest. Reuse the Control 7.2 remediation process rather than creating a parallel one. See the CIS Control 7 Continuous Vulnerability Management Guide.
4. Procure and authorise. Define qualification criteria, confirm written authorisation from every asset owner and from cloud or hosting providers whose policies apply, and agree stop conditions and emergency contacts. See the Penetration Testing Provider Buyer's Guide.
5. Run the first external test (18.2). Pilot on lower-risk services or a staging copy for fragile systems, with owners on call. Agree that critical findings are reported immediately, not held for the final report.
6. Remediate and retest (18.3). Load every finding into the ticketing system with an owner and due date, then retest. Record risk acceptances against the policy with an expiry.
7. Add internal testing and validation (18.5 and 18.4, IG3). Hold a joint review with the detection team after each test, compare tester activity with the alerts raised, and raise change requests for detections, firewall rules or EDR policies that should have fired.
8. Run the retrospective and close down. Confirm removal of all test identities and test data, update the programme with lessons learned and set the next test date.

Rolling back: stop conditions in the programme are the governance equivalent of a rollback. If testing causes instability, the named contact halts it, the provider records the state, and the change is reverted by the system owner under normal change control. Record the event in the retrospective and adjust limitations before resuming.

Small versus large organisations

Between annual tests, breach and attack simulation and continuous automated red teaming can show whether controls still work, but they do not replace a test by a qualified party for 18.2 or 18.5. See the Breach and Attack Simulation (BAS) Guide and the Red Teaming Guide.

Evidence and assessment

The CIS Controls Assessment Specification defines inputs and measures for each Safeguard. It refers to three governance artefacts: GV53 (penetration testing programme documentation), GV54 (most recent external penetration report) and GV55 (most recent internal penetration report).

Two practical consequences follow. First, 18.3 is assessed across two consecutive external reports, so keep the previous report and make sure findings carry stable identifiers between providers. Second, the 18.3 compliance formula in the specification as published adds unremediated in-scope findings to the numerator; read it with the measure definitions and agree with your assessor how it will be interpreted, rather than relying on a single score.

Metrics that show it is working

• Repeat finding rate: findings in the latest report that also appeared in the previous one (the core of the 18.3 measure). It should fall over successive tests.
• Time to remediate by severity: median days from report to verified fix, against the policy limits.
• First retest pass rate: share of fixes that pass the first retest; a low rate signals fixes closed on ticket status alone.
• Detection rate: share of tester actions that raised an alert or were investigated, and the 18.4 ratio of required detection changes completed.
• Critical service coverage: share of tier 1 services tested within the last twelve months.
• Test identity hygiene: test accounts, keys and roles still active after engagement close (target zero).

Mapping to other frameworks

These are NHI Mgmt Group mappings by outcome, not the official CIS mapping; the CIS Controls Navigator publishes CIS's own mappings for v8.1. Payment card environments also have requirement 11.4 of PCI DSS v4.0.1, which sets its own methodology, annual and after-change frequency and retesting rules; if you are in scope for both, write the 18.1 programme to the stricter of the two.

Common pitfalls

• Scope drawn to pass the audit: testing only the systems named in a contract while excluding the identity provider, jump hosts or cloud roles that reach them. The Capital One Breach 2019 turned on an internet-facing firewall whose cloud role could read customer data in S3, exactly the kind of chained path a narrow scope never examines.
• A scan sold as a test: an automated scan report with no exploitation or attack narrative does not meet the CIS distinction between Controls 7 and 18, and an assessor reading the report will see it.
• Findings closed without retest: ticket status changes but the next report shows the same issue, which the 18.3 measure exposes directly.
• Stale programme document: the test happens every year but the programme has not been reviewed for two, which fails 18.1 regardless of test quality.
• Test identities left behind: VPN accounts, cloud keys and admin roles created for testers remain active for months after the report.
• No feedback into detection: testers move through the network unseen and nobody asks why, so IG3 organisations fail 18.4 and repeat the blind spot.

Practitioner checklist

• Confirm Controls 1 to 7 are operating at IG1 before commissioning tests for Control 18.
• Publish a penetration testing programme covering scope, frequency, limitations, contacts, remediation and retrospectives, approved and dated.
• Review the programme at least every twelve months and after significant change.
• Name identity providers, privileged access tools, CI/CD and cloud control planes in scope, or record why they are excluded.
• Define what a qualified party means and keep evidence of tester qualification and independence.
• Run an external test at least annually (IG2) and an internal test at least annually (IG3).
• Track every finding to a verified retest or a signed, time-limited risk acceptance, and keep the previous report for comparison.
• Hold a detection review after each test and raise changes for missed techniques (IG3).
• Issue tester credentials through a vault with expiry, and confirm removal in the closure report.

Standards and references

• CIS: Control 18 Penetration Testing
• CIS: Controls Assessment Specification, Control 18
• CIS: The Ongoing Evolution of the CIS Critical Security Controls
• NIST: SP 800-53 Rev. 5 Security and Privacy Controls
• NIST: Cybersecurity Framework 2.0
• NCSC: Penetration testing guidance

Related NHI Mgmt Group resources: CIS Controls Guide · Penetration Testing Guide · CIS Control 7 Continuous Vulnerability Management Guide · Penetration Test Scoping Checklist · Threat-Led Penetration Testing Guide