Penetration Testing Guide: Types, Scoping and PTaaS

Many penetration tests are bought to satisfy an auditor, scoped too narrowly to find what matters, and closed with a report nobody retests against. A well-run test is one of the few ways to see how weaknesses chain together in your environment, but only if you choose the right type, scope it with the people who own the risk, authorise it properly and act on the findings. This guide covers test types, black, grey and white box approaches, how penetration testing differs from scanning, red teaming and breach and attack simulation, the main methodologies, scoping and legal authorisation, CHECK and CREST, reporting, retesting, frequency, PTaaS and choosing a provider. It is written for security managers and buyers who commission tests, and for the engineers who host and fix them. It does not describe attack techniques or how to use testing tools.

Key takeaways

• A pen test is assurance, not discovery. The NCSC says to use it "not as a primary method for identifying vulnerabilities", and that "you should know what the penetration testers are going to find, before they find it."
• Scope decides value. Involve risk owners, define technical boundaries in writing, and include the identity systems, APIs and cloud control planes attackers actually use.
• Authorisation must be written and complete. Get sign-off from the asset owner, every hosting or SaaS provider whose rules apply, and anyone whose systems sit in the path.
• Test credentials are real identities. Accounts, keys and VPN profiles given to testers need owners, expiry, monitoring and verified removal.
• Retesting closes the loop. PCI DSS 11.4.4 requires exploitable findings to be corrected and retested; build retest into every statement of work.

Pen test, vulnerability scan, red team and BAS

These terms are often used interchangeably by sellers. They answer different questions and need different budgets.

Run scans first and fix the obvious, so testers spend paid days on what scanners cannot find. See the Vulnerability Management Guide. Use BAS to check between tests that controls have not drifted. See the Breach and Attack Simulation (BAS) Guide.

Types of penetration test

Social engineering and physical tests touch people directly; pair them with awareness work rather than naming individuals. See the Security Awareness and Human Risk Management Guide. For financial entities in the EU, threat-led penetration testing under DORA has its own scoping and approval rules. See the DORA Guide.

Black, grey and white box

• Black box (opaque box): testers get only a target name. It mirrors an outside attacker but spends much of the budget on reconnaissance and can miss authenticated functionality.
• Grey box: testers get user-level accounts, network diagrams or API documentation. It is the usual default because it spends test days on depth, and it reflects the realistic case of an attacker with stolen credentials.
• White box (clear box): testers get source code, configuration and architecture access. It finds the most per day and suits critical applications and cloud configuration reviews.

CIS Safeguards 18.2 and 18.5 accept either "a clear box or an opaque box". Choose by the question: "could an outsider get in?" favours black box; "how bad is it if a user account is compromised?" favours grey or white box.

Methodologies

A provider should name the methodology in its proposal and report. PCI DSS 11.4.1 requires a defined penetration testing methodology based on industry-accepted approaches, so "our proprietary approach" with no reference is a warning sign.

Scoping, rules of engagement and legal authorisation

The NCSC advises scoping with "all relevant risk owners", technical staff and a representative of the test team, producing a document stating "the technical boundaries of the test", test types, timeframes in resource days, scenarios, compliance needs and reporting requirements. Share your latest vulnerability assessment with the testers.

• Targets: IP ranges, hostnames, application URLs, cloud account and subscription IDs, and explicitly excluded systems. Check ownership of every IP and domain before it goes in.
• Objectives and scenarios: for example, "from a standard employee laptop, can a tester reach payment data?" Scenarios focus effort better than asset lists alone.
• Timing and windows: testing hours, change freezes, and whether production or a production-like environment is tested.
• Stop conditions: what testers must do on finding a critical issue, evidence of a previous compromise, or personal data; named contacts reachable at short notice, as the NCSC recommends.
• Prohibited actions: typically denial of service, destructive changes, data exfiltration beyond agreed proof files, and pivoting into third-party systems.
• Data handling: how test evidence is encrypted, where it is stored, how long it is kept and how it is destroyed.

Legal authorisation

Unauthorised access is an offence in most jurisdictions, including under the UK Computer Misuse Act 1990, so the contract and a signed authorisation letter must come from someone with authority over every in-scope asset. Third-party hosted assets need their own permission:

• AWS: customers may test listed services such as EC2, RDS, CloudFront, Aurora and Lambda without prior approval, but command and control testing needs approval, and denial of service, request flooding, Route 53 zone walking, S3 bucket takeover and subdomain takeover are prohibited. Simulated events need a request at least two weeks in advance.
• Microsoft cloud: testing "must be performed within your own tenant or assets for which you have explicit authorization"; denial of service and phishing of Microsoft employees are prohibited, and vulnerabilities in Microsoft services must be reported immediately.
• SaaS and managed services: check each supplier's terms. Many prohibit testing or require notice; PCI DSS 11.4.7 requires multi-tenant service providers to support customers' external penetration testing.

Where identity fits: credentials given to testers

Grey and white box tests depend on access you hand over: domain user accounts, application accounts per role, API keys, cloud IAM roles, VPN profiles and sometimes privileged accounts for configuration review. These are non-human or temporary identities in all but name, and they often outlive the engagement. The Microsoft Midnight Blizzard breach hinged on a forgotten test tenant and an OAuth app, showing how test identities left in place become an attacker's entry point.

• Issue, do not share: create named, tagged accounts per tester, never shared or reused real staff accounts, with the engagement reference in the description field.
• Expire by default: set account expiry, short-lived tokens and time-bound role assignments that end with the test window. See the JIT Access Guide.
• Deliver secrets securely: share keys and passwords through a vault or one-time secret link, not email or the statement of work. See the Secrets Management Guide.
• Least privilege, then escalation by agreement: start testers with the privilege the scenario assumes; grant higher privileges only for defined configuration reviews and through your privileged access tooling.
• Monitor and verify removal: alert on use of test identities outside the window and confirm deletion of accounts, keys and app registrations in the closure report.
• Treat findings about credentials as priority: hard-coded keys, over-privileged service accounts and reusable tokens found by testers are often the most damaging results. See the Service Account Security Guide.

Choosing a provider: schemes, PTaaS and tooling

• CHECK: the NCSC scheme "under which NCSC assured companies can conduct authorised penetration tests" of public sector and critical national infrastructure systems. The NCSC "recommends that HMG organisations use testers and companies which are part of the CHECK scheme". CHECK reports rate risk as HIGH, MEDIUM, LOW or INFORMATIONAL; CVSS may be added but not substituted.
• CREST: a not-for-profit body that accredits penetration testing companies and certifies individual testers. Accreditation is evidence of company process and staff qualification; ask which certified staff will actually do your work.
• Independence: PCI DSS 11.4 accepts a qualified internal resource or qualified external third party, provided organisational independence of the tester exists.
• Tooling questions: testers commonly use tools such as Nmap, Burp Suite and Metasploit, often on distributions such as Kali Linux or Parrot OS. Ask how tools are licensed, kept updated and configured to avoid destructive modules, and where captured data is stored.
• AI-assisted testing: if a provider uses AI agents or LLMs, ask what data is sent to which model, who approves autonomous actions, and how agent activity is constrained to scope. See the Red Teaming AI Agents for Identity Abuse.

PTaaS platform integrations need tokens into your ticketing and code systems: treat them as non-human identities with scoped, rotated credentials.

Reporting and retesting

The NCSC expects a report to set out the security issues found, "a method of resolving each issue found" and "an opinion on the accuracy of your organisation's vulnerability assessment." A useful report contains:

• Executive summary: business impact in plain language and the attack paths that matter, not counts.
• Scope and coverage: what was and was not tested, test dates, accounts used and any limitations or blocked areas.
• Findings: description, affected assets, evidence, severity with rationale (CVSS v4.0 or the scheme's scale), and specific remediation.
• Attack narratives: how individual findings chained, which shows where one control fix breaks several paths.
• Detection notes: which actions were seen or missed by monitoring, if the SOC was involved.

The NCSC is clear that "risk assessment and decisions on the application of fixes are your responsibility" and should not be "wholly outsourced to the test team." Feed findings into the main remediation queue with owners and SLAs, then retest. PCI DSS 11.4.4 requires exploitable vulnerabilities and security weaknesses found during testing to be corrected and testing repeated to verify the corrections.

Frequency and compliance triggers

Annual is a floor. The NCSC warns that it is "not uncommon for a year or more to elapse between penetration tests". Add event triggers: major releases, new internet-facing services, cloud migrations, mergers, identity platform changes and significant network re-architecture. See the PCI DSS 4.0 Guide.

Running a testing programme

1. Establish the programme: CIS Safeguard 18.1 expects defined scope, frequency, limitations, points of contact, remediation and retrospective requirements. Rank assets by risk and set a multi-year test calendar.
2. Prepare: run authenticated scans and fix known issues; reconcile scope with attack surface data. See the Attack Surface Management Guide.
3. Scope and authorise: statement of work, rules of engagement, signed authorisation, third-party permissions, test identities issued.
4. Pilot carefully: for fragile or first-time targets, test a staging copy first or start production testing in a low-traffic window, with system owners on call.
5. Execute with live escalation: critical findings reported immediately, not held for the report.
6. Stop or roll back: if a service degrades, testers pause on request, record what they ran, and the owner restores from known-good configuration; resume only once the cause is understood.
7. Report, remediate, retest: debrief, assign findings, retest within the agreed window, and update detection rules for techniques the SOC missed.
8. Close out: remove test identities, collect evidence of data destruction and run a retrospective on scope and value.

Small vs large organisations: a small organisation gets most value from one annual grey box external and web application test plus retests, bought with a clear methodology. A large organisation needs a risk-ranked calendar, a framework contract with several providers to avoid single-tester blind spots, PTaaS for high-change applications, and red teaming once detection is mature.

Detection and measurement

Use tests to measure defences as well as find flaws. Give the SOC the test window but not always the detail, and compare what was logged and alerted with the testers' activity timeline. Useful metrics:

• Repeat finding rate: share of findings that appeared in a previous test.
• Time to remediate critical and high findings, against SLA.
• Retest pass rate: share of findings verified fixed at first retest.
• Detection rate: share of tester actions that generated an alert or were seen by the SOC.
• Coverage: share of critical assets tested within the last 12 months and after their last significant change.
• Test identity hygiene: test accounts and keys still active after engagement close (target zero).

Framework mapping

Common failure patterns

• Scope written to pass: the CDE or crown-jewel system tested in isolation while the identity provider and jump hosts that reach it are excluded.
• Scanning sold as a pen test: a branded scanner report with no manual testing, attack narratives or methodology reference.
• Test identities left behind: accounts, app registrations and keys outlive the engagement, as the test tenant did in the Midnight Blizzard breach.
• No retest: findings marked fixed by ticket status, and the same issues reappear next year.
• Missing third-party permission: testing a hosted service against its provider's rules, or an IP range that belongs to someone else.
• Treating the test as the security programme: a year between tests with no scanning, BAS or detection validation in between.

Practitioner checklist

• Run and remediate authenticated vulnerability scans before each test.
• Scope with risk owners and include identity, API and cloud control plane paths.
• Require a named methodology (NIST SP 800-115, PTES, OWASP WSTG or MASVS, OSSTMM) in proposal and report.
• Obtain signed authorisation and check every cloud and SaaS provider's testing rules.
• Issue tester accounts and keys through a vault with expiry, and verify removal at close.
• Agree stop conditions, escalation contacts and immediate reporting of critical findings.
• Test internal and external scope at least annually and after significant change.
• Build retesting into every contract and verify fixes before closing findings.
• Compare tester activity with SOC detections and tune rules after each test.

Standards and references

• NIST: SP 800-115, Technical Guide to Information Security Testing and Assessment
• NCSC: Penetration testing
• NCSC: CHECK penetration testing
• CIS: Control 18, Penetration Testing
• OWASP: Web Security Testing Guide
• OWASP: Mobile Application Security (MASVS and MASTG)
• PTES: Penetration Testing Execution Standard
• ISECOM: OSSTMM

Related NHI Mgmt Group resources: Breach and Attack Simulation (BAS) Guide · Vulnerability Management Guide · Attack Surface Management Guide · PCI DSS 4.0 Guide · Service Account Security Guide