The value of a penetration test depends far more on who actually tests, for how many days, and how honestly they report than on the brand or badge on the proposal. Buyers often compare quotes on price and accreditation logos, then receive a report that reads like scanner output, findings that cannot be reproduced, or a retest that was never in the contract. This vendor-neutral guide covers choosing a testing provider, whether a consultancy, a specialist firm or a penetration testing as a service (PTaaS) platform: what to define first, how to read accreditation schemes and tester qualifications, how to compare quotes, which contract and data handling terms to insist on, and how to run a pilot engagement. It is for security managers, procurement teams and assurance leads. What a test is and how test types differ is covered in our Penetration Testing Guide; agreeing the scope of a single engagement is covered in our companion Penetration Test Scoping Checklist.
Key takeaways
- Buy people, not a logo. Company accreditation shows process; it does not tell you who will test you. Ask for the named team, their qualifications and their share of the tester days.
- Know which scheme you actually need. UK public sector and critical national infrastructure work may call for NCSC CHECK; most other buyers can use any qualified, independent provider, with CREST or similar accreditation as one piece of evidence.
- Make quotes comparable. Ask every bidder for tester days by grade, test type and role, with reporting and retest days shown separately. A much cheaper quote often means fewer manual days.
- The report is the product. Request a redacted sample before you shortlist, and score it on reproducibility, risk rating rationale and remediation detail.
- The provider becomes a custodian of your weaknesses. Credentials, findings and evidence sit in its systems, so data handling, deletion and the security of its own platform belong in the contract.
Define requirements
Write these down before you invite bids. They decide which providers can qualify at all and stop a scoping call from turning into a sales pitch.
- Driver: PCI DSS v4.0.1 11.4.2 and 11.4.3 require internal and external testing at least every 12 months and after significant change, by a qualified internal resource or external third party with organisational independence. The PCI SSC penetration testing supplement says this means the tester "must be organizationally separate from the management of the target systems". CIS Controls v8.1 Safeguard 18.2 says external tests "must be conducted through a qualified party". Customer contracts, ISO/IEC 27001 surveillance and go-live sign-off are the other common drivers.
- Scheme requirement: CHECK is, in the NCSC's words, "the scheme under which NCSC assured companies can conduct authorised penetration tests" of public sector and critical national infrastructure systems. The NCSC states that for other organisations testing "does not need to be conducted by a CHECK service provider".
- Estate and volume: count external IPs, internal sites, web applications and their user roles, APIs, mobile apps, cloud accounts and segmentation boundaries, and how many tests a year you expect. One test a year suits a project contract; many releases a year push towards a framework agreement or a PTaaS subscription.
- Clearance and location: whether testers need national security vetting (CHECK team members must hold SC clearance as a minimum), and where testers and evidence may be located.
- Other assurance you already buy: a test is a point-in-time assessment. Continuous validation and prioritisation sit in the Exposure Management and CTEM Guide; outside researchers sit in the Bug Bounty and Vulnerability Disclosure Guide; objective-led adversary emulation sits in the Red Teaming Guide. Do not buy a penetration test to answer a red team question.
- Control obligations: NIST SP 800-53 Rev. 5 CA-8 and CA-8(1) (independent penetration testing agent or team), NIST CSF 2.0 GV.SC-05 (cybersecurity requirements in supplier contracts), GV.SC-06 (due diligence before a supplier relationship) and ID.IM-02 (improvements from tests), ISO/IEC 27001:2022 Annex A 5.20 (security within supplier agreements) and, where testers will see personal data, GDPR Article 28 processor terms.
| Delivery model | Suits | Watch for |
|---|---|---|
| Large consultancy | Multi-country estates, CHECK or regulated work, many concurrent tests, framework agreements | Senior names at the bid, junior testers on the job; subcontracting to associates |
| Specialist or boutique firm | Deep expertise in one area (applications, cloud, OT, mobile); direct access to the testers | Capacity for urgent retests; continuity if a key tester leaves; insurance limits |
| PTaaS platform | Frequent releases, many web and API targets, findings delivered into engineering tools | How many manual tester hours sit behind each test; how much is automated scanning |
| Internal team | Large organisations with steady demand and deep system knowledge | Independence: PCI DSS requires organisational separation; fresh eyes for annual tests |
Small organisations usually do best with one provider on a fixed-scope contract that includes a retest. Large organisations often run a panel of two or three providers under a framework, allocate tests by specialism and compare results across them.
Evaluation criteria
An illustrative split is 25% team and qualifications, 20% methodology and depth, 20% report quality, 15% data handling and provider security, 10% retest and support, 10% commercials. Agree weights before the first bid arrives.
| Criterion | What good looks like | Weak signs |
|---|---|---|
| Company assurance | Current accreditation for the specific service bought, verified on the scheme's own list: the NCSC's CHECK company list or the CREST Marketplace (CREST says members undergo independent assessment of "business processes, governance, data security practices" and are "subject to continual reassessment") | Accreditation for a different service; logos with no listing; claims of equivalence from a separate regional body (CREST's own buyer guidance warns that CREST ANZ membership alone is not recognised by CREST International) |
| Named team | CVs for the actual testers with qualifications matched to the work. CHECK Team Leaders must hold at least a UK Cyber Security Council "Principal" Security Testing title, Team Members a "Practitioner" title. CREST exams run from CPSA ("an entry level exam") through CRT to CCT INF and CCT APP. Practical exams such as OSCP's "24-hour practical assessment" show hands-on ability | No names before contract; one senior CV for a team; qualifications irrelevant to the test type |
| Methodology | Documented approach aligned to NIST SP 800-115, the OWASP Web Security Testing Guide for applications and PTES; clear split of manual and automated work; explains how business logic and authorisation flaws are tested | "Proprietary methodology" that cannot be shown; days mostly spent running scanners |
| Report quality | Redacted sample with scope, methodology and limitations statements, testing narrative, tools used and clean-up (the PCI SSC report outline); reproducible evidence; "a method of resolving each issue found" (NCSC); severity on CVSS v4.0 or CHECK ratings with rationale | Findings pasted from scanner output; no limitations section; severity with no reasoning |
| Data handling and provider security | Encrypted evidence storage, named access, deletion on a set date with written confirmation, following the collection, storage, transmission and destruction stages in NIST SP 800-115 section 7.4; independent assurance of the provider's own platform | Reports emailed as attachments; evidence kept indefinitely; no answer on where data sits |
| Retest and support | Retest of fixed findings included or priced up front with a validity window; tester available to explain findings to engineers | Retest sold as a new engagement; no access to testers after the report |
| Commercials | Day-by-day breakdown by grade and activity; clear assumptions and exclusions | A single lump sum that does not change when scope does |
The PCI SSC supplement is blunt about qualifications: "Appropriate penetration testing experience and qualifications cannot be met by certifications alone." Pair certifications with the experience questions it suggests, such as whether the tester "performed assessments against organizations of similar size and scope".
Pricing and quotes
Most consultancy quotes are tester days multiplied by a day rate, sometimes packaged as a fixed price. PTaaS platforms commonly sell subscriptions or per-test credits. The cost drivers are the same either way: number of targets and user roles, authenticated versus unauthenticated testing, environment complexity, out-of-hours work, cleared staff, reporting days and retests. To compare quotes, send every bidder the same scope pack and require days per test type, grade per day, reporting and project management days, retest terms and listed assumptions. When two quotes for the same scope differ sharply, compare the manual testing days first and ask which findings the cheaper one would not reach.
Contract and authorisation terms
| Clause | What to require |
|---|---|
| Authorisation | Signed written permission from someone with authority over every in-scope asset, plus third-party consents. Unauthorised access is an offence under section 1 of the UK Computer Misuse Act 1990; in the US the Computer Fraud and Abuse Act applies, and the Department of Justice's 2022 good-faith research policy binds only federal prosecutors. This is not legal advice: have counsel review. |
| Scope and rules of engagement | Incorporated by reference to the signed scope document, with stop conditions and proof-of-concept-only exploitation |
| Data protection | Processor terms where personal data may be seen (GDPR Article 28); evidence location, encryption, retention and deletion certificate |
| Confidentiality and ownership | Report and findings owned by the client; no reuse of identifiable findings in marketing or training material |
| Personnel and subcontracting | Named testers; no subcontracting or substitution without written approval; vetting where required |
| AI-assisted testing | Disclosure of any AI tools used, which data reaches which model, and human approval for actions against targets |
| Liability and insurance | Professional indemnity and liability cover appropriate to the systems tested, with evidence of current policies |
| Retest | Scope, price and validity window for retesting, so the correction and retest cycle in PCI DSS 11.4.4 can be completed |
Where identity fits
A testing provider ends up holding some of your most sensitive identity material: test accounts, API keys and cloud roles issued for authenticated testing, credentials it recovers during the test, and the report that explains how to reuse them. PTaaS platforms add integration tokens into your ticketing and code repositories. The Red Hat Consulting GitLab breach showed the risk: customer engagement reports held by a consultancy exposed tokens and connection strings. Require expiring, named test credentials, rotation of any secret the tester read, scoped and rotated integration tokens, and deletion of evidence on a fixed date. See the Third-Party Access Guide and the Secrets Management Guide.
Questions to ask vendors
- Which named people will test us, what share of the days will each deliver, and which qualifications and comparable engagements does each hold?
- For which services are you currently accredited, under which scheme, and where can we verify that listing ourselves?
- How many of the quoted days are manual testing, and which tools run automatically? How do you test authorisation and business logic flaws that scanners miss?
- Can we see a redacted report for a similar test, including the limitations section and an attack narrative?
- What do testers do when they find a critical issue, evidence of a previous or active compromise, or personal data, and how fast will we hear?
- Where will our evidence and credentials be stored, who can access them, when are they deleted and how do you prove it? What independent assurance covers your own systems?
- Do you use AI tools or agents during testing, what data do they receive and who approves their actions?
- What does a retest cover, how long after the report is it valid and what does it cost?
Red flags
- Unwilling to name testers before signature, or a bid team that will not be the delivery team.
- A quote that does not change when you double the scope, or no breakdown of days.
- Sample reports that are mostly scanner output, with no reproduction steps or limitations section.
- No questions about fragile systems, third-party hosting or cloud provider policies at the scoping stage.
- Willingness to start testing without signed authorisation or before third-party consent arrives.
- Vague answers on where your data goes, or evidence kept indefinitely.
- Pressure to bundle remediation work with the test: a provider that also manages or fixes the target systems is no longer organisationally separate from them, which the PCI SSC supplement expects of a tester.
Proof of concept
For a service, the proof of concept is a short paid pilot plus a structured review of how each shortlisted provider scopes and reports.
- Shortlist from scheme lists: confirm each bidder's accreditation for the service on the NCSC or CREST listing, and collect named CVs and a redacted sample report.
- Run the scoping call as a test: give each bidder the same scope pack and note who asks about fragile systems, third-party permissions, test credentials, windows and data handling.
- Pilot on a known target: commission a small test of a staging application or segment where you already know several weaknesses, including an authorisation flaw a scanner will not find. Measure what was found, what was missed and how many findings could not be reproduced.
- Exercise the rules: check that daily start and stop notices arrived, that the SOC could match activity to the tester's log, and that any critical issue was phoned through as agreed.
- Score the report and the retest: have the engineers who will fix the issues rate clarity, then fix one or two findings and request the retest.
- Roll back: disable test accounts and keys, rotate any secret exposed, remove allowlist entries and obtain written confirmation that evidence has been deleted.
Once a provider is in place, track: share of findings reproduced by your own team, days from test end to report, retest closure rate for Critical and High findings, findings that recur in the next annual test, and test credentials still active after the agreed end date.
How NHI Mgmt Group can help
We provide independent requirements definition, RFP support and evaluation of security testing services, including how test credentials and provider access to your systems are controlled. Browse vendors in our products directory or contact us.
Related NHI Mgmt Group resources: Penetration Testing Guide · Red Teaming Guide · Vulnerability Management Buyer's Guide · Third-Party Risk Management Guide · Application Security Testing Buyer's Guide